Glob pattern
A glob pattern is a wildcard-based string pattern, using characters like `*` and `?`, used to match multiple filenames or paths at once instead of listing them individually.
Overview
Glob patterns are a shorthand syntax for matching sets of file paths using wildcard characters, originally from Unix shells and now used across most programming languages and tools that work with the filesystem. The most common wildcards are * (matches any sequence of characters within a path segment), ** (matches recursively across directories, in tools that support it), and ? (matches a single character). Rather than enumerating every file explicitly, a glob pattern lets you describe a shape that many files share.
Glob patterns in DuckDB
DuckDB accepts glob patterns anywhere a file path is expected in functions like read_parquet, read_csv, and read_json, which is the standard way to query many files as a single logical table.
Copy code
-- all Parquet files in a directory
SELECT * FROM read_parquet('data/*.parquet');
-- recursive match across subdirectories
SELECT * FROM read_parquet('data/**/*.parquet');
-- multiple explicit patterns
SELECT * FROM read_parquet(['data/2026-01/*.parquet', 'data/2026-02/*.parquet']);
Adding filename := true (or the filename parameter, depending on version) to these functions attaches a column indicating which source file each row came from, which is useful for auditing or partition-aware filtering.
A caveat with object storage
When reading from S3 via the httpfs extension, the ? single-character wildcard is not supported due to how it interacts with HTTP URL encoding — only * is reliably supported for S3 paths. Local and HTTP(S) filesystems don't have this restriction. Combining glob patterns with DuckDB's Hive-style partition detection (recognizing key=value directory structures) is a common pattern for querying partitioned Parquet datasets directly from a data lake.
Related terms
DuckDB can read CSV, Parquet, JSON, and other file formats directly with SQL table functions, whether the files are local, remote over HTTP/S3, or matched in bulk with a glob pattern.
LIKE operator →The LIKE operator performs pattern matching on string values, using % as a wildcard for any number of characters and _ for a single character.
httpfs extension →The httpfs extension is DuckDB's built-in module for reading and writing files directly over HTTP(S) and from S3-compatible object storage, Azure, and Google Cloud Storage, without downloading them first.
Object storage →Object storage is a data storage architecture that manages data as discrete objects — each with data, metadata, and a unique identifier — in a flat namespace, accessed over HTTP APIs rather than a traditional file system hierarchy.
Regular expressions in SQL →SQL engines support regular expressions through functions and operators like REGEXP_MATCHES, REGEXP_REPLACE, and SIMILAR TO for pattern-based text matching and extraction.
fsspec →fsspec (Filesystem Spec) is a Python library that provides a single, consistent interface for reading and writing files across local disks, cloud object stores, HTTP, and many other storage backends.
FAQS
Yes, DuckDB's glob matching supports the ** wildcard to match across nested directories, in addition to the standard single-level * wildcard.
Yes. Passing a list of patterns, such as ['s3://bucket-a/.parquet', 's3://bucket-b/.parquet'], lets read_parquet and similar functions read from multiple locations in a single call.
