DuckLake
DuckLake is an open table format, created by the DuckDB team, that stores lakehouse metadata in a transactional SQL database instead of files — while keeping the actual data as Parquet files in object storage.
Overview
DuckLake reimagines the lakehouse table format around a simple idea: table metadata (schema, snapshots, file lists, statistics) belongs in a database, not in a growing pile of JSON and Avro files on object storage. Formats like Apache Iceberg and Delta Lake track metadata as files that must be listed, opened, and parsed to plan a query; DuckLake instead stores that same information as rows in a regular SQL database — DuckDB, PostgreSQL, or SQLite — and relies on that database's own ACID transactions to guarantee consistency.
Why it matters
As Iceberg and Delta tables accumulate history, their file-based metadata (manifest lists, manifest files, transaction logs) can itself become large and slow to plan against, and multi-table transactions across several tables are awkward because each table's metadata is tracked independently. Because DuckLake's metadata lives in one SQL database, a single transaction can atomically update multiple tables at once, and querying "what changed" or planning a scan is a SQL query against the catalog rather than a series of file reads.
Data storage
DuckLake still writes the actual table data as Parquet files in ordinary object storage (S3 or compatible), so the bulk-storage economics are unchanged from Iceberg or Delta Lake — only the metadata layer moves into a database. DuckLake reached a 1.0 release with backward compatibility guarantees, adding features like sorted tables, partitioning, data inlining for small writes, and Iceberg-compatible deletion vectors.
Using DuckLake
DuckLake is built into DuckDB and supported directly in MotherDuck. A DuckLake "database" is attached like any other:
Copy code
ATTACH 'ducklake:metadata.ducklake' AS my_lake (DATA_PATH 's3://my-bucket/data/');
CREATE TABLE my_lake.events (id INTEGER, event_type VARCHAR, ts TIMESTAMP);
INSERT INTO my_lake.events VALUES (1, 'signup', now());
SELECT * FROM my_lake.events;
The metadata.ducklake file (or a Postgres/SQLite connection) holds the catalog; DATA_PATH points to where Parquet data files are written.
Related terms
A data lakehouse is an architecture that combines the low-cost, flexible storage of a data lake with the ACID transactions, schema enforcement, and performance features of a data warehouse.
Table format →A table format is a metadata layer on top of data files in a lake or object storage that defines what constitutes a table — its schema, partitioning, and file list — enabling ACID transactions, schema evolution, and time travel across multiple query engines.
Apache Iceberg →Apache Iceberg is an open table format that adds ACID transactions, schema evolution, and time travel to large analytic tables stored as files in a data lake.
metadata →Metadata is information that describes other data.
Data lake →A data lake is a centralized repository that stores raw structured, semi-structured, and unstructured data at any scale, in its native format, until it's needed for analysis.
Delta Lake →Delta Lake is an open table format, originally developed by Databricks, that adds ACID transactions, schema enforcement, and time travel to Parquet data stored in a data lake.
FAQS
Iceberg and Delta Lake track table metadata as files (JSON and Avro manifests, or a JSON transaction log) alongside the data in object storage. DuckLake stores that same metadata as rows in a transactional SQL database — DuckDB, PostgreSQL, or SQLite — while still storing table data as Parquet files.
Table data is still written as Parquet files in ordinary object storage such as S3, the same as Iceberg or Delta Lake. Only the catalog and snapshot metadata are moved into a SQL database.
Use ATTACH with a ducklake: prefix, for example: ATTACH 'ducklake:metadata.ducklake' AS my_lake (DATA_PATH 's3://bucket/data/'), then create and query tables under my_lake using standard SQL.
