Data lakehouse
A data lakehouse is an architecture that combines the low-cost, flexible storage of a data lake with the ACID transactions, schema enforcement, and performance features of a data warehouse.
Overview
A data lakehouse layers warehouse-like reliability on top of data-lake storage. Data still lives as files (typically Parquet) in cheap object storage such as S3, but an open table format adds a metadata layer that provides ACID transactions, schema enforcement and evolution, time travel, and efficient query planning — capabilities that plain files on a lake don't offer on their own.
The table format layer
The lakehouse pattern is possible because of open table formats: Apache Iceberg, Delta Lake, Apache Hudi, and newer entrants like DuckLake. Each tracks table metadata — which files make up a table, their schema, partitioning, and snapshot history — so multiple engines can read and write the same tables consistently. Iceberg and Delta store that metadata as JSON/Avro files alongside the data; DuckLake instead stores it in a transactional SQL database (DuckDB, Postgres, or SQLite), trading file-listing overhead for regular database transactions.
Why it matters
Before the lakehouse pattern, organizations often ran two systems: a data lake for cheap storage of everything, and a separate data warehouse for governed, query-ready data — with expensive ETL pipelines copying data between them. A lakehouse lets a single copy of the data, in open formats, serve both raw storage and high-performance analytics, reducing duplication and staleness.
DuckDB and the lakehouse
DuckDB can act as a query engine directly against lakehouse tables. The iceberg extension reads and writes Apache Iceberg tables, and the delta extension reads Delta Lake tables:
Copy code
INSTALL iceberg;
LOAD iceberg;
SELECT * FROM iceberg_scan('s3://bucket/warehouse/orders');
INSTALL delta;
LOAD delta;
SELECT * FROM delta_scan('s3://bucket/warehouse/customers');
DuckDB is also the engine behind DuckLake, a lakehouse table format that stores catalog metadata in a SQL database rather than files, and is supported directly in MotherDuck.
Related terms
DuckLake is an open table format, created by the DuckDB team, that stores lakehouse metadata in a transactional SQL database instead of files — while keeping the actual data as Parquet files in object storage.
Table format →A table format is a metadata layer on top of data files in a lake or object storage that defines what constitutes a table — its schema, partitioning, and file list — enabling ACID transactions, schema evolution, and time travel across multiple query engines.
Delta Lake →Delta Lake is an open table format, originally developed by Databricks, that adds ACID transactions, schema enforcement, and time travel to Parquet data stored in a data lake.
Databricks →Databricks is a cloud data and AI platform built by the original creators of Apache Spark, centered on the lakehouse architecture that unifies data lakes and warehousing.
Apache Iceberg →Apache Iceberg is an open table format that adds ACID transactions, schema evolution, and time travel to large analytic tables stored as files in a data lake.
Data lake →A data lake is a centralized repository that stores raw structured, semi-structured, and unstructured data at any scale, in its native format, until it's needed for analysis.
FAQS
No. A data lake is raw storage of files with no transactional guarantees. A lakehouse adds an open table format (Iceberg, Delta Lake, Hudi, DuckLake) on top of that storage to provide ACID transactions, schema management, and time travel.
The most common are Apache Iceberg, Delta Lake, and Apache Hudi, all of which store metadata as files next to the data. DuckLake is a newer format that stores the same kind of metadata in a SQL database instead.
Yes. DuckDB has dedicated extensions for Apache Iceberg (iceberg) and Delta Lake (delta) that let you query those tables with standard SQL, either from files directly or via a catalog.
