Data catalog
A data catalog is a centralized inventory of an organization's datasets, tracking metadata like schema, location, lineage, and ownership so people and systems can discover, understand, and access data.
Overview
A data catalog is metadata about data: it doesn't store the underlying rows, but tracks what datasets exist, where they live, what their schema looks like, who owns them, and often how they relate to one another (lineage). Catalogs serve two audiences — humans, who use them to discover and understand available data, and systems, which use them to resolve table names to physical locations when planning queries.
Catalogs in the lakehouse stack
In a lakehouse architecture, a catalog has a very concrete technical role: it maps a table name to the current metadata pointer for that table, so an engine can find the right Iceberg manifest, Delta transaction log, or DuckLake database entry to read. Common catalog implementations include the Iceberg REST catalog specification, AWS Glue Data Catalog, Hive Metastore (which predates and heavily influenced the others), and Databricks' Unity Catalog.
Beyond table resolution
Enterprise data catalogs (like Alation or Collibra) go further, layering in business glossaries, data quality metrics, access controls, and search across data warehouses, lakes, BI tools, and dashboards — helping analysts find "the right" table among many similarly named ones.
Catalogs and DuckDB
DuckDB can attach to lakehouse catalogs to resolve and query tables managed by other engines. For Iceberg, this means attaching a REST catalog directly:
Copy code
INSTALL iceberg;
LOAD iceberg;
ATTACH 'my_warehouse' AS my_iceberg (
TYPE iceberg,
ENDPOINT 'https://catalog.example.com'
);
SELECT * FROM my_iceberg.sales.orders;
MotherDuck also functions as a catalog and query layer of its own — databases and schemas shared through MotherDuck act as a discoverable catalog of tables that any attached DuckDB client can query.
Related terms
Apache Iceberg is an open table format that adds ACID transactions, schema evolution, and time travel to large analytic tables stored as files in a data lake.
Data lakehouse →A data lakehouse is an architecture that combines the low-cost, flexible storage of a data lake with the ACID transactions, schema enforcement, and performance features of a data warehouse.
DuckLake →DuckLake is an open table format, created by the DuckDB team, that stores lakehouse metadata in a transactional SQL database instead of files — while keeping the actual data as Parquet files in object storage.
PyIceberg →PyIceberg is the official Python implementation of Apache Iceberg — a pure-Python library for reading, writing, and managing Iceberg tables and catalogs without requiring Spark or a JVM.
Table format →A table format is a metadata layer on top of data files in a lake or object storage that defines what constitutes a table — its schema, partitioning, and file list — enabling ACID transactions, schema evolution, and time travel across multiple query engines.
metadata →Metadata is information that describes other data.
FAQS
A database stores actual data. A data catalog stores metadata about data — schemas, locations, ownership, and lineage — so people and query engines can find and resolve real tables without directly browsing storage.
A catalog maps a table name to its current metadata pointer (an Iceberg manifest list, a Delta transaction log location, or a DuckLake database), which is how a query engine finds the right version of a table to read without listing storage directly.
Yes. DuckDB's iceberg extension can attach to Iceberg REST catalogs (or Glue, Hive Metastore-compatible catalogs) with ATTACH ... (TYPE iceberg), letting it resolve and query tables managed by other engines through the same catalog.
