---
title: "Apache Hive"
description: "Apache Hive is a data warehouse system built on Hadoop that lets users query large datasets stored in HDFS or object storage using HiveQL, a SQL-like language, which Hive translates into distributed jobs."
canonical: "https://motherduck.com/glossary/apache-hive/"
related:
  - title: "DuckLake Architecture Deep Dive"
    url: "https://motherduck.com/blog/ducklake-architecture-deep-dive/"
  - title: "Data Warehousing How-to | MotherDuck Docs"
    url: "https://motherduck.com/docs/key-tasks/data-warehousing/"
  - title: "DuckDB & Iceberg : The Future of Lightweight Data Management | MotherDuck"
    url: "https://motherduck.com/videos/duckdb-iceberg-the-future-of-lightweight-data-management/"
gated_asset:
  title: "DuckLake on MotherDuck"
  url: "https://motherduck.com/product/ducklake/"
---

# Apache Hive

> Apache Hive is a data warehouse system built on Hadoop that lets users query large datasets stored in HDFS or object storage using HiveQL, a SQL-like language, which Hive translates into distributed jobs.

## Overview
Apache Hive gave the Hadoop ecosystem a SQL interface over data that was otherwise only accessible through low-level MapReduce jobs. Users write HiveQL, a language closely modeled on SQL, and Hive compiles it into a distributed execution plan — originally run as MapReduce jobs, and later on faster engines like Tez or Spark.

## Table management and the Hive Metastore
Hive introduced the Hive Metastore, a service that tracks table schemas, partitions, and storage locations — separate from the query execution engine itself. This design proved influential well beyond Hive: the Hive Metastore's table and partition model became a de facto standard, and many modern lakehouse catalogs (including Iceberg's Hive catalog implementation and AWS Glue, which is Hive Metastore-compatible) still speak its interface today.

## Hive's partitioning convention
Hive also popularized the `key=value` directory partitioning scheme still used broadly across the data lake ecosystem — for example, `year=2026/month=01/` — which lets query engines prune irrelevant directories without reading them, independent of whether Hive itself is involved.

## Limitations that motivated newer formats
Hive tables track their file list by listing directories at query time, which becomes slow and unsafe with many files or concurrent writers, and doesn't provide snapshot isolation. These limitations were the direct motivation for Apache Iceberg, originally built at Netflix specifically to replace Hive-style tables with explicit, transactional metadata.

## Hive concepts in DuckDB
DuckDB doesn't run HiveQL or connect to a Hive Metastore directly, but it inherits Hive's partitioning convention: `read_parquet()` and `read_csv()` accept a `hive_partitioning` option that recognizes `key=value` folder names and exposes them as query-able columns, letting DuckDB prune partitions the same way Hive-aware engines do:

```sql
SELECT year, month, SUM(amount)
FROM read_parquet('s3://bucket/orders/*/*/*.parquet', hive_partitioning = true)
WHERE year = 2026
GROUP BY ALL;
```