---
title: "Hive partitioning"
description: "Hive partitioning is a directory-naming convention, originally from Apache Hive, that encodes column values in folder paths (like year=2026/month=01/) so query engines can skip irrelevant files without reading them."
canonical: "https://motherduck.com/glossary/hive-partitioning/"
related:
  - title: "DuckLake Architecture Deep Dive"
    url: "https://motherduck.com/blog/ducklake-architecture-deep-dive/"
  - title: "Ingest Partitioned S3 Parquet on a Schedule | MotherDuck Docs"
    url: "https://motherduck.com/docs/cookbook/flight-scheduled-s3-ingest/"
  - title: "Dive into Pivoting! | MotherDuck Dive Gallery"
    url: "https://motherduck.com/dive-gallery/dives/dive-into-pivoting/"
gated_asset:
  title: "DuckLake on MotherDuck"
  url: "https://motherduck.com/product/ducklake/"
---

# Hive partitioning

> Hive partitioning is a directory-naming convention, originally from Apache Hive, that encodes column values in folder paths (like year=2026/month=01/) so query engines can skip irrelevant files without reading them.

## Overview
Hive-style partitioning organizes files on disk (or in object storage) into nested directories named after column values, using a `key=value` convention. For example, a table partitioned by year and month might be laid out as:

```
orders/
  year=2025/
    month=11/
      part-0001.parquet
  year=2026/
    month=01/
      part-0001.parquet
```

A query engine that understands this convention can look at the `WHERE` clause of a query, determine which partition directories could possibly contain matching rows, and skip reading every other directory entirely — a technique called partition pruning.

<glossary-callout guide="duckdb-book-brief" />

## Why it originated in Hive
This layout comes from Apache Hive, which used directory structure as its primary mechanism for organizing and pruning large tables on HDFS. The convention outlived Hive itself and is now used broadly across the data lake ecosystem, independent of any particular query engine.

## Tradeoffs
Hive partitioning is simple and works with any file format, but it has limits: too many small partitions ("over-partitioning") creates a large number of tiny files, which hurts performance, and changing partition columns requires physically rewriting the directory layout. Modern table formats like Apache Iceberg address this with "hidden partitioning," tracking partition transforms in metadata rather than requiring a specific folder structure.

## DuckDB and Hive partitioning
DuckDB recognizes Hive-partitioned paths automatically when reading Parquet or CSV files, exposing the partition keys as regular columns:

```sql
SELECT year, month, SUM(amount) AS total
FROM read_parquet('s3://bucket/orders/*/*/*.parquet', hive_partitioning = true)
WHERE year = 2026
GROUP BY ALL;
```

DuckDB will skip files outside `year=2026` entirely, without needing to open them.