DOCS
← Back to Glossary
partitions
Last updatedPartitions in data systems refer to the logical or physical division of large datasets into smaller, more manageable segments.
Partitions in data systems refer to the logical or physical division of large datasets into smaller, more manageable segments. This technique is used to improve query performance and data management efficiency. In databases like DuckDB, partitioning can be implemented using the PARTITION BY clause in window functions or the PARTITION keyword in certain SQL statements. For example:
Copy code
SELECT
year,
sales,
AVG(sales) OVER (PARTITION BY year) as avg_yearly_sales
FROM sales_data;
Related terms
Partition pruning →
Partition pruning is a query optimization that skips reading entire partitions of a dataset when a query's filters make it impossible for those partitions to contain matching rows.
Hive partitioning →Hive partitioning is a directory-naming convention, originally from Apache Hive, that encodes column values in folder paths (like year=2026/month=01/) so query engines can skip irrelevant files without reading them.
PARTITION BY →PARTITION BY divides rows into independent groups for a window function or an output operation, so calculations reset for each group without collapsing rows the way GROUP BY does.
Query planner →The query planner is the database component that translates a parsed SQL statement into an execution plan — a tree of operators describing how the query will actually be run.
Data skew →Data skew is an uneven distribution of data across partitions or keys in a distributed system, causing some worker nodes to process far more data than others and become bottlenecks.
Apache Hive →Apache Hive is a data warehouse system built on Hadoop that lets users query large datasets stored in HDFS or object storage using HiveQL, a SQL-like language, which Hive translates into distributed jobs.
