Throughput

Overview

Throughput measures a system's total processing capacity: how many operations, requests, rows, or bytes it can get through in a given amount of time (for example, queries per second, or rows scanned per second). It's distinct from latency, which measures how long any single operation takes. A system can have high throughput with relatively high per-operation latency (by processing many operations concurrently or in large batches), or low throughput despite low latency per operation (if it can only handle one request at a time).

What limits throughput

Throughput is typically bounded by whichever resource saturates first: CPU (compute-bound workloads), disk or network I/O (data movement-bound workloads), memory bandwidth, or concurrency limits (connection pools, thread counts, lock contention). Improving throughput usually means either parallelizing work across more resources (more cores, more machines) or reducing the amount of resource each unit of work consumes (better compression, more efficient execution, avoiding unnecessary data movement).

Throughput in analytical engines

DuckDB is designed around maximizing throughput for analytical (OLAP) queries on a single machine: it uses a vectorized, columnar execution engine that processes data in batches rather than row by row, and automatically parallelizes a query's work across all available CPU cores. This lets a single DuckDB process scan and aggregate very large volumes of data quickly, and its columnar storage and compression reduce the amount of I/O needed to read from disk, which raises effective throughput further for I/O-bound queries against files like Parquet.

Copy code
-- DuckDB automatically parallelizes a full-table scan and aggregation
-- across available threads without any explicit configuration
SELECT category, sum(amount), count(*)
FROM read_parquet('orders/*.parquet')
GROUP BY ALL;

Throughput versus scale

Single-node throughput has limits set by the hardware of that one machine. Distributed systems (Spark, distributed warehouses) trade added coordination overhead for the ability to scale throughput further by adding more machines. For workloads that fit within a single machine's resources, a highly optimized single-node engine like DuckDB can often deliver throughput competitive with, or exceeding, a distributed system, without the operational cost of managing a cluster.