Latency

Overview

Latency measures how long something takes from start to finish, typically expressed for a single operation (as opposed to throughput, which measures how much work completes over time). In distributed systems and databases, latency is usually reported as a distribution rather than a single number, because most systems have some fast requests and some slow ones — commonly summarized using percentiles like p50 (median), p95, and p99, since the tail of the distribution often matters more to user experience than the average.

Components of latency

End-to-end latency for a database query, for instance, can include network round-trip time, connection setup, query planning/optimization, actual execution time, and time to serialize and transfer the result. Reducing latency typically means attacking whichever component dominates: caching to avoid repeated computation, indexing to avoid full scans, connection pooling to avoid repeated handshakes, or moving compute closer to data to cut network round trips.

Measuring latency percentiles

Copy code
SELECT
    quantile_cont(response_ms, [0.5, 0.95, 0.99]) AS latency_percentiles
FROM requests;

For very large request logs, approx_quantile gives a fast, bounded-error estimate of the same percentiles when exact computation is unnecessarily expensive.

Latency versus throughput

Latency and throughput are related but distinct, and often in tension: a system can have low latency per request but low throughput if it can't handle many requests concurrently, or high throughput with high per-request latency if it processes requests in large batches. Query engines like DuckDB are generally optimized for low latency on interactive, single-query analytical workloads on one machine, since it executes queries using all available cores on the local hardware rather than distributing work across a cluster — which is also why very large, distributed workloads may favor a system built for high aggregate throughput across many nodes instead.