Google BigQuery

Overview

Google BigQuery is a serverless cloud data warehouse on Google Cloud. It is powered internally by Dremel, Google's distributed query technology, and lets users run SQL over large datasets without provisioning clusters, tuning nodes, or managing storage. You load or reference data, write SQL, and BigQuery allocates the compute needed to run each query.

Serverless model

Because BigQuery is serverless, there are no instances to size or keep running. Compute is expressed in "slots" (units of processing capacity) that the service assigns automatically. This removes cluster management, but it also means query cost and performance are driven by how much data each query scans and how much compute it needs.

Pricing models

BigQuery offers two main approaches to compute pricing:

• On-demand — you pay per volume of data scanned by each query (billed per byte, commonly discussed per TB).
• Capacity-based — you reserve or autoscale slots for more predictable spend at scale.

Storage is billed separately from compute.

BigQuery and DuckDB

BigQuery targets fully managed, cloud-scale warehousing where Google manages all infrastructure. DuckDB is an open-source, in-process OLAP engine for single-node and local analytics, with MotherDuck as its serverless cloud counterpart. Teams often use DuckDB for local development, testing, or cost-efficient processing of files, and BigQuery for centralized, governed cloud warehousing. Because both speak SQL and read columnar formats, moving between them is common. DuckDB can query exported data directly:

Copy code
-- DuckDB reading Parquet exported from BigQuery
SELECT event_type, COUNT(*) AS n
FROM read_parquet('gs://exports/events/*.parquet')
GROUP BY ALL
ORDER BY n DESC;

In BigQuery's standard SQL, an equivalent query reads a table directly:

Copy code
SELECT event_type, COUNT(*) AS n
FROM `project.dataset.events`
GROUP BY event_type
ORDER BY n DESC;