Snowflake (data cloud)

Overview

Snowflake is a fully managed cloud data platform used for data warehousing, analytics, and data sharing. It runs on top of AWS, Azure, and Google Cloud, but abstracts the underlying infrastructure so users interact only with databases, warehouses, and SQL. Its defining characteristic is a multi-cluster, shared-data architecture that separates storage, compute, and cloud services into layers that scale independently.

Architecture

Snowflake splits work across three layers:

• Storage — data is held in compressed, columnar files in cloud object storage, managed by Snowflake.
• Compute — independent "virtual warehouses" (compute clusters) read from that shared storage. You can spin up multiple warehouses against the same data without contention, and pause them when idle.
• Cloud services — metadata, query optimization, security, and transaction management.

Because storage and compute are decoupled, teams can resize or add warehouses without copying data, and pay for compute only while queries run.

Snowflake and DuckDB

Snowflake uses a proprietary engine and always runs in the cloud. DuckDB is an open-source, in-process OLAP engine that runs locally or embedded in applications, and MotherDuck is its serverless cloud counterpart. Both DuckDB/MotherDuck and Snowflake share the storage/compute separation idea, but DuckDB targets single-node analytics, local development, and low-overhead querying of files, whereas Snowflake targets managed, elastically scaled cloud warehousing. A common pattern is to prototype or process data locally with DuckDB, then load into Snowflake, or read Snowflake-exported Parquet with DuckDB:

Copy code
-- DuckDB reading Parquet exported from a warehouse
SELECT region, SUM(amount) AS revenue
FROM read_parquet('s3://exports/sales/*.parquet')
GROUP BY ALL
ORDER BY revenue DESC;

Standard Snowflake SQL is ANSI-compatible with its own extensions:

Copy code
SELECT region, SUM(amount) AS revenue
FROM sales
GROUP BY region
ORDER BY revenue DESC;