HEY, FRIEND 👋
I hope you're doing well. I'm Simon, and I am happy to share another monthly newsletter with highlights and the latest updates about DuckDB, delivered straight to your inbox.
In this July issue, I gathered the usual 10 updates and news highlights from DuckDB's ecosystem. Please enjoy a portable analytics stack based on R2 and DuckLake, how DuckDB over ClickHouse was chosen, latest news from DuckCon & releases, and a custom WASM build on SedonaDB for spatial SQL directly to the browser.
If you have feedback, news, or any insights, they are always welcome. 👉🏻 duckdbnews@motherduck.com.
Nicolas Renkamp
Nicolas is Global Head of Platform Product Portfolio, Data & AI at Merck Group (EMD Group), based in Heidelberg, Germany. His team has leaned heavily on DuckDB to cut the cost and operational overhead of their Spark pipelines.
He is also the creator of Foundry DevTools, an open-source developer productivity library for Palantir Foundry. Nicolas recently shared how Merck uses DuckDB at DuckCon, following an earlier DuckDB meetup talk. Thanks, Nicolas, for spreading the love for the Duck across the community!
Connect with Nicolas on LinkedIn.
 | Top DuckDB Links this Month |
TL;DR: Yuki details a portable analytics stack built on DuckDB and DuckLake for ephemeral compute and shared state, minimizing reliance on traditional data warehouses.
dlt ingests NYC Open Data to Cloudflare R2 (backed by DuckLake), with metadata in Neon Postgres. SQLMesh handles transformations, natively using DuckDB as its execution engine and supporting both standard SQL and Python/Polars models. Its virtual data environments enable cost-effective iteration via uv run sqlmesh plan dev for development previews and uv run sqlmesh run for scheduled production deployments. GitHub Actions provides CI/CD for version control and automated runs. The result is a flexible, open-source, local-first alternative for smaller teams that want to avoid vendor lock-in and heavy infrastructure. Code at GitHub.
TL;DR: PostHog re-architected its data warehouse from a multi-tenant ClickHouse cluster to single-tenant DuckDB instances to improve data modeling flexibility and tooling compatibility.
Eric noted that ClickHouse, while excellent for fast analytics, struggled with diverse data modeling and schema changes and lacked a cost-based query optimizer. The new architecture runs dedicated single-tenant DuckDB instances, managed by a lifecycle service for efficient resource use. It exposes a Postgres wire protocol endpoint that translates Postgres-compatible queries into DuckDB SQL for seamless integration with existing tools. A custom DuckHog extension enables local compute and data manipulation with tools like Pandas, while DuckLake provides storage-compute separation through an S3-backed catalog.
TL;DR: duckrun provides a DuckDB SQL engine that reads and writes Delta Lake, locally or on OneLake, S3, GCS, and ADLS.
Mimoune, the author, explains that DuckDB handles SQL execution, while delta-rs performs all writes via an Arrow C-stream. A key implementation detail is snapshot-pinned writes: reads are fixed to a snapshot, and concurrent writes committing against a stale version are explicitly rejected with CommitFailedError rather than silently overwriting, enabling robust concurrent operations. The project also extends dbt-duckdb with Delta-backed materializations and supports multi-catalog setups.
TL;DR: DuckCon #7 in Amsterdam (June 24, 2026) previewed DuckDB 2.0, including a VARIANT type ("imagine JSON is fast"), triggers, async I/O for object stores, and partitioning with GROUP BY, plus production talks from Spotify, MariaDB, and Snowflake.
Mark presented the 2.0 roadmap: async I/O makes reading Parquet on S3 and other network storage much faster, and the C API rework keeps extensions stable across versions. MariaDB is embedding DuckDB as a storage engine and comparing with ClickBench and other benchmarks; Snowflake's pg_lake runs DuckDB as a sidecar with Iceberg logic rewritten in C inside Postgres, syncing with Polaris Catalog via a ~10-second cron; SQLFrame (built on SQLGlot) transpiles PySpark to DuckDB without code changes. Another talk showed a DuckLake lakehouse running on Hetzner for under €15/month, roughly a third of AWS cost. Slides and videos are linked above, plus the keynote, my notes, and a summary post.
TL;DR: CereusDB is a custom WebAssembly build of Apache SedonaDB that brings spatial SQL directly to the browser.
Tobi compiles SedonaDB's spatial engine (built on Apache DataFusion and Arrow) to Wasm, optionally integrating GEOS, PROJ, GDAL, and S2 geography via Emscripten. This enables browser-side spatial joins, distance queries like ST_DWithin, and ST_KNN, running as a single-partition, in-memory MVP. Package sizes range from a minimal 4.0 MB Brotli build with 130 ST_* functions to a full 8.5 MB build adding 33 RS_* raster functions for GeoTIFF/TIFF processing. The API offers db.sqlJSON() for query execution and db.registerRemoteParquet(name, url) for ingestion, covering many geo workflows entirely client-side.
TL;DR: Ramona's independent measurements of DuckDB's Quack client-server protocol show remote.query() (pushdown mode) has bounded ~2x overhead, while ATTACH mode overhead grows with scan size and fails on joins.
Her lab analysis shows pushdown mode scales comparably to in-process DuckDB for analytical aggregations (α≈0.33 vs α≈0.35), with the ~2x delta attributed to the server running at 2-4 threads versus 8 in-process. Conversely, ATTACH 'quack:host' streaming scans show α≈0.55 overhead that grows with data size and currently block multi-table joins with a "Not implemented Error". Even so, Quack pushdown consistently outperforms PostgreSQL on analytical aggregations, with the gap widening as data scales (α≈0.33 vs PostgreSQL's α≈0.60).
TL;DR: MotherDuck Flights provides an agent-native, per-run Python environment with direct DuckDB access, and dlt naturally serves as a guardrail for agent-built pipelines.
Martin highlights that Flights executes arbitrary Python with requirements.txt dependencies and automatically injects MOTHERDUCK_TOKEN for secure connections. Creation and scheduling are exposed as SQL functions, e.g., call md_create_flight(...). Billing is based on runtime consumption (~60 cents/hour), making frequent, short, idempotent tasks cost-effective. dlt handles declarative pipelines, schema evolution, and incremental loading inside the Flight runtime (e.g., pipeline.run(repo_rows(), table_name="repos")), abstracting SQL merge and schema drift logic for agent-generated pipelines. Local testing is straightforward with uv run.
TL;DR: Vu positions DuckDB as an embedded OLAP system that removes the friction data practitioners face with traditional client-server databases.
He notes the embedded design avoids the overhead of database setup, complex ingestion and extraction processes, and external state dependencies that often trouble data scientists working in scripting languages like Python or R. The key insight is that DuckDB integrates analytical capabilities directly into the host process, simplifying workflows and offering a self-contained solution for local data analysis.
TL;DR: DuckDB shipped two simultaneous patch releases, v1.5.4 (Variegata) and v1.4.5 LTS (Andium), focused on bugfixes, performance, and security hardening.
v1.5.4 fixes VARIANT casts reading wrong rows under a filter, MERGE INTO binding of WHEN NOT MATCHED clauses, and variant shredding, plus crashes from a gzip write overflow and a double free in Arrow GeoArrow CRS serialization. It adds an OPERATOR_ROW_GROUPS_SCANNED Parquet metric, jemalloc heap trimming, and an experimental vacuum_rebuild_indexes ATTACH option.
v1.4.5 LTS backports out-of-bounds security fixes, zstd performance improvements, and * NOT SIMILAR TO 'pattern' support, plus fixes for integer overflow detection, unbounded row group growth on indexed tables, and a CSV reader buffer-boundary read. The team said v2.0.0 lands this fall.
TL;DR: Floe, a Polars-powered data contract runtime, adds DuckDB and MotherDuck sink support for validated ingestion with advanced write modes.
Alexis, the main author, states the sink supports both local .duckdb files and MotherDuck md: remote databases. Built on native DuckDB 1.5.0, it feeds Arrow RecordBatches directly into MERGE INTO operations with SCD1/SCD2 support and full Delta-sink parity, eliminating extra serialization steps. A process-wide single-writer connection cache fingerprints resolved MotherDuck tokens (e.g., ${ENV_VAR}) for secure, distinct cache entries. The sink ships as a separate companion distribution, floe-duckdb (Docker, Homebrew), which the lean floe CLI auto-detects and transparently delegates to, giving data engineers a columnar execution gate to validate data before it reaches trusted storage.
2026-08-04. h: 09:00. The Venetian, Las Vegas, NV
America's largest AI conference. MotherDuck is a sponsor, so catch CEO Jordan Tigani on the Data Engineering panel: "Building the Modern Data Stack for AI: What's Working and What's Not."
2026-09-15. h: 09:00. Las Vegas, NV
The world's largest gathering of dbt users, 2,200+ data leaders shaping the future of analytics and AI. MotherDuck is sponsoring again, so hope to catch you there.
2026-09-23. h: 09:00. Olympia, London, UK
The UK's largest data, analytics, and AI conference at Olympia London: two days, 400+ sessions, and exhibitors from Snowflake and Databricks to early-stage startups. MotherDuck will be there.
