I hope you're doing well. I'm Simon, and I am happy to share another monthly newsletter with highlights and the latest updates about DuckDB, delivered straight to your inbox.
In this August issue, I gathered the usual 10 updates (+2 bonus) and news highlights from DuckDB's ecosystem. Please enjoy reading about giving agents their own databases, having a single database connection that grants access to over 30 external DBs, a sneak peek into the upcoming v2 async feature, and how DuckDB is used for graph analytics and as a data layer for multiple tools.
Kyle is co-founder at Greybeam and the author of the DuckDB Internals series. Part 1 walked through the high-level design decisions that make DuckDB fast and went viral on Hacker News. He just released Part 2 on vectorized execution, which you will also find in the links below.
Thanks, Kyle, for these deep technical dives. In a world of short content, this is much appreciated.
Connect with Kyle on LinkedIn.
TL;DR: Joe proposes an agent-native architecture where each agent leverages its own embedded DuckDB instance for direct, peer-to-peer analytical data exchange, moving computation closer to the agents.
Joe explored an alternative to centralized data platforms for AI agents by giving each agent (peer) a dedicated, embedded DuckDB instance using Quack. Quack peers communicate directly via loopback TCP ports, exchanging "immutable analytical slices". Each slice is uniquely identified by a SliceRef dataclass with catalog_id, dataset, DuckLake snapshot_id, a contract_digest of its Malloy-defined semantic model, and a slice_digest. This approach decentralizes computation and analytical state, addressing concurrency and rapid chaining of results in agent workloads.
💡 One bonus article as related: Martin wrote about Your coding agent keeps a diary, with a separate DuckDB for AI with dlt extraction.
TL;DR: The new DuckDB ADBC extension enables DuckDB to connect directly to over 30 external databases via the Arrow Database Connectivity (ADBC) API, leveraging Apache Arrow for efficient data transfer.
Sam explained that this extension offers a unified interface to systems like Snowflake, Databricks, and PostgreSQL, moving beyond individual vendor extensions. It utilizes Apache Arrow for fast, zero-copy, columnar data transfers, bypassing the performance bottlenecks of row-based APIs. Users can execute queries directly with read_adbc('profile://mydb', 'SELECT * FROM games') or use ATTACH 'profile://mydb' AS mydb (TYPE adbc) for persistent connections supporting SELECT, INSERT, COPY, and CTAS. ATTACH currently lacks predicate and projection pushdown, requiring read_adbc for optimized large dataset operations, but this is a planned enhancement.
TL;DR: A big one for the upcoming DuckDB v2.0 (Fall 2026), introducing asynchronous I/O for Parquet and CSV files, significantly improving query performance by better utilizing remote storage bandwidth.
Pedro explains the implementation involves separate REGULAR worker and ASYNC I/O thread pools (up to 256 threads), coupled with a read-ahead queue and asynchronous memory governance to proactively fetch data and prevent out-of-memory issues. Benchmarks show dramatic speedups such as TPC-H Q6 SF100 on S3 with Parquet improved 3x (from 8.23s to 2.84s), or saturating network bandwidth, while CSV queries saw a nearly 20x speedup (from 878s to 45s), even under concurrent workloads and memory constraints.
TL;DR: WebDuck is designed for web hosting providers who want to offer their customers a ready-to-use administration interface for DuckDB databases and data analytics out of the box, just like the database admin tools that come with any hosting package.
As the name says, Webduck provides a UI with an overview dashboard, projects, and browsing views. It implements a DuckDB storage engine with per-file reader-writer locks to ensure safe concurrent access, aligning with DuckDB's single-writer model. Mike, the creator, highlighted the offloading of blocking engine and auth calls to a threadpool via asyncio.to_thread in api/db.py and api/admin.py.
TL;DR: DuckGQL, a C++17 extension, integrates a growing ISO GQL subset into DuckDB, allowing graph pattern queries and mutations with DuckDB's native relational storage and execution engine, and provides an explicit CSR layer for graph algorithms.
DuckGQL leverages typed vertex and edge tables, supporting bulk import and GQL operations. The extension uses dynamically built, connection-local CSR projections for algorithms, automatically invalidated by mutations, and property indexes utilize native DuckDB ART indexes, extending DuckDB's capabilities for direct graph analytics.
TL;DR: Perspective, a visualization and analytics component for large and/or streaming datasets, ships a DuckDB Virtual Server in v5.0.0 that replaces its built-in query engine with pushdown SQL, so <perspective-viewer> drives DuckDB directly instead of ingesting data.
It adds DuckDBHandler, a VirtualServerHandler implementation for both Python and browser duckdb-wasm. ViewerConfig options like group_by, filters, sorts and expressions are translated into SQL, with results read back via Apache Arrow IPC. Because DuckDB owns the query layer, its aggregates and expression syntax surface in the UI rather than perspective-server's: Andrew's screen recordings show 6mm rows from NYC Open Data and native time_bucket used inside the Column Expression Editor to build time-series bins. NYC Open Data Demo, Browser demo.
TL;DR: Kyle follows up on his highly shared Part 1, highlighting that DuckDB's vectorized execution processes data in 2048-row batches, reducing function call overhead and improving CPU IPC over the row-at-a-time Volcano model.
Key elements include DataChunks and four Vector types (Flat, Constant, Dictionary, Sequence), unified by a UnifiedVectorFormat for streamlined operator logic and delayed materialization. Filters leverage selection vectors to avoid data copying. Kyle explains how DuckDB is using interpreted execution with precompiled functions, optimizes inner loops with validity masks and C++ compiler-driven SIMD. The transition to a push-based execution model enhances parallel processing, enabling independent pipelines for sources, operators, and sinks, crucial for UNION ALL.
TL;DR: A Rust crate that enables Apache DataFusion to interact with DuckLake tables via various catalog backends, including SQLite, DuckDB-file, Postgres, and client-only DuckDB Quack transports.
The core integrates as a DataFusion TableProvider and catalog provider, facilitating WRITE, DELETE, UPDATE, and maintenance operations. Developers can leverage the DuckLakeSessionContext to execute DuckDB-shaped SQL commands like ATTACH 'ducklake:sqlite:metadata.sqlite' AS lake (DATA_PATH 'data/') for seamless DataFusion integration with DuckLake catalogs.
TL;DR: Example of how to use MotherDuck's MCP, Dives, and Flights to build a personal Disney World ride planner, dynamically integrating live and historical data for optimal decision-making.
Alex detailed how remote MCP-enabled AI agents, like Claude, can orchestrate database operations and app construction. Data was fetched from live APIs using read_json for parallel querying across park endpoints.
Historical wait times were regularly refreshed via a MotherDuck Flight, the new agent-native data pipeline. The solutions provide a live ride-and-show planner and a live ride planner. Check out the blog above if you go there or want to see how these look and how to build your own for another open API you have access to.
TL;DR: Self-hosting DuckDB scales from local to cloud, but each step adds operational complexity, especially for concurrency and persistence.
Mehdi highlights a practical map for self-hosting DuckDB, detailing the increasing operational "boxes" at each stage. An initial local setup on NVMe is the fastest, but moving data to S3 introduces network bottlenecks and authorization overhead. Scaling compute to the cloud (e.g., EC2) can mitigate this, and distributed queries can be achieved via fan-out with Lambda functions for partial GROUP BY and SUM/COUNT. For concurrent readers, dedicated compute per user flattens tail latency, while DuckDB-WASM pushes compute to the browser. He mentions that many writers require either the new quack_serve protocol for serialized writes or a DuckLake setup with a managed Postgres catalog for ACID transactions.
His core takeaway is to self-host, but don't accidentally build a database company.
2026-08-18. h: 08:00. Online
You've seen a hundred demos of AI writing SQL. The query was never the hard part. dltHub, MotherDuck, and Lightdash build the whole stack live in under an hour: one schema travels from AI-generated ingestion to dbt models and a governed semantic layer.
2026-08-26. h: 09:00. Online
Half the industry says AI agents are useless without a semantic layer. The other half ships agents that run fine on a pile of markdown files. Jacob Matson and Alex Monahan ran the evals and share what plain-text context vs a Malloy semantic layer showed.
2026-09-03. h: 09:00. Online
Schemas drift, bad loads slip in, and someone always needs last Tuesday's numbers. Alex Monahan, co-author of DuckLake: The Definitive Guide, joins Hoyt Emerson (Early Signal) to walk through time travel, schema evolution, and the change feed live.
2026-09-15. h: 09:00. Las Vegas, NV
The world's largest gathering of dbt users, 2,200+ data leaders shaping the future of analytics and AI. MotherDuck is sponsoring again, so hope to catch you there.
2026-09-23. h: 09:00. Olympia, London, UK
The UK's largest data, analytics, and AI conference at Olympia London: two days, 400+ sessions, and exhibitors from Snowflake and Databricks to early-stage startups. Come find us at booth P60.
2026/08/10 - Adithya Krishnan, Garrett O'Brien, Sheila Sitaram
AI agents can now sign up for MotherDuck with a single API call - no forms, no credentials, no human in the loop (unless you want to be, of course). Experiment faster, share data, or just give your agents a ducking fast set of tools for analytics.
2026/08/13 - Alex Monahan
Use an agent to build data apps with the combo of Vercel V0 and MotherDuck! V0 makes prototyping and deployment easy, while MotherDuck Flights provide seamless data ingestion and MotherDuck handles up to billions of rows. Your agent can even sign up for MotherDuck for you! See a full example app and the prompt that kickstarted it!
