Live Demo: Get Started with MotherDuck - speedrun queries with the *NEW* Instant SQL!

Simon is a Data Engineer and Technical Author with 20+ years of experience in the data field. He's the author of the Data Engineering Blog (ssp.sh), curator of the Data Engineering Vault (vault.ssp.sh), and currently writes a book about Data Engineering Design Patterns (dedp.online). Simon maintains an awareness of open-source data engineering technologies and enjoys sharing his knowledge with the community.

Simon Späti

## Hey, friend 👋

Hello. I'm&nbsp;<a href="https://www.ssp.sh/">Simon</a>, and I am excited to share another monthly newsletter with highlights and the latest updates about DuckDB, delivered straight to your inbox.
In this March issue, I gathered 10 links highlighting the updates and news from the ecosystem of DuckDB. This time, we have a gsheet extension that reads and writes data to GSheets, as well as Duckberg reading from Iceberg tables. We explore zero-cost data stacks that you can build for no cost or use DuckDB distributed.&nbsp;
If you have feedback, news, or any insights, they are always welcome. 👉🏻&nbsp;<a href="mailto:duckdbnews@motherduck.com">duckdbnews@motherduck.com</a>.


<style type="text/css">
.newsheading {
 border: 0px;
}

.newsheading img {
 border: 0px;
 vertical-align: middle;
 margin-bottom: 16px;
}

.row {
 display: flex;
 flex-direction:column; 
 margin-bottom: 10px;
}

.rowreverse {
 flex-direction:column-reverse;
}

@media screen and (min-width: 600px) {
 .row {
 flex-direction: row;
 }
.columnTextLeft h3 {
 margin-left: 0px !important;
 margin-right: 20px !important;
}

.columnTextRight h3 {
 margin-left: 20px !important;
}

.columnTextLeft p {
 margin-left: 0px !important;
 margin-right: 20px !important;
}

.columnTextRight p {
 margin-left: 20px !important;
} 
}

.columnPic {
 flex: 30%;
}


.columnText {
 flex: 70%;
}

.columnText h3 {
 margin-top: 0px !important;
}


</style>

<table class="newsheading" style="margin-bottom: 5px;">
 <tr>
 <td class="mobileoff"><img src="https://22616816.fs1.hubspotusercontent-na1.net/hubfs/22616816/featured_member_graphic.png" width="64" height="64" /></td>
 <td><h2>Featured Community Member</h2></td>
 </tr>
</table>

<div class="row">
 <div class="columnPic"><img src="https://22616816.fs1.hubspotusercontent-na1.net/hubfs/22616816/1690556744956.jpeg"></img></div>
 <div class="columnText columnTextRight">
 <h3>Runji Wang</h3>
 Runji Wang from Beijing, China, enjoys developing low-level systems using Rust, including but not limited to operating systems, storage systems, database systems, and stream processing systems. Visit his GitHub for more details: <a href="https://github.com/wangrunji0408">GitHub</a>.
He recently contributed to Smallpond through DeepSeek, a lightweight framework for distributing compute with DuckDB (more on that below!).
Thanks for your contribution, and welcome to the flock!
 </div>
</div>

<table class="newsheading" style="margin-bottom: 5px;">
 <tr>
 <td class="mobileoff"><img src="https://22616816.fs1.hubspotusercontent-na1.net/hubfs/22616816/top_links_graphic.png" width="64" height="64" /></td>
 <td><h2>Top DuckDB Links this Month</h2></td>
 </tr>
</table>



### [Announcing DuckDB 1.2.0](https://duckdb.org/2025/02/05/announcing-duckdb-120)

TL;DR: DuckDB 1.2.0 is out for a couple of weeks and introduced significant features, optimizations, and compatibility improvements, enhancing data handling and performance.
DuckDB version 1.2.0, codenamed “Histrionicus,” presents a suite of technical enhancements, including new features like ALTER TABLE ... ADD PRIMARY KEY, support for Latin-1 and UTF-16 encodings in CSVs, multi-byte delimiters, and improved Parquet compression options. The release also addresses breaking changes, such as an updated random function and changes in map indexing behavior. Find the full features in the above link or on <a href="https://github.com/duckdb/duckdb/releases/tag/v1.2.0">GitHub</a>.
Note that DuckDB 1.2.1 was just released yesterday. You can check the full changelog <a href="https://github.com/duckdb/duckdb/releases/tag/v1.2.1">here</a>.


### [Reading and Writing Google Sheets in DuckDB](https://duckdb.org/2025/02/26/google-sheets-community-extension)

TL;DR: The GSheets community extension for DuckDB enables seamless reading and writing of Google Sheets, enhancing workflow automation possibilities.
The GSheets extension in DuckDB allows users to securely interact with Google Sheets for both ad-hoc queries and automated workflows. The extension leverages in-browser OAuth for authentication, simplifying the process to just logging into Google, while scheduled tasks can utilize DuckDB Secrets for persistent authentication. The extension supports reading specific sheets and ranges using additional parameters, with the ability to adjust data type inference and manage headers via SQL commands. Example code snippets such as <code>INSTALL gsheets FROM community;</code>, <code>LOAD gsheets;</code>, and <code>COPY (FROM range(10)) TO 'https://docs.google.com/spreadsheets/d/...?' (FORMAT gsheet);</code> illustrate the syntax for installing and using the extension. This integration allows us data practitioners to automate data exports and imports, enhancing productivity and reducing manual data handling efforts.



### [The Zero Cost Stack](https://rasmusnes.com/posts/stock-advisor-stack/)

TL;DR: Building a zero-cost data engineering stack with DuckDB and MotherDuck for efficient data storage and analytics.
Rasmus discusses using DuckDB for local development and MotherDuck for production storage, achieving a seamless switch between the two with a simple configuration toggle. By utilizing <code>INSERT OR REPLACE INTO</code> for idempotency, the solution maintains only the latest data, ensuring efficient storage use. DuckDB's lightweight compression significantly minimizes disk usage, fitting the entire dataset within MotherDuck's free tier limits. The stack also integrates Go for data extraction and loading, leveraging GitHub Actions for orchestration, running on a public repository with 4-vCPU, 16GiB runners. Streamlit serves as the dashboarding tool, directly querying MotherDuck for real-time analytics. Takeaway: Demonstrates how combining open-source tools and strategic cloud service choices can build a robust data pipeline with no operational cost. Code available on <a href="https://github.com/rasnes/stock-advisor">GitHub</a>.



### [Duckberg: Python package for querying iceberg data through duckdb](https://github.com/slidoapp/duckberg)

TL;DR: Duckberg leverages the power of PyIceberg and DuckDB to facilitate efficient querying of large Iceberg datasets using a Python package.
Duckberg combines PyIceberg and DuckDB to provide a Pythonic approach for querying large Iceberg datasets stored on blob storage. The package supports various Iceberg catalog types like REST, SQL, Hive, Glue, and DynamoDB. Key features include listing tables and partitions and executing SQL queries on data lake files that contain the necessary data, thus optimizing query performance. For example, <code>db.select(sql=query).read_pandas()</code> retrieves data into a Pandas DataFrame.



### [Try DuckDB for SQL on Pandas](https://youtu.be/8SYQtpSk_OI?si=L74qJTRMuKu0PuQd)

TL;DR: This video shows how DuckDB seamlessly integrates SQL querying with Python DataFrames.
Arjan demonstrates how DuckDB provides a unique workflow where developers can easily transition between DataFrames and SQL within a single tool. The 20 minutes video showcases multiple implementation patterns, including executing SQL directly on DataFrames using Python variable references (duckdb.query(df) with special SQL syntax like SELECT * FROM df), creating in-memory or persistent connections via duckdb.connect(), and explicitly registering DataFrames as tables with connection.register().&nbsp; He highlights DuckDB's extended SQL capabilities with examples of DESCRIBE, EXPLAIN, and SUMMARIZE commands for analyzing table structure, query execution plans, and statistical summaries. The practical implementation shown is available on <a href="https://github.com/ArjanCodes/examples/tree/main/2025/duckdb">GitHub</a>.


### [Exploring UK Environment Agency data in DuckDB and Rill](https://rmoff.net/2025/02/28/exploring-uk-environment-agency-data-in-duckdb-and-rill/)

TL;DR: Showcasing the efficient ingest and analysis of UK Environment Agency data, showcasing the power of DuckDB for rapid prototyping and Rill for visualization.
Robin uses DuckDB to load and transform JSON data from the UK Environment Agency's flood monitoring API. The process involves creating staging tables using the read_json function in DuckDB, such as <code>CREATE TABLE readings_stg AS SELECT * FROM read_json('https://environment.data.gov.uk/flood-monitoring/data/readings');</code>. By unnesting JSON arrays, Robin transforms these into more query-friendly formats, enabling joins across tables like readings, measures, and stations to enrich the data. The article highlights a potential pitfall with the API's default 500-record limit, addressed using query parameters like ?today to fetch fuller datasets, resulting in over 170k readings, adjusting <code>maximum_object_size</code> to handle larger JSON files. Robin visualizes it with Rill and comments that it's an interactive dashboard tool with a fast and intuitive setup.


### [DuckDB goes distributed? DeepSeek’s smallpond takes on Big Data](https://mehdio.substack.com/p/duckdb-goes-distributed-deepseeks)

TL;DR: DeepSeek's smallpond extends DuckDB's capabilities to distributed computing, offering new possibilities for handling large datasets.
DeepSeek has introduced <a href="https://github.com/deepseek-ai/smallpond">smallpond</a>, a new framework that leverages DuckDB to achieve distributed computing by integrating with the Ray framework. This approach allows DuckDB to process large datasets by distributing the workload across multiple nodes. The framework uses a Directed Acyclic Graph (DAG) execution model, which optimizes operations by deferring computation until necessary. For instance, operations like write_parquet(), to_pandas(), or compute() trigger the execution. Each task is executed in its own DuckDB instance using Ray’s parallel execution capabilities, sorting 110.5TiB of data in just over 30 minutes. Practical Guide: Data volume under 10 TB is "likely unnecessary" and potentially "slower than vanilla DuckDB", according to <a href="https://www.definite.app/blog/smallpond">Mike</a>, as well as the complexity of setting up 3FS (DeepSeek's storage layer) which is impractical for most analytics use cases.




### [DuckDB vs. coreutils](https://szarnyasg.org/posts/duckdb-vs-coreutils/)

TL;DR: DuckDB performs competitively against UNIX tools in processing large CSV files, particularly when leveraging multi-threading in larger datasets.
In a recent comparison, DuckDB demonstrated its ability to efficiently handle CSV file operations typically managed by UNIX commands like wc -l, particularly when leveraging multi-threading capabilities. Gábor conducted benchmarks using a range of datasets, from 300 MB to 15 GB, showing DuckDB's SQL execution with <code>SELECT count() FROM read_csv(...)</code> can outperform traditional tools under certain conditions. For example, on a large Linux server, DuckDB processed 108 million lines in 2.2 seconds compared to the single-threaded wc at 3.2 seconds. However, when GNU Parallel was used to multi-thread the wc command, it reduced processing time to 0.5 seconds, surpassing DuckDB. While DuckDB's parallel CSV reader is highly efficient, performance gains can vary based on the environment and toolchain.




### [FastAPI Integration with DuckDB](https://github.com/buremba/duckdb-fastapi/)

TL;DR: DuckDB FastAPI enables bidirectional integration between REST APIs and DuckDB queries.
This experimental package offers two key functionalities: (1) Automatically generate REST endpoints from your DuckDB macros with <code>CREATE MACRO get_sample() AS TABLE SELECT 1 as t</code> becoming available at /macro/get_sample; and (2) Query FastAPI endpoints directly from DuckDB using <code>ATTACH 'http://localhost:8000/.duckdb' AS myapi; SELECT * FROM myapi.get_item('123');</code>. The integration works in both directions - creating APIs from DuckDB macros and accessing FastAPI endpoints as DuckDB tables - providing a seamless SQL interface to REST services.



### [New DuckDB Newsletter: Learn DuckDB by example](https://learningduckdb.com/newsletters/welcome-to-learning-duckdb/)

TL;DR: New Newsletter "Learning DuckDB by example". The #1 edition highlights recent developments and practical SQL tips for enhancing analytical database performance, showcasing queries with <a href="https://sql-workbench.com/">SQL Workbench</a> (Online Data Analysis &amp; Data Visualizations).

The newsletter focuses on four main categories: SQL Tips &amp; Tricks, which includes a list of useful SQL queries and their explanations. Tobias also shares DuckDB community news and interesting articles/resources. Check it out at <a href="https://learningduckdb.com/">Learning DuckDB</a>.



<table class="newsheading" style="margin-bottom: 5px;">
 <tr>
 <td class="mobileoff"><img src="https://22616816.fs1.hubspotusercontent-na1.net/hubfs/22616816/upcoming_events_graphic.png" width="64" height="64" /></td>
 <td><h2>Upcoming Events</h2></td>
 </tr>
</table>



### [Panel: Scaling DuckDB to TBs and PBs with Smallpond, MotherDuck and homegrown solutions](https://lu.ma/5946jam3)

**Tuesday, March 11 10:30 EST - Online**

​DeepSeek's smallpond has taken the data world by storm with its distributed DuckDB capabilities and impressive 110TB benchmarks. But what are your real options for scaling DuckDB today?
​Join our expert panel featuring Jordan Tigani (CEO, MotherDuck), Jake Thomas (Data Foundations, Okta), and Mehdi Ouazza (Data Engineer, MotherDuck) as they cut through the hype and discuss scaling strategies from single-node powerhouses to distributed architectures.


### [Build a Real-Time CDC Pipeline with Estuary & MotherDuck](https://lu.ma/5789p0ru)

**Thursday, March 27 9AM PST - Online**

​Want to seamlessly move data from your transactional database to a fast, serverless analytics engine?
​Join MotherDuck and Estuary for a live webinar where we’ll show you how to build a real-time Change Data Capture (CDC) pipeline.

DuckDB Ecosystem: March 2025

Vector Technologies for AI: Extending Your Existing Data Stack

World-class team of engineers and leaders with experience working on databases and cloud services at AWS, Databricks, Elastic, Facebook, Firebolt, Google BigQuery, Neo4j, SingleStore, Snowflake and more

MotherDuck team

It’s 2025. AI is everywhere, and yet—we’re still typing SQL by hand? That ends today.

<iframe width="560" height="315" src="https://www.youtube.com/embed/XJWPbQVbfaQ" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen style="margin-bottom: 15px"></iframe>
 
 

Meet **[Quack To SQL](https://motherduck.com/quacktosql/)**, the world’s _first_ AI model that lets you query your database by quacking. Yes, **literally quacking**. Powered by advanced duck-sound recognition, it runs locally in your browser—just like DuckDB.

No setup. No typing. Just pure, honking productivity.

Want the top 10 customers by revenue? Just quack it.

> 🧠 “But why?”
> 🦆 “Why not.”

<video width="1280" height="720" loop autoplay muted playsinline style="border: 2px solid rgb(56, 56, 56); margin-bottom: 24px;max-width: 100%; height: auto">
 <source src="https://motherduck-com-web-prod.s3.amazonaws.com/assets/img/demo_quack2sql_e653e5203f.mp4" type="video/mp4">
 Your browser does not support the video tag.
</video>

This is more than innovation—it’s inclusion. Billions of ducks have been ignored in the data space for too long. We’re changing that. (And yes, we’re working on sign-quack support for non-vocal ducks and humans too.)

To celebrate this milestone, we’re offering something special:
<div style="max-width: 100px;">
 <img 
 src="https://motherduck-com-web-prod.s3.amazonaws.com/assets/img/duckets_8548a88982.png" 
 style="width: 100%; height: auto;" 
 alt="duckets" />
</div>

Every successful Quack To SQL user will earn **10 Duckets** — real, physical coins from our exclusive Physical Coin Offering (PCO). Unlike traditional cryptocurrencies, these coins come with **zero transaction fees** and **unlimited offline availability**—because sometimes, the best currency is the one you can actually hold.
 

So go ahead. Stop coding. Start [quacking](https://motherduck.com/quacktosql/).

Prompting? That’s so 2024. Welcome to Quack-to-SQL.

Subscribe to motherduck blog

PREVIOUS POSTS

DuckDB Ecosystem: March 2025

Vector Technologies for AI: Extending Your Existing Data Stack