Announcing MotherDuck: Hybrid Execution Scales DuckDB from your Laptop into the Cloud

Hybrid execution: cloud and laptop working together

Data scientists, analysts, and engineers love DuckDB because it works great no matter where their data lives. Since many data professionals have powerful laptops sitting 85% idle, they often want to bring the data to their local machine to make it even more efficient to crunch, especially when performing ad hoc analysis and development. MotherDuck lets you analyze this local data locally, while still JOINing with data processed in the cloud, giving you efficient use of all your compute resources.

In the example below, the table yellow_cab_nyc lives in MotherDuck in the cloud, and I have a CSV on my laptop table with currency conversions. We want to see the average cost of NYC taxi trips by passenger count in different currencies by JOINing these two tables. Yes, we’re seamlessly joining data on my laptop with data in the cloud!

You can even do hybrid query execution with data stored in s3, with MotherDuck securely storing and managing your AWS credentials.

Note, these examples are part of our sample datasets and queries, feel free to run them yourself!

You might wonder how this works under the covers. By connecting your DuckDB instance to MotherDuck, you establish a radically different type of distributed system - one, in which one node is MotherDuck in the cloud, and another node is wherever your DuckDB lives, be it your laptop or a lambda, Python or CLI, JDBC or MotherDuck’s own web app. Both nodes execute queries in concert in the most optimal way, automatically routing parts of queries to the right location.

MotherDuck includes a web notebook and Git-style Collaboration

Want to run some quick SQL queries without downloading and installing DuckDB? The MotherDuck web application provides a notebook-like UI. This enables you to analyze local CSVs and parquet files, upload them and manage them alongside your other data stored in MotherDuck.

As a DuckDB-in-the-cloud company, naturally MotherDuck embeds DuckDB in its web application using WASM. Results of your SQL queries are cached in this DuckDB instance, enabling you to instantly sort, pivot, and filter query results!

Want to share your DuckDB data with colleagues? Using SQL, you can create a shareable snapshot of your data, which your colleagues can easily attach in MotherDuck.

This SQL command will return a shareable URL which can then be used by your colleague to access the shared database.

Anywhere you can Duck, you can MotherDuck

DuckDB has been starred by over 10k developers on GitHub, and it might be due to the simplicity of getting up and running with a downloadable, open source analytics engine. We want to continue (and improve!) that amazing experience as we bring DuckDB to the cloud.

One way to do this is by ensuring MotherDuck works well with many of the most popular technologies in the modern data stack, including ingestion, orchestration and BI+Visualization tools.

We strive to make MotherDuck as easy to adopt as DuckDB. To that end, any DuckDB instance in the world running in Python or CLI can connect to MotherDuck with a single line of code. Suddenly, by running this command your DuckDB magically becomes supercharged by MotherDuck. Such ease of onboarding could only have been possible via close collaboration with the creators of DuckDB!

Continuing to Make Analytics Ducking Awesome

One of the primary reasons we were driven to build a serverless analytics platform on top of DuckDB was their fast-paced innovation. Many features in DuckDB have gone from thoughts in academic papers to committed code in a few weeks.

We’re launching MotherDuck now and doing weekly releases because we admire and want to emulate this speed of execution. Thanks in advance for all the feedback you can provide to make MotherDuck a better product!

Get Started

Request an invite now to get started using MotherDuck, and join the flock on slack.motherduck.com.

And, if you’re in San Francisco next week, don’t forget to register for the MotherDuck Party, watch DuckDB co-creator Hannes keynote the Data + AI conference, and join MotherDuck co-founder Ryan Boyd in his technical session.