Today DuckDB Labs announced “Quack”, the client-server protocol for DuckDB. This is exciting for a bunch of reasons and marks a new step in the evolution of DuckDB. Virtually every other database has a client-server protocol, and it is natural for one to exist in DuckDB. People have already been building them on their own, with various levels of polish, so it makes sense for DuckDB to build an official version.
We at MotherDuck are strong believers in DuckDB as client and as a server; we have been running like this for nearly four years, and are excited to see this concept getting mainstream adoption in DuckDB. We expect to support Quack for MotherDuck users later this year. We’ve been getting our wings dirty with a preview version of Quack, and we’re looking forward to what the new protocol can offer the DuckDB community.
What is Quack?
Quack lets you stand up a DuckDB instance in a server and connect to it from other clients, also running DuckDB. Quack communicates using HTTP, which means that it should be highly robust and work with all kinds of network environments. It uses a custom protocol, serializing DuckDB’s internal data vector blocks rather than transcoding them to another format. This reduces overhead since no transformation needs to be done on either the client or the server.
The primary driving benefit is that this will allow multiple DuckDB processes to all write to the same database. If you run DuckDB in normal, embedded mode, opening a database for writing causes the file to be locked; this means that you can only have one writer at a time. But if the writer process is a server, you can connect to it from any number of clients.
At the moment, authentication, authorization, and security are basic. There is a shared token between servers and clients, and that needs to be used to connect. By default, this uses HTTP and not HTTPS, so communication is not encrypted. And since DuckDB has no notion of users, there isn’t a way to give different types of access to different types of users. Of course, DuckDB offers rich extensibility, so people can build these features themselves.
There are, of course, other cool things that can be done with this, like being able to use DuckDB as a DuckLake catalog server. And I’m looking forward to seeing what other clever things folks are going to come up with.
How is MotherDuck different from Quack?
MotherDuck is a cloud-hosted DuckDB. Quack lets you connect to a remote DuckDB you run as a service. If you squint, it sounds like you could just run DuckDB with the Quack extension on an EC2 server somewhere and you have a data warehouse, right? The answer is, “Well, it depends on what you need.
We at MotherDuck celebrate and encourage people to run Quack on their own. It drives innovation, it pushes the DuckDB ecosystem forward, and by proxy, it pushes MotherDuck forward. There are going to be more use cases for DuckDB, and that’s a great thing for anyone who wants to see DuckDB succeed. Open source communities have been great for many cloud services, and MotherDuck is no exception.
DuckDB is a unique database with many use cases. We’ve spent the last few years at MotherDuck shaping its qualities and helping make it an engine that powers a robust data warehouse. Along the way, we’ve solved a few important challenges. I thought it would be a good idea to highlight some of the “why MotherDuck vs self-hosting DuckDB” in light of this new functionality.
Multi-user permissions. DuckDB has always been a single-user database, but organizations have different users with different needs and different levels of access. Quack allows many users with a shared token to all talk to the same DuckDB database, but those users all effectively have the same identity, with access to the same data. User management is crucial for deploying a production data warehouse, which is why MotherDuck allows you to create and manage multiple users in your organization, and to synchronize users via SCIM. We’ve also built service accounts, which can be used by ingestion and BI tools as a shared resource.
The concept of sharing data sits right next to the multiple user model. In MotherDuck, you grant access to databases via shares, which are access grants given to organizations, specific users, or the public. DuckDB itself doesn’t have a concept of sharing, it is all-or-nothing; you have access or you don’t.
Authentication. Authentication is the act of proving who you are to a computer or service, and a number of standards have arisen that encapsulate the hard parts of this. Authentication in Quack is as basic as possible, just a shared secret. This may be fine for some simple scenarios, but in production you’ll want more options. In MotherDuck, you can authenticate using browser-based auth, short-lived and long-lived token based authentication, or also use Single Sign On from your favorite auth provider.
Separation of Storage and Compute. One of the key innovations in cloud data management is separating compute and storage. MotherDuck separates storage from compute by running our own differential storage engine. We run a quack-ton of DuckDB instances, and this lets you failover from one instance to another seamlessly, scale instances up or down, and never worry about the persistence of your data. It just works. While Quack can write to networked storage or Iceberg, you're still basically tied to a single compute instance. In MotherDuck, you can easily scale out reads to multiple DuckDB instances, or shutdown your DuckDB when you don’t need it. It will come back within a few milliseconds when you do.
Tiered Storage. MotherDuck’s storage system is tiered to allow low latency and durability while also not costing a ton of money. Data is stored at the lowest level on object storage, but then cached in fast SSDs and in memory. The object storage layer gives durability and low cost, while the shared SSD and memory cache provide low latency.
Differential Storage. DuckDB locks the database file when it is opened for writing. It also maintains just one copy, the most recent snapshot, of the data. A Quack-powered DuckDB server has the same behavior. At MotherDuck, we have built a differential storage engine that is tightly integrated with DuckDB. This allows multiple readers on other machines to read a consistent snapshot of the databases. It also allows time travel and zero-copy clones for efficient mutations. In Quack, if the database is opened for writing, you can’t also have readers, or at least not from different DuckDB instances.
Hypertenancy. MotherDuck gives each user their own DuckDB instance. This means that different users are isolated from each other, and you never have to worry about whether another user or another workload is slowing you down. If you have thousands of users, each one still runs independently, on dedicated hardware, and you can scale down to zero immediately.
One thing we’ve learned in building MotherDuck is that while a single DuckDB instance is powerful, it is also fairly easy to overwhelm it if you have a lot of concurrent requests. MotherDuck supports Read Scaling, which can spin up multiple DuckDB instances to handle high demand. This means you don’t have to worry about whether you can handle high loads–you can easily scale out more instances in response.
Serverlessness. MotherDuck instances start instantly in response to a query request, and shut down after queries complete. So you never have to worry about starting up and shutting down instances. Analytics workloads are ideal targets for a serverless architecture, and we believe that managing instance lifetimes isn’t something you’ll find joy in doing yourself. If you run a Quack server, you have to start and stop instances manually.
Support, SLA, and Observability. MotherDuck has a 99.9% availability SLA for the business tier, and all paying users get support. Problems get addressed quickly. If you want to actually run a production data warehouse, it is very helpful to have someone you can turn to that can get you back up and running quickly. Moreover, MotherDuck has tools that can let you understand what is going on with your DuckDB instances. Which in a perfect world you would never need, but can come in handy when you do.
Postgres Endpoint: Earlier this year, we released our Postgres endpoint to allow users to connect to MotherDuck through the Postgres ecosystem. Though DuckDB support is growing, virtually every tool already knows how to speak Postgres. With the endpoint, any Postgres client can connect to and use MotherDuck as if it were a Postgres database. We all believe DuckDB is the future, but having a bridge to existing systems allows not changing everything at once.
Quacking on
We’ve earned our stripes building with DuckDB across core data warehousing features like our fleet of Ducklings (compute instances) to extending MotherDuck as a platform with features like Dives and our MCP Server.
As for what’s next with Quack, we plan to support Quack as another endpoint type. This will mean you could connect to MotherDuck in the same way you connect to a Quack server. We don’t have a timeline set in stone, but we’re going to shoot for shipping this with DuckDB 2.0.
In the meantime, we’re excited to continue growing the open source community around DuckDB–from hosting in-person meetups, to educating developers, to writing about the latest in the ecosystem. The Quack announcement really is great news for DuckDB users and the DuckDB community. It is going to allow a lot of people to use DuckDB in new ways. We’re excited to support it at MotherDuck, and can’t wait to see what people build.
