LEARN HOW TO PREPARE YOUR DATA WAREHOUSE FOR AGENTIC AI WORKLOADSRegister for the webinar

Skip to main content

Read Scaling

Connecting read-heavy applications or BI tools with many concurrent users through a single MotherDuck account can sometimes lead to performance bottlenecks. By default, all connections using the same account share a single cloud DuckDB instance, called a "duckling". In addition to your read/write duckling, you can use Read Scaling to spin up additional read-only ducklings for read-heavy workloads.

These replicas are eventually consistent. Results may lag a few minutes behind the latest database state. This tradeoff prioritizes high availability and performance while achieving near real-time synchronization across all replicas.

Read Scaling

Configuring a Read Scaling Duckling Pool​

Creating a Read Scaling token​

To use Read Scaling, you use a read scaling access token from the MotherDuck UI when generating an access token or via the REST API.

Connect with a read scaling token​

Once you have a read scaling token, you can use it to connect to MotherDuck from any DuckDB client as you would with any other authorization token. See Connecting to MotherDuck.

Duckling Assignment​

Read scaling ducklings remain idle until a connection is initialized from a DuckDB client. When a DuckDB client connects to MotherDuck with a read scaling token, the connection is assigned to one of the read scaling replicas. As more users connect, additional ducklings are spun up until you reach your Read Scaling Duckling Pool size.

If the number of connections exceeds your pool size, new connections are assigned to existing ducklings in a round-robin fashion.

The default Read Scaling Duckling Pool Size is 4 and can be increased up to 16. This is a soft limit, so if you need more ducklings in your pool, please contact support.

Permissions​

A read scaling token grants permission for read operations (SELECT) while restricting write and administrative operations (updating tables, creating new databases, attaching or detaching databases).

Ensuring Data Freshness​

In read scaling mode, ducklings sync changes from the primary read-write instance within a few minutes which works for most use cases.

If your application requires stricter synchronization, you can manually trigger updates to be more frequent by:

  1. Calling CREATE SNAPSHOT on the writer duckling
  2. Calling REFRESH DATABASES on any read scaling ducklings

This approach guarantees that readers see the most recent snapshot.

Watch Out

Creating a snapshot of a database will interrupt any ongoing queries interacting with that database.

Best Practices​

Here are a few tips to get the most out of MotherDuck's read scaling capabilities.

Optimize your Read Scaling Duckling Pool size​

For the best experience, aim for one duckling per concurrent user to take advantage of DuckDB's single-node power and efficiency. You can scale up as much as you need by configuring a maximum pool size based on expected concurrency and cost considerations. Users are also able to share ducklings if needed. While the default limit is 16 replicas, this is a soft limit. Get in touch with MotherDuck support if you need more.

Leverage local processing where possible​

Consider using DuckDB WASM to run client instances directly in the browser when possible to fully utilize client resources.

Maintain user-duckling affinity with session_hint​

To ensure users consistently connect to the same replica (improving caching and consistency), the DuckDB connection string supports the session_hint parameter:

  • Clients providing the same session_hint value are directed to the same replica. This improves caching effectiveness, provides a more consistent view of data across queries for that user and offers better isolation between concurrent users.
  • This parameter can be set to the ID of a user session, a user ID, or a hashed value for privacy.

By leveraging read scaling tokens and session_hint, you can efficiently scale read operations and group user sessions for optimal performance.

Instance caching with dbinstance_inactivity_ttl​

Some DuckDB client library integrations support an instance cache to keep connections to the same database instance alive for a short period after use. This improves read scaling by helping maintain session affinity even across separate queries or short connection gaps. This caching behavior boosts the effectiveness of session_hint, making it more likely that frequent queries from the same client land on the same duckling, even with short breaks between connections. See Connecting to MotherDuck for more details.