Introducing Flights: agent-native data pipelines in MotherDuckJoin the livestream

Skip to main content

ShadowTraffic

ShadowTraffic is a synthetic data generation tool for simulating production traffic to your backend. It integrates with MotherDuck for development workflows that read from or write to MotherDuck.

How it works with MotherDuck​

ShadowTraffic can generate read and write traffic against a MotherDuck database for development, testing, and load-shaping workflows.

Prerequisites​

  • ShadowTraffic 1.10.0 or later.
  • A MotherDuck token.
  • A target MotherDuck database.

Setup​

  1. Create a MotherDuck token and store it in an environment variable such as MOTHERDUCK_TOKEN.

  2. Add a ShadowTraffic connection with kind: motherduck:

    {
      "connections": {
        "md": {
          "kind": "motherduck",
          "connectionConfigs": {
            "token": {
              "_gen": "env",
              "var": "MOTHERDUCK_TOKEN"
            },
            "db": "mydb"
          }
        }
      }
    }

  3. Add generators that write to tables or run read queries through that connection.

Authentication and configuration​

  • Use token and db for the standard MotherDuck connection.
  • Use jdbcUrl only when you need to control the full JDBC connection string.
  • Use queryParams for MotherDuck connection parameters such as attach_mode.
  • Use batchConfigs to tune write batch timing and row count.

Important notes​

  • ShadowTraffic writes asynchronously. By default it commits when 1000 ms pass or 10000 rows accumulate, whichever happens first.
  • Automatic table creation is enabled by default. Set tablePolicy: manual if you want to manage tables yourself.
  • For UPDATE and DELETE operations, generators need a where map so ShadowTraffic can identify rows.

Use cases​

  • Generate synthetic write traffic into MotherDuck tables.
  • Simulate reads and writes while testing downstream systems.
  • Use automatic table creation for quick generator iteration, then switch to manual table control for production-like tests.

Related content​