AWS Bought DuckLabs and Users Want Their Own Agents

Show notes

AWS bought DuckLabs, the company employing the core team behind DuckDB. DuckDB remains open source, but the deal still changes the strategic map around one of the fastest-growing databases in the data stack. Mehdi and Dumky start with the obvious question: what does AWS actually want? Faster development around DuckDB and S3 is one answer. A stronger developer-facing data story is another. Then there is the agent workload: agents need cheap, isolated compute close to data, and an embedded analytical database is a natural fit. The less technical possibility is just as important—AWS gets a rare team and prevents another hyperscaler or data platform from hiring it.

For MotherDuck, the acquisition creates a deadline rather than an ending. AWS may have distribution and infrastructure, but it is not known for delightful analytical interfaces. The opportunity is to keep building the broader product around DuckDB: developer ergonomics, collaboration, governance, and a usable home for analysts and agents. The precedents cut both ways. Databricks built a durable business around Spark and Confluent did the same around Kafka, while other open-source infrastructure companies struggled once cloud platforms absorbed their core technology. Execution now matters more than speculation.

The conversation moves from ownership to architecture with SeaweedFS's two-format lakehouse. Iceberg remains the structured, governed source of truth for analytical scans. Lance stores embeddings and supports fast vector retrieval. DuckDB becomes the SQL bridge that joins the two without inserting another ETL hop. That leads into a broader design pattern: expose more of an agent's context through SQL. A single query interface can span transactional data, lakehouse tables, vector indexes, usage history, and cost data while remaining inspectable by humans.

Spotify's evaluation funnel adds a process layer. LLM evals should verify relevance, coherence, and quality before a team spends scarce traffic on an online experiment. They do not replace A/B tests, which validate whether real users and production systems respond as predicted. The useful loop runs both ways: evals filter weak candidates before an experiment, and experiment outcomes reveal whether the judge is measuring anything that matters. CodexBar makes a related problem tangible for developers by surfacing local Codex and Claude usage, even if translating subscription quotas into a true dollar cost remains messy.

Josh Rosen's lessons for AI-powered data warehouses raise the observability stakes. When a model generates facts, summaries, columns, or SQL, ordinary table lineage is no longer enough. Teams also need to know which model acted, what prompt and tools shaped the result, and how that work changed the data. Agent activity needs the equivalent of query history. Without it, a warehouse can accumulate AI-generated rows faster than anyone can explain where they came from.

The final stories ask how much infrastructure should disappear behind an interface. WalGit is a stateless Git server packaged as one Rust binary and backed by object storage, a deliberately small alternative to an all-in-one forge. DuckDB's optimizer makes the opposite argument at the query layer: hide engine-specific rewrites so humans and agents can write readable SQL while the database adapts the plan to the data. The episode closes on agent UX. Shipping an agent inside every SaaS product is not automatically useful, but neither is exposing only an MCP server. New users may need an in-product agent to cross the cold-start gap; power users may want composable tools for their own agent. The product still has to earn the interface.

Key takeaways

• AWS's acquisition of DuckLabs changes the incentives around DuckDB, but it does not change DuckDB's open-source status.
• MotherDuck's differentiation is the product and collaborative experience around DuckDB, not exclusive ownership of the engine.
• Iceberg and Lance solve different access patterns; DuckDB can bridge analytical tables and vector retrieval through SQL.
• LLM evals verify quality before an experiment. Online experiments validate whether the change produces the intended user and business outcome.
• AI-generated data needs model, prompt, and tool lineage in addition to ordinary table and query history.
• Good query optimizers let humans and agents keep SQL readable without giving up engine-specific performance.
• In-product agents help with discovery and cold starts; MCP and other headless interfaces give advanced users control. Strong products will likely need both.