Together AI: Scaling a Data Platform while Scaling the Company
On top of compute-heavy warehouses, agent-driven queries would create a serious cost problem. MotherDuck’s architecture is an excellent fit for the kind of exploration and unpredictable queries that agents need to be useful.
Pablo Ferrari
Director of Data Engineering
How a $3.3B AI infrastructure company chose MotherDuck over Redshift, Athena, and ClickHouse as their analytics serving layer — powering 128 self-service Hex users, 40 read replicas, and unconstrained AI agent queries.
Together AI is the AI Native Cloud powering millions of AI-native companies and enterprises like Cursor, Decagon, Salesforce, and Zoom. Their product is a full-stack platform that runs AI — GPU clusters, inference APIs, and fine-tuning infrastructure. Understanding how that product performs is key to how they run the business.
At a $3.3B valuation and scaling the org rapidly, the stakes for getting data right are high. Data drives capacity decisions, pricing, product roadmap, and the dashboards that their board and executives review. A data platform that can't scale, or scale affordably, is a liability. In a stack built to be flexible, MotherDuck took on a leading role.
- Fastest on 100GB benchmarks against Athena, Redshift, and ClickHouse during testing
- AI agent workloads run freely — Secoda and Hex AI agents, as well as custom-built agents, run smaller, exploratory queries onMotherDuck without the cost spikes that would require rate-limiting on pricier warehouses
- 128 unique Hex users across product teams, C-suite, and board — all self-serving on gold tables
- 40 read scaling replicas supporting concurrent analytics access at scale
Building Streaming Pipelines at Together AI
Together AI's streaming pipelines started with Julia Zhang — a software engineer, not a data person — trying to give her colleagues access to their own company data.
"I'm normally just a software engineer. I don't work in data," Julia explains. "It was really just: how could we get access to our own data quickly?"
For a company building AI infrastructure, access matters. Engineers needed to debug production issues. Teams needed visibility into what customers were actually doing.
Julia ran a formal benchmark. She tested four options — Amazon Athena, Redshift, ClickHouse, and MotherDuck — using the 100GB version of the TPC-DS benchmark, a standard analytical workload that mirrors ad hoc SQL with joins and complex logic. The evaluation criteria were practical: run standard SQL, don't require specialized expertise, and don't cost a fortune to explore.
ClickHouse looked promising but couldn't clear the bar. "I was never even able to fully run all of the TPC-DS queries because it was difficult to translate into SQL that ClickHouse accepts," Julia says. Of the full TPC-DS suite — standard ANSI SQL — only 10–20 queries ran without modification. The rest required custom rewrites to work with ClickHouse's syntax.
"We were just a team of software engineers. We want to just write some basic SQL and have it work."
MotherDuck ran the full suite and came out fastest. The cost model sealed it.
"I didn't have to worry about accidentally costing the company a lot of money with my side data projects. And I could feel pretty comfortable that it would be able to serve my team for their needs."
Julia didn't stop at standing up a database. She connected Hex, an AI data exploration platform the team has quickly come to love.
Building a Durable, Efficient Platform
Pablo Ferrari came to Together AI by way of his own venture in the space. A computer scientist and entrepreneur, Pablo had been building a startup in AI infrastructure before ultimately joining Together.
He joined forces with Julia to scale Together's data platform — and brought data engineering and distributed systems expertise to the team. As the company took off, the data platform grew to a 12-person team spanning streaming, data warehouse, infrastructure, and data services. The architecture got a proper medallion structure. Federated analytics lets product teams own their own dashboards instead of waiting on a central BI team. And MotherDuck's role as the datamart grew.
Today, Together's architecture cleanly routes data from central storage to the right compute layer for each job–across internal and customer-facing use cases.
Raw events flow from 60+ service streams via Kinesis and Kafka, land in S3 as Iceberg tables, and pass through a first data warehouse for transformation — dbt and Airflow running the pipelines on a schedule. That layer handles the ETL work, where costs are predictable.
Once data reaches gold, a custom YAML-parameterized sync tool moves it into MotherDuck. Everything downstream — Hex dashboards, semantic layer models, AI agent queries — runs against MotherDuck. Real-time streams from Kinesis also flow directly into MotherDuck via streaming — around 5–6 TB per month on the larger tables alone — bypassing the warehouse entirely for observability cases. This enables investigations and debugging, use cases that demand near-realtime performance alongside parallel batch processing.
The serving layer is now MotherDuck across the board. 40 read scaling replicas support 128+ concurrent users — product teams, executives, and board members — all running self-service Hex dashboards without involving the data team.
"Once the data warehouse processes are done and tables are Certified Gold tables, they go into MotherDuck as a serving layer," Pablo explains. "People can create Hex dashboards on top of them."
The cost difference is stark. Serving interactive dashboards and running exploratory queries on MotherDuck costs a fraction of what they'd pay on another compute engine — a meaningful lever for a company scaling at this pace.
Serverless Analytics: AI Agents Without the Bill Shock
The clearest test of MotherDuck's cost model comes from AI agents.
Together AI uses both Secoda (for data catalog and discovery) and Hex (for semantic layer and BI) — both of which expose AI agents that analysts and product teams use to query data. The problem with agents: they're unpredictable. They run many queries, often inefficiently. In a typical warehouse, unconstrained agent usage translates directly into painful bills. With MotherDuck's serverless model, the team keeps their ducks (and agents) in a row.
"When you look at the queries agents make — even though we do our best to document the tables — the agents do a lot of queries, and sometimes not the most efficient ones," Pablo says. "On top of compute-heavy warehouses, agent-driven queries would create a serious cost problem. MotherDuck's architecture is an excellent fit for the kind of exploration and unpredictable queries that agents need to be useful."
Together AI runs these agents without rate-limiting, quotas, or engineering oversight. The same workloads that would require guardrails on more expensive compute run without concern on MotherDuck.
"The MotherDuck-Hex combo is fantastic," Pablo says. "It's been transformative for the company."