--- title: "Aligned on Numbers: Internal Data Apps with Agents | MotherDuck | MotherDuck" description: "MotherDuck's Chief of Staff walks through an AI token spend tracker and a weekly business review Dive, and how Guides keep every metric definition consistent." canonical: "https://motherduck.com/videos/aligned-on-numbers-chief-of-staff-data-apps/" --- [BACK TO VIDEOS](/videos/) [Stream](/videos/?category=Stream#videos-and-webinars-library)[MotherDuck Features](/videos/?category=MotherDuck%20Features#videos-and-webinars-library)[AI ML and LLMs](/videos/?category=AI%20ML%20and%20LLMs#videos-and-webinars-library) # Aligned on Numbers: How Our Chief of Staff Builds Internal Data Apps with Agents at MotherDuck Oct 15, 2026 Zoey Weaver, who leads growth at MotherDuck, interviews Kaitlyn Henry, MotherDuck's Chief of Staff and formerly Chief of Staff at dbt Labs. Kaitlyn walks through two internal data apps built on MotherDuck, an AI token spend tracker and the weekly business review, and explains why Guides keep every team working from the same metric definitions. Both dashboards in the demo use dummy data. ## Tracking AI token spend with a Flight Frontier labs make usage data hard to get. Admin portals show spend per user and per API key, but unless you are on an enterprise plan there is no API, only CSV downloads for some data and screen scraping for the rest. Kaitlyn says she has never written a line of code or a line of SQL by hand, and she still built a [Flight](https://motherduck.com/product/flights/) that downloads the CSV reports and scrapes the admin screens every Monday at 9 a.m., structures the results into MotherDuck, and feeds a dashboard. She says it took under two hours. The dashboard breaks spend down by model and by surface, with Claude Code driving most of it and Cowork climbing. In the demo data, annualized spend is about $197K, roughly the cost of one headcount. The question she watches is how high that number can get before it is worth looking at open-weight models. She also reaches out to individual users when their usage spikes, and a Claude skill can give each person a summary of their own usage. On ROI, she treats small spend like headcount: is AI making the team as productive as one more person? Beyond that, she argues ROI is an individual question because everyone uses these tools differently, so the work is helping each person use them well and making agent management part of a manager's job. ## Checking a pipeline an agent built Because the usage Flight scrapes screenshots, trust is the hard part. Kaitlyn lets the Flight pull the data, asks the model to explain how it processed it, spot checks the results, then folds the fixes back into the Flight prompt. She says that gets her about 90% of the way there. Her other rule: if you own a dashboard, look at it at least every two weeks, so you have an instinct for when a number is wrong. ## The weekly business review Dive and Guides The weekly business review Dive is meant to be the canonical source of truth for how the business is doing: pacing against the quarterly target, revenue from new versus existing customers, weekly changes in expansion and contraction, and an AI chat box for asking what drove a change. Building the dashboard is the easy part. Getting everyone to agree on what the numbers mean is not. [Guides](https://motherduck.com/docs/key-tasks/guides/) are how MotherDuck does it. They are large markdown files that define about 25 key business metrics and how to calculate them, and they sit directly on top of MotherDuck, so every Dive and every question asked in Claude uses the same calculation. Her warning for operators: without them, someone says last week's AWR was 4.8, the CEO says 4.7, and the whole meeting goes to arguing about the number. To settle a definition like churn in a usage-based business, she picked the two or three stakeholders who cared most (the CEO, the head of finance, and the customer success lead), got them aligned, and codified it once. Everyone else picked it up through their tools. ## Guides, dbt, and where metric logic should live MotherDuck still uses dbt for transformation but has no hard semantic model. After running a BI vendor's semantic modeling next to Guides, Kaitlyn found Guides did the same job with far less work. She says per-query a guide costs more than logic in the data model, since an LLM has to interpret it, but maintaining a data model over time is costly too, and guide logic can be moved into the model gradually. She argues definitions belong in the warehouse. A friend on LangChain's data team told her their agents all query the warehouse directly and skip the semantic layer, so definitions need to live as close to the data as possible. She also described moving off a dedicated BI tool after shipping Dives internally, with the two data engineers reporting they had done almost no work. Her closing view: building dashboards is largely solved, and the next problem is trust in the data, which means back-end work that non-technical people can now do. TABLE OF CONTENTS - Tracking AI token spend with a Flight - Checking a pipeline an agent built - The weekly business review Dive and Guides - Guides, dbt, and where metric logic should live Start using MotherDuck now! [Try 7 Days Free](https://auth.motherduck.com/authorize?app_source=web&response_type=code&client_id=bza3KWQpxRAFlTlRFXUo29AOg9xD7zcp&redirect_uri=https%3A%2F%2Fapp.motherduck.com%2F&state=STATE&auth_flow=signup&screen_hint=signup&ext-ph_distinct_id=f08690b0-dc66-4c76-8e59-3a351b95dcce) ## FAQS ### What are Guides in MotherDuck? Guides are large markdown files that define your key business metrics and how to calculate them. Kaitlyn's guide covers about 25 metrics. It sits directly on top of MotherDuck, so every Dive and every question asked in Claude references the same definitions. See the [Guides docs](https://motherduck.com/docs/key-tasks/guides/). ### Does MotherDuck use dbt or a semantic layer? MotherDuck uses dbt as a transformation tool but has no hard semantic model. It previously ran a BI vendor's semantic modeling alongside Guides and found Guides did the same job with much less work. Kaitlyn calls it a philosophical choice each team has to make for itself. ### Is it more expensive to use a guide than to put the logic in the data model? Per query, yes. An LLM interpreting a guide costs tokens, while logic embedded in SQL does not. But maintaining a data model over time is expensive too, and you can move guide logic into the model gradually, for example by telling the LLM to always use a specific SQL query for a given metric. ### How do you get a company to agree on a metric definition? Pick the two or three stakeholders who care most (for churn at MotherDuck: the CEO, the head of finance, and the customer success lead), get them aligned, and codify the definition once in a guide. Everyone else picks it up through their dashboards and queries. Expect to do real change management at the start. ### How do you check a data pipeline that an agent built? Let the Flight pull the data, ask the model to explain how it processed it, spot check the results, and put the fixes back into the Flight prompt. Kaitlyn says that gets her about 90% of the way there. She also looks at each dashboard she owns at least every two weeks to keep her instinct for the numbers sharp. ## Related Videos [43:57](/videos/jev-for-analytics-use-cases/)[Oct 8, 2026](/videos/jev-for-analytics-use-cases/) ### [Jev for Analytics: 5 Use Cases](/videos/jev-for-analytics-use-cases/) Mehdi Ouazza uses MotherDuck's prompt_jev() to turn text columns into columns you can GROUP BY. He classifies 100,000 consumer complaints, then walks through five use cases on real data: AI agent logs, Hacker News sentiment, sales calls, and two kinds of job-ad analysis. Stream MotherDuck Features AI ML and LLMs [51:58](/videos/build-a-data-agent-workshop/)[Oct 1, 2026](/videos/build-a-data-agent-workshop/) ### [Build a Data Agent in 60 Minutes](/videos/build-a-data-agent-workshop/) Jacob Matson walks through a working data chat agent he built: MCP tools, an agentic loop, a tuned system prompt, hard-coded read-only guardrails, and telemetry. He demos it live against a real dataset, walks through the actual codebase, then shows the same backend re-platformed into a Slack bot. Stream MotherDuck Features AI ML and LLMs Data Pipelines [49:02](/videos/prompt-jev-sql-text-classification/)[Sep 29, 2026](/videos/prompt-jev-sql-text-classification/) ### [prompt_jev(): SQL-Native Text Classification](/videos/prompt-jev-sql-text-classification/) MotherDuck's new prompt_jev() function runs text classification directly from SQL using Jev, a decision-focused model from typesafe.ai. In a live demo, Jacob Matson, Dumky de Wilde, and Hamilton Ulmer showed it filtering, disambiguating, and re-ranking real data without leaving SQL. The number they kept coming back to: 100,000 rows classified at 89% accuracy in 40 seconds, compared to 32 minutes and $37.58 for a comparable LLM call. Stream MotherDuck Features AI ML and LLMs Data Pipelines [View all](/videos/)