A Practical Guide to Context Management for Data Agents

Context management means giving your data agent enough background — business definitions, how your tables relate to each other, what "revenue" actually means at your company — so it writes correct SQL instead of guessing.

Without that background, you get a familiar problem: ask the same question twice, get two different answers. The model doesn't know your org excludes refunds from revenue, or that "active customer" means someone who logged in within 90 days, not just anyone with an account. So it improvises, and it improvises differently each time.

Get the context right and the agent becomes useful for real business decisions. Get it wrong and you're debugging hallucinated joins at 11pm.

LLMs were trained on massive amounts of natural language, so they parse English definitions well. Plain English descriptions of your data model give the best accuracy and are the easiest to maintain.

Adding SQL context — sample queries for your most common questions — matches English on accuracy and runs faster because the model spends less time exploring. Semantic model formats like Cube or LookML tend to do worse; LLMs just haven't seen enough of those languages to handle them reliably.

Write your definitions in plain English and include SQL snippets for the queries people actually ask.

Start with no context. Run questions against your data, see where the LLM gets answers wrong, and add context to fix those specific failures. Focus on what the LLM can't figure out from your column and table names alone — how your business defines a metric, which tables join to which, what "active" actually means in your schema. Don't dump everything you have into the context file. Irrelevant context adds noise and slows things down. You can start with MotherDuck's MCP server or a markdown file on your machine.

Share your context file through GitHub or a shared workspace like MotherDuck. Pick someone to own keeping it current. Once non-technical users start relying on your data agents, add test cases that check answers against known correct results. Track questions that fall outside your test coverage so you can expand it. If your context files grow unwieldy, automate some of the curation.

You can, but you'll get better results if you translate it to English first. Semantic modeling languages like Cube and LookML predate LLMs, and LLMs aren't great at interpreting them. If you feed your raw semantic model straight to an LLM, you'll get worse answers than if you write out the same definitions in plain English. It's an extra step, but converting your semantic model into natural language descriptions makes a real difference in response accuracy.