What If We Don't Need the Semantic Layer?

A Brief History of Semantic Pain

To understand why I think the semantic layer is unnecessary, we have to understand why it was born.

The entire business intelligence industry was built on a foundational compromise: sacrificing analytical flexibility for query performance. In the 1990s, relational databases were too slow for complex analysis. The solution was the OLAP cube, a multi-dimensional data structure that solved the performance problem through pre-aggregation. Before a user asked a question, the system pre-calculated and stored the answers for every combination of dimensions like time, geography, and product.

This delivered incredible speed. But the price was inflexibility. The cube was a rigid grid. If a user's question required a dimension not included in the original design, it was impossible to answer without a data engineer redesigning and reprocessing the entire cube. Analytics was confined to a pre-defined set of questions.

The semantic layer emerged as the solution. In 1991, Business Objects patented a "relational database access system using semantically dynamic objects," the concept that would become the "Universe". In theory, the idea was elegant: create an abstraction layer that translates complex database schemas into business-friendly terms. Users could drag and drop familiar concepts like "Revenue" and "Customer" without knowing that Revenue came from joining three tables with a specific WHERE clause.

But reality told a different story. Creating and maintaining a Universe required specialized skills. Universe Designer certification became a career path. Every vendor at the time seemed to add a competitive solution - Cognos had PowerCubes and Impromptu, there was Hyperion Essbase, and of course, old reliable - Microsoft's SSAS.

Each vendor's semantic layer was proprietary. Each required specialists. If you've ever written a semantic query, you know the feeling. You stare at a query that looks nothing like the SQL you learned, wondering why something as simple as "show me sales by region" requires navigating a maze of brackets, axes, and cube syntax.

We'd gone from one gatekeeping problem (complex databases) to another (specialized BI tools). The semantic layer was supposed to democratize data access. Instead, it created a new priesthood of Universe Designers, MDX developers, and Cognos specialists. Meanwhile, the real semantic layer, the one everyone actually used, was an Excel file called revenue_master_FINAL_v3.xlsx that Bob from Finance emailed around every Monday.

Here's what's easy to forget: all of this complexity existed because of a performance constraint. OLAP cubes and pre-aggregation were necessary because queries were painfully slow. But that constraint has largely evaporated. Modern analytical databases (like MotherDuck) can run complex queries fast and economically, without pre-computing every possible answer. The technical justification for the semantic layer's architecture? It's gone. But the architecture persists.

Modern Tools, Same Assumption

Modern tools have improved the semantic layer. Platforms like dbt, Looker, and Cube.dev have revolutionized the process of creating semantic layers. With dbt, metric definitions live as version-controlled code. Looker's LookML provides a powerful language for defining relationships. These tools made the definition-based paradigm more robust and maintainable.

Personally, it seems like they haven't questioned the fundamental assumption. They still operate on the premise that a human must manually define every metric, every dimension, and every relationship before a question can be asked. They make the act of building the framework easier, but it's still a framework that limits exploration. "One more metric, Bro! I swear just one more" is the common refrain as an ever deepening backlog of work continues to grow.

The long tail of business questions remains infinite and unanswerable without falling back on the data team. These tools have perfected the definition-based approach. It isn't enough, and I would argue, it's the wrong paradigm for enablement in the first place.

The Paradigm Shift

This idea changed my thinking: every query ever run against a database contains latent semantic information, just sitting there waiting to be used.

Instead of a top-down, prescriptive model, what if we embraced a bottom-up, descriptive one? The semantic layer isn't something we have to build from scratch. It already exists, implicitly, in our query logs. Query logs are the empirical record of how people actually use data, and that collective behavior represents the true "source of truth" in an organization.

Until recently, mining this latent knowledge was computationally infeasible. But LLMs change everything. They can parse a user's natural language question, search the vast corpus of query history and metadata for relevant patterns, and synthesize that information to generate correct SQL for questions they've never seen before.

This transforms the semantic layer from a static library into a learning system that gets smarter with every new query. Can one learn this power?

Discovery in Practice

We explored this idea at MotherDuck. Evgeniia Egorova, who worked with us while completing her Master's degree, wrote her thesis on exactly this problem. Her methodology automatically generated contextual descriptions for every table and column in a database by mining its query history: parsing thousands of real-world queries, aggregating usage patterns, and using an LLM to synthesize findings into documentation.

The results confirmed what I'd suspected. By mining query history, the system could automatically uncover domain knowledge that would never make it into a manually defined semantic layer.

This example that resonated with me: A user asked: "Among patients with abnormal glutamic oxaloacetic transaminase levels, when was the youngest born?" A text-to-SQL model has no way of knowing what "abnormal" means for that column. But the query history revealed that analysts consistently wrote WHERE GOT > 60. That implicit medical knowledge, never documented anywhere, was automatically synthesized into the column's description. The AI got the query right because it learned from how humans actually worked.

A traditional semantic layer would have required a medical expert to manually document that rule. This approach discovered it by observing behavior.

The pattern was clear: usage frequency is the ultimate relevance signal. In a data warehouse with four different tables containing the word "revenue," query logs reveal which one the organization actually trusts. This bottom-up signal is more robust and resilient than any static, one-time declaration.

The AI-Native Alternative

LLMs can now be extended with what Anthropic calls "skills": modular packages that encode domain-specific expertise. This is even more relevant as both Anthropic and OpenAI have announced integrations for this functionality in recent days. While not explicitly stated, we can make these into living documents that evolve.

Here's what a semantic layer looks like as an AI skill:

Copy code
---
name: Web Analytics
description: Query user and session data using natural language, applying company-specific definitions.
---

## Database Context
- Primary events table: `analytics.events`
- User sessions defined by 30-minute inactivity window
- Use `is_bot = false` to exclude crawler traffic

## Common Patterns
- "Daily active users" → COUNT DISTINCT user_id WHERE event_date = today
- "Session duration" → Use session_end - session_start, exclude bounces

## Business Rules
- A "new user" is first seen in the last 7 days
- Exclude internal IP ranges from all metrics
- "Engaged session" requires 2+ page views or 30+ seconds

These skills can also connect to databases directly through protocols like MCP (Model Context Protocol). Conveniently, MotherDuck just released our remote MCP server and initial use cases have been incredibly promising when using the latest LLMs.

AI can discover the semantic layer, encode it, and evolve it.

The Consistency Problem

I know what you're thinking: "But what about accuracy?"

Let me be clear about what problem we're actually solving. We've all been in that meeting where three people bring three different versions of the same metric, and the boss says "What is this? You need to figure out which one is right." (Spoiler: all three numbers came from Excel files with "final" in the filename.) That's the nightmare scenario, and it's not one we can afford to make worse.

Here's the key insight: the AI skills approach actually helps with consistency, as long as everyone uses the same skills. When the logic for "daily active users" or "qualified lead" lives in a shared skill, everyone querying that metric gets the same answer (usually). The skill becomes the single source of truth.

So let's compare this to traditional semantic layers: they demand 100% accuracy, which means they can only address a small, well-defined island of questions. Only the ones someone took time to formally model get canned answers, everything else falls back to the data team.

AI skills let you expand that island dramatically. The core metrics that show up in board meetings? Those still need rigorous, shared definitions encoded in dashboards. But the thousands of exploratory, long-tail questions that drive real differentiation? Those can be answered on the fly, using discovered patterns, even if they're occasionally imperfect.

The goal is to stop letting the pursuit of perfection block access to the 99% of questions that never get an answer today - by building an answering machine.

The AI-powered system is anti-fragile. Every user interaction, especially corrections, becomes a signal that enriches the system for everyone. This model enables continuous improvement. Traditional semantic layers are most accurate the day they're built, then slowly decay until someone rebuilds them.

Do We Need the Semantic Layer?

So back to my opening question: What if we've been solving the right problem (making data accessible) in the wrong way?

I think the answer is yes. With AI, we can stop defining what questions can be asked and start discovering what questions have been asked.

Query history mining is one way to enrich our understanding of the data we already have. AI skills are another. Both treat the collective activity of data users as the ultimate source of truth. They use AI to scale human expertise, codifying the tribal knowledge that's practiced every day but hard to get written down.

Dashboards aren't going anywhere. They ground us in a shared sense of truth and help us build intuition about our data. AI just lets us go further, asking the follow-up questions that dashboards can't anticipate.

The future looks like answering more questions, faster, with less friction.

If we can make semantic layers obsolete, what else have we been doing backwards in data?