Is BI Too Big for Small Data?

Principle 1: Shift from Exploration to Interpretation

"The hard part is not creating this chart," Stancil argues. "The hard part is interpreting it."

Most BI tools are built for exploration. They give you endless options to filter, pivot, and visualize. But when your data is sparse, these features don't lead to clarity; they just create more confusing charts. The real bottleneck isn't getting the data; it's figuring out what, if anything, it means. Stancil's insight is that "interpretation of data is often a lot harder than exploration."

This is where speed and interactivity become critical. To interpret an ambiguous chart, you need to form and test hypotheses rapidly. Is this dip because of the holiday? Let me pull last year's data. Is it a specific user segment? Let me filter by plan type. Is it a bug? Let me look at error logs from the same period.

If each query takes minutes to run, you lose your train of thought. The friction of waiting kills the iterative cycle of questioning that is essential for interpretation. When you can test ideas as fast as you can think of them, you shrink the gap between question and answer, making the hard work of interpretation just a little bit easier.

Principle 2: Embrace Unscalable Work

This might sound like heresy to data professionals, but it’s Stancil’s most powerful point. He tells the story of a friend tasked with analyzing the sentiment of articles on a specific topic. She started building a complex AI model, only to realize there were just seven articles.

"Why am I building a tool to look at seven articles?" Stancil asks. "Go read them." It takes 20 minutes and yields a far richer understanding than any model could. This "unscalable" approach is incredibly effective for customer data. Instead of trying to find trends across thousands of users in a noisy dataset, go look at the raw activity of a single user.

With a tool like MotherDuck, you don't need a complex pipeline to do this. You can query your raw event data directly to "read the story" of an individual user's journey. For example, let's say you want to understand what your most active user from the last week was actually doing.

Copy code
-- Stancil's point: Sometimes the best insight comes from looking at one customer.
-- With MotherDuck, you don't need a complex pipeline to do this.
-- Just query your raw data directly.

-- First, find our most active user this week
WITH user_activity AS (
    SELECT
        user_id,
        COUNT(event_id) AS event_count
    FROM events
    WHERE event_timestamp >= NOW() - INTERVAL '7 day'
    GROUP BY 1
    ORDER BY 2 DESC
    LIMIT 1
)
-- Now, let's pull their entire event stream to "read the story" of their session
SELECT
    e.event_timestamp,
    e.event_type,
    e.properties
FROM events e
JOIN user_activity ua ON e.user_id = ua.user_id
ORDER BY e.event_timestamp;

The result of this query isn't a high-level chart; it's a narrative. You can see every click, every page view, every action this power user took, step-by-step. This is the "unscalable" insight Stancil talks about. It doesn't tell you what all users are doing, but it gives you a deep, qualitative understanding of what an engaged user's journey looks like. That's often far more valuable than another "up-ish" chart.

Principle 3: Honestly Assess Your Data's Scale

Stancil's final piece of advice is to be honest about the scale of your data. Don't use a "big data" sledgehammer for a "small data" nail. Big data problems are real, and they require big data tools like Snowflake or BigQuery. If you're managing petabytes of data for a global enterprise, those are the right tools for the job.

But MotherDuck and DuckDB were built for the other 99% of us. We excel in the vast space below that massive threshold, where most companies operate and where the challenges are different. It's not about wrangling petabytes; it's about getting fast, reliable insights from datasets that fit on your laptop or a modest server. It's about using the right tool for the job.