The Surprising Birth Of DuckDB ft. Co-creator Hannes Mühleisen

The Origin Story

At CWI (Dutch Research Institute for Computer Science), Hannes worked in the Database Architectures group researching how data systems should be built. A surprising discovery changed everything.

"They Weren't Using Databases at All"

When talking to the R community (data practitioners doing serious analytical work):

• They stored data in CSV files
• Some had hand-rolled data frame engines
• Everything was slow and limited
• They really didn't like databases

Hannes was confused: "I really love databases. Why aren't they using our stuff?"

The Two Problems

A New Category is Born

"It turns out we invented a whole new class of database systems: the in-process analytics thing."

SQLite is amazing but designed for transactional workloads, not analytical ones. This category didn't exist.

The Prototype: MonetDB Lite

First attempt: Hack an existing system (MonetDB) into an in-process analytical database.

Result: Worked well enough to get people excited, but hit architectural issues:

• Can't just exit the process on errors (you're inside someone else's program)
• Can't use APIs that change global state (working directory, locale)
• "Really trivial stuff, but it adds up"

Starting From Scratch

Hannes and Mark Raasveldt made a bold decision: build a completely new database system.

The risk:

• "Monumental undertaking"
• "Not successful 99% of the time"
• "Not great for your career as a researcher—you disappear for 5 years not writing papers"

The advantage: They already had product-market fit from the prototype. They knew what people wanted.

The Grind

"Mark and I basically did nothing else—evenings, weekends—just hacking on DuckDB for a couple of years."

SQL is complex. Supporting all the operators, edge cases, and details took massive effort.

Open Source: 2019

Immediate positive feedback. The design resonated because it came from real user problems, not theoretical research.

The Lesson

"My definition of success is not to write papers—it's to have impact. In data systems research, you have impact by making systems."

The R community's willingness to try "crazy stuff that was barely working" and give honest feedback was essential. This kind of practitioner collaboration is rare in academia—but it's why DuckDB exists.