Making Big Data Feel Small

Ryan Dolley

ryandolley

MotherDuck ... bringing big innovation to SQL to make it more like a real language and less like a 'groovy ancient programming language from the 70s.' Check it out.

Robert Lancer

robertlancer

The progress on the MotherDuck UI is amazing! It went from minimal utility to my preferred place to write SQL in a matter of months

Caleb Fahlgren

@calebfahlgren

The amount of data I am storing in @motherduck rn for a new project is criminal 🤣 Decided to try it out since I am a big @duckdb user. Is pretty good so far. Have few hundred million records.

Simon Späti 🏔️

@sspaeti

I'm just checking a little about the hosted DuckDB by @motherduck. What a nice user experience. It automatically detects errors, and `accept & run` fixes common errors. Love it.

Jacob Matson

@matsonj

using sql for data cleaning vs excel (reason 879 to learn sql) levenstein distance between 590strings in excel (350k combinations): excel: 25 mins of spinning. duckdb: 1.2s

Bob Currier

bobcurrier

If you're not familiar with DuckDB, you definitely want to give it a look. I've recently converted all my existing code from using CSV files and SQLite databases for exploratory analysis to parquet files, DuckDB and MotherDuck. The transformation in my workflow has been nothing short of astounding. 10GB of CSV files turned into 1GB of parquet files that are now stored in S3 buckets. Using DuckDB and MotherDuck, I can run exploratory queries seamlessly thanks to the DuckStack with dual execution with local processing and cloud scale. My query development time has decreased by 90%. And I'm loving the team feature of the MotherDuck platform. OCEANCODA is rocking the small data life! #duckdb #python #motherduck #eda #ai #ml (Graphic courtesy of Manning Publications Co. book 'DuckDB in Action.' ) Check it out.

evidence

@evidence_dev

DuckDB is increasingly being built into every data tool, but DuckDB isn't designed to be used as a cloud data warehouse @motherduck is solving that problem, and they’re doing it in a delightful, uniquely DuckDB way

Ethan

@@ethanf_17

DuckDB is my new best friend for analytics. It's so fast, makes it easy to manipulate dataframes, ingests anything, and exports it to whatever form I want. Parquet, Arrow dataframes, and DuckDB are the future for analytics.

uwe geercken

@uweeegeee

#duckdb is really impressive: good terminal client, solid, easy and fast import of csv data and impressive speed on a 20mio data set. With automatic schema creation. Easy export to parquet and more. I think I am gonna spend more time on it...

Etienne Posthumus

@epoz

I have been kicking the tyres of @motherduck - and it is 🤯 Being able to do hybrid queries over local and humongous remote datasets are incredible. The potential applications are making my head spin with awe and excitement. Well done.

Jamin Ball

@jaminball

Another awesome & insightful post from the @motherduck team. The last few decades have been about scale-out. The next decade will be about scale-up! What you can accomplish with a single machine these days is incredibly impressive

Danny Blumenthal

dpblumenthal

Our friends at MotherDuck stand out in the data space for how consistently they keep shipping awesome new features.

Nico Ritschel

@nicoritschel

I just deployed to Rill cloud backed by Motherduck and it worked great— wonderful experience... In-browser CLI auth with Rill Cloud, added my Motherduck token, invited some viewers, I'm live. Took 3 minutes.

MotherDuck

@motherduck

The statistics from the 2023 StackOverflow survey have been released, and @duckdb has been recognized in several categories, including a noteworthy position among the top 30 most popular databases 🎉

Damon Cortesi

@dacort

Because, I don't have enough time on my hands, I created a DuckDB extension that allows you to import data by scanning an Athena table. https://github.com/dacort/duckdb-athena-extension

George Fraser

@frasergeorgew

How fast is DuckDB compared to the best commercial data warehouses? I decided to benchmark it myself. Short version: very fast! But it's not (yet) great at scaling up to many cores. https://fivetran.com/blog/how-fast-is-duckdb-really

MotherDuck

@motherduck

@duckdb is empowering machine learning use cases! 🤖 Any good model training starts with some good data exploration. @huggingface now enables SQL queries using DuckDB on any dataset stored on the Hub!

DuckDB

@duckdb

DuckDB Co-Creator @hfmuehleisen and MotherDuck Founder @jrdntgn in this new podcast - among other things about the origin story of DuckDB and later MotherDuck:

Garrett McClintock

@GJMcClintock

just built a pipeline using @dltHub and @motherduck and it's very nice. 10/10 experience once you get the hang of it and very scalable.

Andrew Madson

andrew-madsen

Instant SQL! This is one of the highlights of Data Council... Instant SQL caches a sample of your table instantly, and synchronously updates the query results, AS YOU TYPE!

KKovacs [ス]

@kkovacs

I don’t want to say that #DuckDB is a silver bullet because there is no such thing, but it’s surprisingly projectile-shaped, very shiny, and killed all the various werewolves (ok, data problems) I’ve thrown at it

evidence

@evidence_dev

DuckDB is increasingly being built into every data tool, but DuckDB isn't designed to be used as a cloud data warehouse @motherduck is solving that problem, and they’re doing it in a delightful, uniquely DuckDB way...

Ryan Dolley

ryandolley

MotherDuck ... bringing big innovation to SQL to make it more like a real language and less like a 'groovy ancient programming language from the 70s.' Check it out.

Robert Lancer

robertlancer

The progress on the MotherDuck UI is amazing! It went from minimal utility to my preferred place to write SQL in a matter of months

Caleb Fahlgren

@calebfahlgren

The amount of data I am storing in @motherduck rn for a new project is criminal 🤣 Decided to try it out since I am a big @duckdb user. Is pretty good so far. Have few hundred million records.

Simon Späti 🏔️

@sspaeti

I'm just checking a little about the hosted DuckDB by @motherduck. What a nice user experience. It automatically detects errors, and `accept & run` fixes common errors. Love it.

Jacob Matson

@matsonj

using sql for data cleaning vs excel (reason 879 to learn sql) levenstein distance between 590strings in excel (350k combinations): excel: 25 mins of spinning. duckdb: 1.2s

Bob Currier

bobcurrier

If you're not familiar with DuckDB, you definitely want to give it a look. I've recently converted all my existing code from using CSV files and SQLite databases for exploratory analysis to parquet files, DuckDB and MotherDuck. The transformation in my workflow has been nothing short of astounding. 10GB of CSV files turned into 1GB of parquet files that are now stored in S3 buckets. Using DuckDB and MotherDuck, I can run exploratory queries seamlessly thanks to the DuckStack with dual execution with local processing and cloud scale. My query development time has decreased by 90%. And I'm loving the team feature of the MotherDuck platform. OCEANCODA is rocking the small data life! #duckdb #python #motherduck #eda #ai #ml (Graphic courtesy of Manning Publications Co. book 'DuckDB in Action.' ) Check it out.

evidence

@evidence_dev

DuckDB is increasingly being built into every data tool, but DuckDB isn't designed to be used as a cloud data warehouse @motherduck is solving that problem, and they’re doing it in a delightful, uniquely DuckDB way

Ethan

@@ethanf_17

DuckDB is my new best friend for analytics. It's so fast, makes it easy to manipulate dataframes, ingests anything, and exports it to whatever form I want. Parquet, Arrow dataframes, and DuckDB are the future for analytics.

uwe geercken

@uweeegeee

#duckdb is really impressive: good terminal client, solid, easy and fast import of csv data and impressive speed on a 20mio data set. With automatic schema creation. Easy export to parquet and more. I think I am gonna spend more time on it...

Etienne Posthumus

@epoz

I have been kicking the tyres of @motherduck - and it is 🤯 Being able to do hybrid queries over local and humongous remote datasets are incredible. The potential applications are making my head spin with awe and excitement. Well done.

Jamin Ball

@jaminball

Another awesome & insightful post from the @motherduck team. The last few decades have been about scale-out. The next decade will be about scale-up! What you can accomplish with a single machine these days is incredibly impressive

Danny Blumenthal

dpblumenthal

Our friends at MotherDuck stand out in the data space for how consistently they keep shipping awesome new features.

Nico Ritschel

@nicoritschel

I just deployed to Rill cloud backed by Motherduck and it worked great— wonderful experience... In-browser CLI auth with Rill Cloud, added my Motherduck token, invited some viewers, I'm live. Took 3 minutes.

MotherDuck

@motherduck

The statistics from the 2023 StackOverflow survey have been released, and @duckdb has been recognized in several categories, including a noteworthy position among the top 30 most popular databases 🎉

Damon Cortesi

@dacort

Because, I don't have enough time on my hands, I created a DuckDB extension that allows you to import data by scanning an Athena table. https://github.com/dacort/duckdb-athena-extension

George Fraser

@frasergeorgew

How fast is DuckDB compared to the best commercial data warehouses? I decided to benchmark it myself. Short version: very fast! But it's not (yet) great at scaling up to many cores. https://fivetran.com/blog/how-fast-is-duckdb-really

MotherDuck

@motherduck

@duckdb is empowering machine learning use cases! 🤖 Any good model training starts with some good data exploration. @huggingface now enables SQL queries using DuckDB on any dataset stored on the Hub!

DuckDB

@duckdb

DuckDB Co-Creator @hfmuehleisen and MotherDuck Founder @jrdntgn in this new podcast - among other things about the origin story of DuckDB and later MotherDuck:

Garrett McClintock

@GJMcClintock

just built a pipeline using @dltHub and @motherduck and it's very nice. 10/10 experience once you get the hang of it and very scalable.

Andrew Madson

andrew-madsen

Instant SQL! This is one of the highlights of Data Council... Instant SQL caches a sample of your table instantly, and synchronously updates the query results, AS YOU TYPE!

KKovacs [ス]

@kkovacs

I don’t want to say that #DuckDB is a silver bullet because there is no such thing, but it’s surprisingly projectile-shaped, very shiny, and killed all the various werewolves (ok, data problems) I’ve thrown at it

evidence

@evidence_dev

DuckDB is increasingly being built into every data tool, but DuckDB isn't designed to be used as a cloud data warehouse @motherduck is solving that problem, and they’re doing it in a delightful, uniquely DuckDB way...