Skip to main content

Building data apps

Data apps are interactive tools designed to offer insights or automate actions using data. Examples include data visualizations and custom reporting tools for business groups.

You might also come across "embedded analytics." This concept brings the kind of data exploration previously confined to dashboards and traditional BI (Business Intelligence) tools right into the software that teams and customers already use.

For an interactive exploration of the capabilities offered by DuckDB Wasm and Motherduck, we invite you to experience our live demo here. Please note that while the data visualization serves as a medium, the focus is on the underlying technology's features.

How to Develop Data Apps?​

Building data apps involves a blend of data handling, storage, and visualization technologies. The development process usually includes:

  • Data Processing: The initial phase involves gathering, cleaning, and formatting data so it's ready for the app.
  • Data Storage: Processed data is then stored in a database or data warehouse to ensure the app can quickly and efficiently fetch the data when needed.
  • Data Visualization: An essential feature of data apps is visualization (Streamlit, Mosaic, D3js, Chartjs, etc), enabling users to see and understand data patterns. As data apps are embedded into the software that teams and customers already use, this often involves JavaScript libraries like D3.js, Chart.js, or Mosaic. Note that Python libraries like Streamlit or Dash are also popular for data apps.

hld

Challenges in Data App Development​

Creating data apps comes with its hurdles, mainly due to data's complexity. Here are two typical challenges:

  1. Data Source Responsiveness: Data apps often need real-time data, which can be tough to swiftly process and analyze.
  2. Data Infrastructure Load: Data apps can demand a lot from data infrastructure, particularly if vast amounts of data need processing and storage.

Typically, after it has been processed, the useful data often finds its home in a cloud data warehouse, an OLAP system. Although you could query your OLTP database (such as Postgres), where the original data is often stored, this approach can exert significant query processing pressure on a system not designed for such tasks. OLAP systems like cloud data warehouses performs better than OLTP for the kinds of queries (e.g. filters, aggregations) that data apps need to run. That being said, some applications have latency requirements which make it hard to query a cloud data warehouse directly. To mitigate this, it's common practice to establish a caching layer. With the above proposed architecture, we get the best of both worlds: the data is stored in a cloud data warehouse, and the data app can query a local cache (DuckDB) for low-latency queries.

Architecture of Data Apps​

MotherDuck's novel Wasm-Powered 1.5-tier architecture​

DuckDB can run anywhere, including in the browser thanks to WebAssembly (Wasm). Wasm is a technology that lets you run code from languages like Rust or C++ in web browsers, making web apps run faster and more efficient alongside JavaScript. Using DuckDB Wasm, client-side JavaScript can process data locally, enabling faster analytics experiences. Once the data is moved to the local context, many user interactions won't require any communication with the cloud, resulting in lower latency and quicker performance for the user. Additionally, less cloud computing translates to lower costs for the developer.

We call 1.5-tier architecture because it combines the client (1-tier) that now has also a part of the database. This mean that the other part of the database is in the cloud, hence the 1.5-tier architecture.

wasm

3-tier architecture​

A 3-tier architecture powers the vast majority of applications today. Managing integrations and updates between the client, server, and database is time-consuming and unwieldy. For users, multiple steps between them and the data may slow performance and speed at scale. In this case, the server will use a DuckDB client to then connect to Motherduck.

3tier

Getting started with data apps and MotherDuck​

To get started with data apps and MotherDuck, check out our MotherDuck Wasm client documentation.