---
title: "data pipeline"
description: "A data pipeline is a series of interconnected processes that extract data from various sources, transform it into a usable format, and load it into a…"
canonical: "https://motherduck.com/glossary/data-pipeline/"
related:
  - title: "Agentic Data Engineering: Build Pipelines End-to-End with AI | MotherDuck"
    url: "https://motherduck.com/videos/agentic-data-engineering-pipelines-ai/"
  - title: "Data Warehousing How-to | MotherDuck Docs"
    url: "https://motherduck.com/docs/key-tasks/data-warehousing/"
  - title: "What if a full data pipeline was one prompt away?"
    url: "https://motherduck.com/blog/full-data-pipeline-one-prompt/"
gated_asset:
  title: "MotherDuck Flights: Agent-Native Data Pipelines"
  url: "https://motherduck.com/product/flights/"
---

# data pipeline

> A data pipeline is a series of interconnected processes that extract data from various sources, transform it into a usable format, and load it into a…

A [data pipeline](https://motherduck.com/learn/what-is-data-ingestion-pipeline/) is a series of interconnected processes that extract data from various sources, transform it into a usable format, and load it into a destination system for analysis or storage. It automates the flow of data through different stages, ensuring data is cleaned, validated, and prepared for consumption by downstream applications or users. Modern data pipelines often incorporate tools like [Apache Airflow](https://airflow.apache.org/) for orchestration, [dbt](https://www.getdbt.com/) for transformation, and [Fivetran](https://www.fivetran.com/) for data extraction. These pipelines can handle both batch and real-time data processing, enabling organizations to make data-driven decisions based on up-to-date information. In the context of DuckDB, you might create a simple data pipeline using SQL to extract data from a CSV file, transform it, and load it into a table:

```sql
-- Extract data from CSV
CREATE TABLE raw_data AS SELECT * FROM read_csv_auto('data.csv');

<glossary-callout video="agentic-data-engineering-pipelines-ai" />

-- Transform data
CREATE TABLE transformed_data AS 
SELECT 
    id,
    UPPER(name) AS name,
    CASE 
        WHEN age < 18 THEN 'Minor'
        ELSE 'Adult'
    END AS age_category
FROM raw_data;

-- Load data into final table
CREATE TABLE final_table AS SELECT * FROM transformed_data;
```

This example demonstrates a basic Extract, Transform, Load (ETL) process within DuckDB.



**Watch:** [Why use DuckDB in your data pipelines ft. Niels Claeys](https://motherduck.com/videos/why-use-duckdb-in-your-data-pipelines-ft-niels-claeys/)