Singer (specification)
Singer is an open-source specification that defines how independent scripts, called taps and targets, communicate data and state using a simple JSON format over stdout and stdin.
Overview
Singer is an open specification, not a product, for moving data between systems. It defines a common protocol so that a data extraction script (a tap) and a data loading script (a target) can be written independently, by different people, in any language, and still work together as long as both follow the spec. Singer was created by Stitch (the company behind the Stitch Data Loader), which was acquired by Talend in 2018; Talend was in turn acquired by Qlik in 2023. The specification itself remains open source and is used by Meltano's ecosystem of community-maintained taps and targets, independent of any single vendor.
How it works
A Singer tap reads from a source and writes newline-delimited JSON messages to stdout. A target reads that same stream from stdin and writes it to a destination. Three message types carry the pipeline's contract:
- SCHEMA: describes the structure of an upcoming stream of records.
- RECORD: an individual row of data.
- STATE: a bookmark (e.g., an updated_at cursor or primary key) that lets the next run resume incrementally instead of reprocessing everything.
Copy code
# a tap and target composed with a standard pipe
tap-postgres --config config.json | target-jsonl --config target-config.json
Copy code
{"type": "RECORD", "stream": "orders", "record": {"id": 101, "amount": 42.50, "updated_at": "2024-01-15T00:00:00Z"}}
{"type": "STATE", "value": {"bookmarks": {"orders": {"updated_at": "2024-01-15T00:00:00Z"}}}}
Why it matters
Singer's stdout/stdin, JSON-based design means a tap and target never need to know about each other's implementation, only the shared message format — which is what let an ecosystem of community-built taps and targets grow independent of Stitch as a company. Meltano is the most active current steward of this open ecosystem, publishing and running Singer plugins through MeltanoHub. Since a Singer target can write to plain files or a database, a common pattern is a Singer/Meltano pipeline landing data as JSON or Parquet that is then queried directly with DuckDB, or loaded onward into MotherDuck for shared analytics.
Related terms
Meltano is an open-source ELT platform built on the Singer specification, letting teams declare data pipelines as code using YAML configuration and a CLI.
JSON →JSON (JavaScript Object Notation) is a lightweight, text-based data interchange format that's easy for humans to read and write, and simple for machines to…
data ingestion →Data ingestion is the process of importing raw data from various sources into a system where it can be stored and analyzed.
Data contract →A data contract is a formal, agreed-upon specification between a data producer and its consumers that defines the schema, format, semantics, and guarantees (like update frequency or backward-compatibility rules) of a dataset.
Table format →A table format is a metadata layer on top of data files in a lake or object storage that defines what constitutes a table — its schema, partitioning, and file list — enabling ACID transactions, schema evolution, and time travel across multiple query engines.
FAQS
The specification is stable and widely implemented, but its original commercial steward, Stitch, has been in maintenance mode as part of Qlik since the 2023 acquisition; the open-source tap/target ecosystem is now primarily carried forward by the Meltano community.
No. Taps and targets are just scripts that read/write the Singer JSON protocol; Meltano is a convenient way to configure, schedule, and run them, but they can be piped together manually or driven by other tools.
All three extract data from a source, but a Singer tap is a standalone script implementing an open, language-agnostic protocol, while Fivetran and Airbyte connectors are built and run within each platform's own managed framework.
