GUIDE, read 10 Nov 2026
OpenLineage is an open event spec that running jobs emit, and Marquez is the reference backend that stores them
OpenLineage is a specification, not a product
OpenLineage describes itself as an open framework for data lineage collection and analysis, built on an extensible specification. Its repository README says it is an LF AI & Data Foundation Graduate project under the Apache 2.0 license. On 10 Nov 2026 the docs site showed 1.55.0 as the current documentation version.
The design point is that a job reports its own lineage as it runs. OpenLineage is designed to record metadata for a job in execution, so collection is emitted by the running integration rather than curated by hand. The consumer receives events and decides what to do with them: store them, draw a graph, or merge them into a catalog.
That makes OpenLineage a transport and a vocabulary, much like a log format that many tools can write and many can read. It is not a place you log into to look at lineage. For that you need a consumer, and Marquez is the reference one.
Facets carry the detail
The spec stays small and adds detail through facets, which are extensions attached to jobs and datasets. One facet matters most for lineage work.
- columnLineage is a dataset facet that records which input columns are used to produce which output columns. It carries a per-column transformation type and subtype, for example DIRECT with subtype TRANSFORMATION or IDENTITY, and a masking flag. That describes the kind of transformation, not the SQL expression itself.
- Where the expression lives. Because the facet describes transformation kind and not the expression, a consumer that wants the SQL must get it from elsewhere. Tools that read the SQL themselves, such as warehouse query-log miners, can fill that gap.
Who emits events
Emitters are integrations that sit inside the tool that runs the job. The OpenLineage dbt page lists two modes, the Artifact Processor and the Structured Log Processor, and its integrations sidebar names Airflow, Flink, Hive, Spark, dbt, Feast, Great Expectations, Presto and Trino. Marquez states its API server works with the integrations for Apache Airflow, Apache Spark, Apache Flink, dbt and Dagster.
| Emitter | How it produces lineage | Notes from the docs |
|---|---|---|
| Apache Spark | Column-level lineage derived from Spark logical plans | On by default |
| dbt | Parses manifest.json, run_results.json and optionally catalog.json after a run | Column data types come from catalog.json |
| dbt (log mode) | Structured log processor extracts lineage while dbt runs | Real time |
| Airflow, Flink, Dagster | Named integrations | Details not read for this guide |
| Hive, Feast, Great Expectations, Presto, Trino | Named in the integrations sidebar | Details not read for this guide |
What Marquez adds
Marquez is the reference implementation of the OpenLineage API, and its README describes it as an LF AI & Data Graduated project. Marquez collects lineage events in real time from running jobs through an OpenLineage-compatible endpoint, and exposes a lineage API so metadata can be queried programmatically, for example to enrich catalogs and data quality systems.
For a team that wants a working lineage store with no license fee, this is the route: run the emitters and point them at Marquez. Both OpenLineage and Marquez are under Apache 2.0, so no license fee applies, and no hosted price was found on 10 Nov 2026.
Which products ingest OpenLineage events
The larger story is that other products accept the events. A team can emit once and let several tools consume.
| Consumer | What its docs say about OpenLineage |
|---|---|
| Snowflake | Accepts OpenLineage events over a REST endpoint so tools like dbt and Airflow can add external lineage |
| DataHub | Ingests OpenLineage events through a REST endpoint and a Spark event listener plugin; Airflow, Dagster and Prefect lineage pages exist |
| Atlan | Supports OpenLineage for Spark, Airflow and Flink with real-time column lineage for SQL and relational sources |
| Alation | Supports OpenLineage events from Spark and Trino, matching them to cataloged sources |
| Collibra | Supports automated OpenLineage ingestion |
What the spec leaves out
Several things are absent from the OpenLineage and Marquez pages read, and the gaps define when you need something else.
- BI-field or metric lineage: none found on 10 Nov 2026 in the core spec or the integrations pages read (docs home, dbt, Spark, Marquez site).
- Propagation of classifications or governance rules along lineage: none found. The facets describe metadata per entity, and propagation is left to consuming catalogs.
- An MCP server or agent-specific API: none found on the docs home, the Marquez site or the READMEs.
- Security certifications, audit logging or a permission model: none found on the pages read.
Where OpenLineage ranks and why
Upstream's published total for OpenLineage and Marquez is 36.0, the lowest of the 16 tools, against Euno at 67.0, DataHub at 62.5 and Secoda at 61.5. Euno is cloud SaaS only and publishes no dollar prices; a team that needs lineage inside its own network, or a full catalog around it, will pair it with another tool. Upstream scores it as a lineage tool, so a spec scores low where a product would offer BI reach, propagation and an agent interface. Its documented strength is that events come from the running job rather than from hand curation.
The right comparison is not OpenLineage versus a catalog. It is OpenLineage plus a consumer versus a catalog that has its own connectors.
When to choose it
Start with the job that breaks most often. If that is an Airflow DAG, install its integration first, point it at Marquez or at a consumer from the table above, and run it for a week. Check three things at the end of the week: whether every dataset you expected appears, whether column edges exist for your SQL jobs, and whether the run history lines up with what your scheduler says happened. A missing dataset usually means a job type the integration does not cover. Missing column edges usually mean the job's SQL could not be parsed into a column mapping.
The fastest route to a useful result is often one consumer plus one emitter. For a dbt shop, that means the dbt integration plus a tool from the consumer table. For a Spark shop, it means the Spark integration, where column-level lineage is on by default. In both cases the question to answer first is whether the consumer shows the column edges the emitter sent, since a consumer may keep only table-level edges from an event.
Add emitters one job type at a time. Adding all of them at once makes a gap hard to attribute.
Use OpenLineage when you run Spark, Airflow or dbt and want the lineage emitted at the source in an open format. Add a consumer that reaches the BI layer and carries meaning along the path. Do not expect the spec to do either. The OpenLineage page and the method hold the evidence and scoring.
Sources
- OpenLineage: openlineage.io/docs, read 10 Nov 2026
- OpenLineage: raw.githubusercontent.com/OpenLineage/OpenLineage/main/README.md, read 10 Nov 2026
- OpenLineage: raw.githubusercontent.com/MarquezProject/marquez/main/README.md, read 10 Nov 2026
- OpenLineage: openlineage.io/docs/spec/facets/dataset-facets/column_lineage_facet, read 10 Nov 2026
- OpenLineage: openlineage.io/docs/integrations/spark/spark_column_lineage, read 10 Nov 2026
- OpenLineage: openlineage.io/docs/integrations/dbt, read 10 Nov 2026
- OpenLineage: marquezproject.ai, read 10 Nov 2026
- Snowflake: docs.snowflake.com/en/user-guide/external-lineage.md, read 10 Nov 2026
- DataHub: docs.datahub.com/docs/lineage/openlineage, read 10 Nov 2026
- Atlan: docs.atlan.com/product/capabilities/lineage/best-practices/capture-and-troublesh, read 10 Nov 2026
- Alation: www.alation.com/blog/alation-data-lineage-updates, read 10 Nov 2026
- Collibra: www.collibra.com/products/data-lineage, read 10 Nov 2026
Last reviewed 10 Nov 2026.