River
Y CombinatorBacked by Y Combinator
Log InStart Free
FREE TEMPLATE

Data Pipeline Documentation Template

A contract that records every threshold your project declares beside what actually happens when it is violated, plus a blueprint per pipeline.

Edit with AIDownload (.docx + .csv)

Free download  ·  No account needed

What you get

Pack of 8 files: 5 documents, 3 spreadsheets

Data Pipeline Documentation Template
  • Reading a Declared ThresholdDocument
  • Ownership and Response TimesDocument
  • Pipeline BlueprintDocument
  • Error Handling GuideDocument
  • Worked Pipeline subscription_revenueDocument
  • Data ContractSpreadsheet
  • Dependency and Consumer RegisterSpreadsheet
  • Failure Runbook IndexSpreadsheet
RiverData Pipeline Documentation Template

Pipeline Blueprint

One document per pipeline that produces a number somebody reads

The test of a finished blueprint: hand it to somebody at eight in the morning with a screenshot and a question, and they get to an answer without opening the orchestrator.

What it reads, including the inputs nothing declares

Every input at the grain it arrives in. The seed file nobody has edited in nineteen months, the mapping table somebody maintains by hand, the list of plan codes hardcoded inside the transformation. None of those appear in a generated lineage graph, and they are where the numbers actually go wrong.

The transformations that are not obvious

Not a restatement of the SQL. Which rows are excluded and why, how a record arriving three days late is attributed, which timezone a day boundary uses, what the deduplication keys on, and every hardcoded value listed individually.

What is not checked at all

The explicit list: volume, freshness on the output rather than the input, referential integrity across a join, plausibility of a total. Naming a gap is documentation. Leaving it out is a promise nobody made, and a blueprint that says the pipeline is tested is worse than one that says nothing, because it stops the reader looking.

How to tell whether today's number is right

The numbered procedure, cheapest check first, naming which passing checks mean nothing for this pipeline specifically. Did the run happen at all, did the output rebuild or only the input arrive, is the volume plausible for this weekday, do the parts add up to the whole. This is the section people actually open.

Who to tell when it is wrong

The consumer list, with the undeclared consumers marked. Whoever reads the number from a scheduled export or a workbook connection will never see an incident banner, and they are usually the ones presenting it.

An on-call analyst never asks how a pipeline works. They ask whether the number is right, and every indicator being green answers less than it looks. A data test's severity defaults to error, and setting it to warn makes the tool skip the error condition entirely. The test can then return ten thousand null revenue rows, print a warning, and let every model downstream build on top of them. The threshold is declared. Nothing is enforced.

Three more mechanisms do the same thing. An expectation takes a tolerance defaulting to one, and its own worked example sets it to 0.66, so a column documented as not null passes while a third of it is empty. Freshness is declared on sources, so a stale mart on a fresh source is green everywhere. And task-level service level agreements were removed outright in Airflow 3, leaving the parameter sitting in the code raising nothing.

So the data contract carries two threshold columns. One transcribes what the project literally says, parameters and severity intact; the other records what happens on violation, and a third names the mechanism where they differ. Filter it to anything but agrees and you have the numbers nobody can stand behind. It is the same shape as the register that reconciles scanner severities, and where the transformations themselves are undocumented, reading them out of the repository comes first. Where the pipeline is being replaced rather than documented, the warehouse migration pack covers that comparison instead.

Eight checks, four of which enforce nothing

The Data Contract, the silent failure it predicted, and the consumer register that says who read the wrong number.

Data Pipeline Documentation Template
RiverData Contract

Data Contract

Illustrative rows for a fictional subscription business, Brightwater. Every mechanism named is what the tools actually do.

CheckThreshold, as declaredEnforcedReconciliation
unique on subscription and monthunique, severity errorfails the runagrees
not null on recognised revenuenot_null, severity warnnothingdeclared only: severity is warn
not null on account idexpect not null, tolerance 0.66fails above 34% nulldeclared only: tolerance permits failures
accepted values on plan codeaccepted_values, error_if > 100fails above 100 rowsdeclared only: threshold is conditional
freshnesswarn_after 12h, error_after 24h, on the sourcethe source onlyenforced elsewhere: not this model
task must finish by 07:00sla of 2 hours on the tasknothingnot enforced: mechanism removed
row count on the outputnonenothingno threshold declared
not null on order idNOT NULL column constraintthe load failsagrees, enforced outside the project

Four of these eight are green in every report and enforce nothing. Row two is the commonest and its history is always ordinary: somebody set the test to warn during a migration to unblock a release, and nobody set it back. Since then the pipeline has been free to publish null revenue rows, and it has.

Row seven is the one nothing generates for you. Every other row starts from something written in the project. That one starts from a number on a dashboard and works backwards to find that nobody ever said what a plausible volume looks like, so a load of four rows instead of four hundred thousand passes every check it has.

Declared and enforced in separate columns. Overwrite the first with the second and a warning-only check reads as coverage forever.

RiverGreen and Wrong

Green and wrong

The failure the contract above predicted three weeks before it happened. The number was stale for two days and nothing alerted.

SignalWhat it saidWhat was true
source freshnesspassThe raw tables did arrive on time. This check was correct and answered a different question.
every data testpassCorrect, on yesterday's rows. The model never rebuilt, so the tests examined data that was already there.
revenue null checkpassMeaningless. Severity is warn, so it cannot fail whatever it returns.
task deadlineno resultThe parameter was removed in a major version upgrade. It has raised nothing since.
output row countno checkWould have caught it in one query. Never existed.
output freshnessno checkWould have caught it immediately. Freshness was only ever on the source.

Every indicator was green and the dashboard was two days stale. The run was failing at a step marked to continue on error, so the orchestrator reported success. Three separate contract rows had already said that none of the passing signals covered this case.

The fix was nine lines of configuration. A freshness rule on the output and a volume band by weekday. It took three weeks rather than three years because somebody could see the gap in a column instead of inferring it from an absence.

Loud failures are the cheap ones. This is the other kind, and it is the kind that reaches a board pack.

RiverConsumers

Dependency and Consumer Register

Who reads the number, and how each one was found. The project knew about one of the three.

ConsumerTypeFound fromDeclaredSees an incident notice
Weekly revenue reviewdashboardproject lineageYesYes
Board pack slidescheduled exportwarehouse query historyNoNo
Finance workbook, live connectionspreadsheetasked a humanNoNo
Downstream staging modelmodelproject lineageYesn/a
nothing, 90 daysmodelwarehouse query historyNocandidate for removal

The workbook was the highest-stakes reader of the pipeline. It had been pulling a number for eleven months, through both null-revenue episodes, with nobody on any notification list. It appears in no lineage graph, and the only reason it is in this register is that somebody asked.

Undeclared consumers are why an impact analysis comes back clean. A breaking change ships against the declared graph, the export starts carrying wrong numbers, and nothing connects the two. Query history over ninety days finds the scheduled ones; the spreadsheets need a conversation.

The last row is the only one that saves money. A model still rebuilding on every schedule with nothing reading it for ninety days, long enough that a quarterly report would have shown up.

A dashboard can carry a banner. A workbook cannot, so it needs a named human.

Edit with AI

What's in the pack

01

Data Contract

One row per check, carrying what the project declares beside what happens on violation, and a reconciliation column naming the mechanism where those two disagree.

02

Dependency and Consumer Register

Who reads each number, filled from query history and from asking people rather than from lineage alone, marking which consumers can never see an incident notice.

03

Failure Runbook Index

Keyed on the symptom rather than the pipeline, because nobody on call is handed the name of a failing model. They are handed a screenshot and a question.

04

Reading a Declared Threshold

The four mechanisms that let a threshold exist and enforce nothing, each read off the tool that causes it, and the fixed vocabulary the contract uses instead.

05

Pipeline Blueprint

The template, annotated inline, including the transformations that are not obvious, the explicit list of what is not checked, and the procedure for judging today's number.

06

Ownership and Response Times

The eight questions a team answers once: who owns what, when the clock starts, and what a consumer who cannot see an incident channel is entitled to.

07

Error Handling Guide

Triage in order of cost, separating the loud failure from the silent one, because everything is green is a different incident from everything is red.

08

Worked Pipeline subscription_revenue

One pipeline end to end: fourteen checks, four of them enforcing nothing, and the two-day stale dashboard its contract predicted three weeks in advance.

How to use it

  1. 1

    Open in River, or take it blank

    Claim the pack in River and give it the project, or take the blank documents and registers and fill them in by hand.

  2. 2

    Answer the ownership questions

    Who owns what, when the response clock starts, and what happens to a threshold that enforces nothing. Every register column checks against those answers.

  3. 3

    Send the project

    A dbt project or just its schema files, a DAG directory, a warehouse schema dump, an expectation suite, or a screenshot of the dashboard nobody trusts.

  4. 4

    Read the rows that do not agree

    The numbers standing on an unenforced threshold, ordered by who reads them. The weekly sweep reports any threshold that gets quietly weakened.

Frequently asked questions

Is this template free?

Yes. Download it and nothing asks for a card, an email or an account. Edit with AI is optional and it is where the work happens: the agent reads whatever project you have and fills the contract, including rows for checks nobody ever wrote. Other packs sit in the template library.

What format are the downloaded files?

Everything arrives as one zip of standard files: the documents in .docx, the three registers in .csv. There is no importer, no proprietary format and no account needed to read any of it. The illustrative rows stay in the sheets so each column shows what it expects.

We already have generated docs from our transformation tool. Why this?

Generated documentation describes what the code says. It cannot tell you that a test set to warn enforces nothing, that a freshness rule covers a source rather than the model anybody queries, or that a hardcoded mapping stopped matching reality nineteen months ago. Those are absences, and nothing generates an absence. A test on the wrong column is the gap the analytics standards pack flags.

Does it work if we do not use dbt?

Yes. The mechanisms differ by tool and the shape does not: every stack has checks that fail a run, checks that only warn, thresholds enforced in the warehouse rather than the project, and numbers with no check at all. Send whatever configuration and schema files you have.

Can it tell me who read a number that turned out to be wrong?

That is what the consumer register is for, and it reads query history rather than trusting declared lineage, because scheduled exports and spreadsheet connections appear in no graph. If it becomes an incident, the postmortem starts from that list. What each of those readers thinks the number means belongs in the field and metric dictionary.

How does it handle the pipelines nobody documented?

By reading the transformation code for the decisions somebody would otherwise reverse-engineer: which rows are excluded, how late-arriving data is attributed, which timezone a day boundary uses, and every hardcoded value listed individually. Where it cannot determine something, it writes down who would know. Those exclusions are also what an analysis write-up has to state before it quotes a number.

Our warehouse schema cannot leave our network. Can we still use this?

It does. The pack runs unchanged in a private AI workspace, which keeps schemas, query history and consumer names inside your own tenancy. Teams under a data residency obligation take that route, and the contract, the registers and the drift sweep behave exactly as they do here.

Sources

  1. dbtseverity, error_if, and warn_if (opens in a new tab)
  2. GitHubgreat_expectations/great_expectations/expectations/core/expect_column_values_to_not_be_null.py at develop (opens in a new tab)
  3. dbtfreshness (opens in a new tab)
  4. Airflow 3.3.2 DocumentationUpgrading to Airflow 3 (opens in a new tab)

Written by

Chandler Supple

Co-founder & CEO, River

Chandler has spent the last six years working in AI, building tools for millions of users and Fortune 100 companies. In 2025, he co-founded River to build a new kind of operating system for AI agents.

LinkedInGitHubX

Reviewed by

Brian Joseff, Co-founder, River

LinkedInX

More free templates

Analytics Request Intake Template

Free analytics request intake template that prices every recurring question's payback in weeks before deciding whether to build it away.

Data Dictionary and Metric Catalogue

Free data dictionary template that counts coverage against every column in your warehouse and prices every metric two teams compute differently.

Data Warehouse Migration Plan Template

Free data warehouse migration plan template that checks every model's output against its old version, not just whether the new pipeline runs.

Reverse ETL Documentation Template

Free reverse ETL documentation template that classifies every synced field by who else edits it before deciding whether to overwrite it.

Find out which numbers nobody can stand behind

Start from the blank documents and registers, or let River read your project and say which numbers rest on a threshold that enforces nothing.

Edit with AI