Data Quality Monitoring Checklist Template
Four documents and five sheets that define every check against a named report consumer, so a failure arrives with an owner attached.
Free download · No account needed
One Detection, Four Verdicts
14 August 2026. The null rate on shipment.delivered_at went from a 0.4% baseline to 3.1%, a factor of 7.8, after a carrier changed its webhook payload.
One condition. Four consumers read that field, and each had declared a different threshold on it.
| Consumer | Their tolerance | Verdict | Severity | Alerted |
|---|---|---|---|---|
| On-time delivery dashboard | 1.0% daily | Breached | 1 Wrong on screen | T. Falconer |
| Carrier scorecard | 3.0% at month end | Breached | 2 Degraded | L. Beaumont |
| Monthly board KPI pack | 5.0% at month end | Within tolerance | none | nobody |
| Customer SLA credit calculation | zero in a credit window | Breached | 1 Wrong on screen | D. Marchetti |
Three alerts from one detection, at two severities, to three people, on three different clocks. The board pack was correctly told nothing: at a 5% month-end tolerance this excursion moves a company-wide on-time percentage by a tenth of a point. Billing tolerates zero, because a null there does not degrade a number, it issues a credit against a customer invoice. That month's credit window held 1,840 shipments, 57 of them null at 3.1%, which is $2,565 at $45 a shipment.
The standard way to build a check suite is to walk the schema. Enumerate the tables, put a null check on every nullable column, add a uniqueness check on every id, and route the lot into one channel. The size of the suite then gets set by the width of the warehouse, and a warehouse does not know which of its columns anybody reads. Pemberton Logistics, a freight brokerage, inherited thirty-four checks built exactly that way and four hundred and eleven detections in one quarter.
Replaying those detections against what six report owners actually said they needed, ninety-six were actionable to a named consumer. Seventy-seven percent mattered to nobody. The cross-tabulation is worse than the headline: sixty-eight of the one hundred and forty-three detections anybody triaged breached nobody's tolerance, and twenty-one that did breach a declared tolerance were never acknowledged. The team was not filtering the noise. They were sampling, and the sample had nothing to do with what mattered.
So in this pack a check belongs to a consumer rather than to a table, which makes severity a property of the (detection, consumer) pair. One field can carry four tolerances and that is correct, not untidy. Written for whoever gets asked why the dashboard was wrong on Tuesday. The definitions underneath belong in a metric register, the extract those checks run against in an export pipeline pack, and the reports worth watching at all get thinned first by a report inventory audit.
What you get
Consumer Tolerances
One row per declared tolerance: the consumer, one named person, the field, the dimension, a number with a period attached, and the severity if it breaches. Forty-seven rows in the worked example, and nothing else in the pack can be filled in before this sheet exists.
Quality Check Register
Every inherited check joined against the tolerance register in both directions, carrying a retain, delete or add verdict with the detections it produced. Deleted rows stay on the sheet with their reason, which is what stops somebody rebuilding them in eighteen months.
Failure Log
One row per detection and consumer pair rather than one per detection, so four consumers disagreeing about one incident is visible side by side. The observed magnitude against its baseline is on every row, because a detection logged as pass or fail cannot be replayed when a tolerance moves.
Owner Assignment
The alert route worked backwards from each consumer's read time, with the severity 1 and severity 3 actions written down before anything fires. A push at 06:30 for a 07:00 dashboard, three working days for a monthly pack, and an immediate page for a run that posts irreversibly.
Time to Fix
What each owner actually did, with acknowledgement and resolution hours and the loss avoided where there was one. Severity 1 acknowledgement is tracked on its own, because cosmetic alerts are batched deliberately and a mean across all three levels measures the batching policy.
How a Tolerance Is Declared
The six mandatory columns, the three-level severity ladder stated as consequences rather than feelings, and the four-thresholds-on-one-timestamp case worked through with what collapsing them costs in each direction.
Replaying an Inherited Suite
The method for scoring a suite you did not build, using detections that already happened. The actionable rate, the coverage gap in both directions, and the cross-tabulation against what was acknowledged, which is where the finding actually lives.
What This Does Not Catch
Five limits with numbers rather than caveats. An undeclared field is unwatched by construction, correct data in a wrong timezone passes every mechanical check, and a tolerance is somebody's guess until it has survived a quarter of acknowledgement history.
How it works
- 1
List the consumers, not the tables
Every artifact that reads data and produces a decision gets a row, with one named owner and the hour or working day they read it. River asks where numbers go rather than where data comes from.
- 2
Get a number out of each owner
For each field and dimension, the level of wrongness at which they would have published something they would want back. If they cannot name one, the field probably is not read and it comes off their list.
- 3
Replay whatever is already running
A quarter of existing detections scored against tolerances that did not exist when the checks were written. Out comes the actionable rate, the alert volume, and the checks no report reads.
- 4
Route each severity to a decision
Alert deadlines worked backwards from each consumer's clock, and the default action on a severity 1 written as a sentence in advance, so the decision takes a minute instead of a meeting.
Frequently asked questions
Is this free, and what format are the downloaded files?
Free, and no account is needed for the download. Documents arrive as .docx and sheets as .csv, so they open in Word, Pages, Google Docs, Excel, Numbers or Sheets without a conversion step. Edit with AI is the other button: it installs the same pack as a private space and fills it in from your own reports and alert history.
Can I not just set a threshold on the check itself?
You can, and well. dbt's test configuration takes conditional expressions on the failure count, so that one duplicate record can warn where ten error. What it takes is one threshold per test. Four consumers reading one timestamp at four thresholds is four register rows resolved when the alert routes, not one config value.
Is it not simpler to give a field one strict threshold?
Simpler, and it fails in a specific way. Take the tightest of Pemberton's four and the finance director gets paged about a condition that never reaches his pack, which costs the credibility the mechanism needs. Take the loosest and the credit run posts $2,565 of credits for deliveries that arrived on time.
Does declaring tolerances per consumer mean the suite gets bigger?
It got smaller in the worked example. Thirty-four inherited checks became thirty-nine, but nine watching fields no report read were deleted and fourteen were added against tolerances nothing had been covering. One check per field and dimension pair, with the per-consumer thresholds sitting on top of it rather than multiplying the checks.
Is quality not an objective property of the data?
Not usefully. The W3C's Data Quality Vocabulary is explicit that it does not define quality formally, and instead supplies a way to describe data so a potential user can judge its fitness for purpose. Here the consumer is that user and the tolerance is the judgment, written before the incident rather than argued about during it.
What if two reports disagree and neither has failed a check?
Then this is the wrong space, because nothing here compares two sources. Every dimension in the pack is mechanical: nulls, duplicates, allowed values, load times, one field against another. Two numbers that disagree while both pass belong in a conflicting report reconciliation, and what gets built at all in a dashboard spec.
Find out how much of last quarter anybody could act on
Send the reports people actually read and whatever alert history you have. River gets a number out of each owner, replays your existing detections against it, and reports the actionable rate.
Edit with AI