Monthly Reliability Review and Toil Estimate
Six documents and four sheets that price your operational load in engineer-days and project it to the quarter it breaches the engineering floor.
Free download · No account needed
A monthly reliability review reports uptime, incident count and mean time to resolution. All three describe the month accurately and change nothing, because none of them is denominated in anything a planning conversation can spend. The review gets read, the numbers get nodded at, and the roadmap that was set before the meeting is the roadmap after it. Nothing in that sequence is anybody's fault. The review simply had no currency to bid with, so it lost every argument it was in before it started.
This pack produces one. Toil Estimate holds occurrences and minutes in separate columns and derives engineer-days, which is the unit a quarter's roadmap, a hiring request and a proposed automation are all already priced in. Google's SRE book defines toil as work that is manual, repetitive, automatable and scales linearly as a service grows, and publishes its own measured average of about a third of engineering time. So the level, on its own, is reassuring.
The trajectory is not, and it is already in that definition. Toil Projection tags each line with the thing that drives it and applies your measured growth rather than the plan. Headcount is held flat, because this is the do-nothing case rather than a forecast. In the worked example a six-engineer team sits at 29 percent today and 53.2 percent four quarters out, before on-call is counted against the remaining half. That date is the finding, and it is the version of the argument that gets work scheduled before the floor is breached.
What you get
Toil Estimate
One row per recurring operational activity, with occurrences from a system and minutes from timed instances kept in separate visible columns. Engineer-days is derived rather than entered, so a disagreement lands on one cell instead of on the whole finding.
Toil Projection
Projects each line by its own driver at your measured growth rate with headcount held flat, and reports two totals side by side: the do-nothing case, and the same case with the recommended automations shipped.
Incident Register
Every incident counted twice, by number and by customer-impact minutes, with both share columns beside each other. The second denominator is what finds the category that is cheap and frequent and the one that is expensive and equally frequent.
SLO Attainment
Three months per row, so direction is readable without computing it. Sorted by direction rather than by pass or fail, because a passing objective that has declined for three months is where intervention is still cheap.
Investment Recommendation
One page, ranked by build days divided by days recovered per quarter rather than by size, with the three ways to decline it written out. Priced in the same unit as the feature work it competes with.
Trend Analysis
The four cross-cuts a per-incident review structurally cannot run: count against impact, repeat failure modes below the cause category, detection source, and objectives that are passing while declining.
Review Procedure
An hour, with the agenda in minutes and the assembly done a day ahead. Includes who has to be in the room for the recommendation to have somewhere to go, and what not to spend time on.
How Toil Is Costed
The reference behind the arithmetic: what counts, what people wrongly include, the four decisions, and why the projection is presented as the do-nothing case rather than as a forecast.
How it works
- 1
Send the incidents
A ticket export of what closed last quarter. River builds the register, categorises by cause, and runs the count-against-impact comparison before it looks at a single dashboard.
- 2
Add the operational counts
Pages taken, tickets by label, failed job reruns, access requests, manual deploys. These live in four systems and any two are enough to start. Occurrences come from a system, never from memory.
- 3
Get the trajectory
River tags each line with what drives it, applies your measured growth from the last four quarters rather than the plan, holds headcount flat, and names the quarter operational load crosses the engineering floor.
- 4
Take the one-page ask
Build days against days recovered per quarter, a payback figure, and every line automation will not fix named with the reason. Plus the cheaper subset, in case the answer is no.
Frequently asked questions
Why engineer-days instead of a percentage?
Because a percentage is a comparison and engineer-days is a currency. A quarter's roadmap, a hiring request and a proposed automation are all already priced in engineer-days, so putting operational load in the same unit lets the four be compared in one sentence instead of four meetings.
Our toil is about 30 percent. Is that bad?
No, and that is the trap. Google's own quarterly surveys put its SREs at about a third against a stated 50 percent ceiling, so 30 percent is unremarkable. The projection is what matters: 30 percent with 23 percent quarterly traffic growth and flat headcount is past the floor in four quarters.
Does this run monthly or quarterly?
The review runs monthly. The toil estimate refreshes quarterly, because a single month of operational load is too noisy to project from and a monthly re-estimate turns a useful number into a chore nobody does. The register and attainment sheet are monthly.
What if the biggest toil line cannot be automated?
Then the recommendation says so explicitly, which is the point. Alert response is a third of the total in the worked example and automating a response to an alert that should not have fired is the wrong work. That belongs to an alert quality review instead.
How is this different from doing postmortems?
A postmortem explains one incident. This finds the mechanism that produced four of them, each with a competent closed postmortem, and prices it. Tracking whether those postmortems' actions were ever finished is its own job, handled by the action follow-through tool.
Do we need SLOs before this works?
No. The incident register and the toil estimate work with no objectives at all. Attainment is the one sheet that needs them, and setting a target you can actually enforce is a separate exercise in the SLO and error budget pack.
What if our runbook coverage is the real problem?
It often is, and it shows up here as an inflated alert response line. Computing which alerts have no current documented response, weighted by the pages that landed at night, is what the on-call handbook pack does. Run that first if your nights are the complaint.
Find the quarter your team stops doing engineering
Send an incident export and your operational counts. River prices the load in engineer-days, projects it at your measured growth, and ranks the automation queue by payback.
Build my review