On-Call Handbook and Runbook Coverage

Every on-call handbook template hands you the same four headings: severity levels, an escalation ladder, a handoff format, and a page about staying calm. The headings were never the problem. What went wrong on the last bad night was that an alert fired, the responder opened the runbook link, and found a document written for a threshold that had moved in April. Nobody knew, because runbook coverage is self-reported, and self-reporting is always wrong in the same direction.

This pack computes coverage instead of asking for it. Prometheus documents the annotations clause on an alerting rule as the place to keep alert descriptions or runbook links, so the join between an alert and its documentation already lives in version control. Runbook Coverage by Alert reads that field for every rule, then adds two dates: when the expression last changed, and when somebody last followed the runbook. Their order splits linked runbooks into current and stale.

Then it changes the denominator. Coverage by rule flatters you, because the undocumented rules barely fire. Coverage by night page is the number that predicts a bad night, and the two routinely disagree by twenty points. Because every finding costs somebody time, Rotation Feasibility checks whether the rotation has any. Two people on-call at once on week-long shifts puts each engineer in a team of six on a third of the calendar, which is a capacity finding rather than a documentation one.