Software & TechnicalFree
Engineering Metrics and Delivery Review
River recomputes delivery metrics over what they can actually see, then shows what each number would do if somebody decided to move it cheaply.
River's delivery review reads the ticket system, the git history, the deploy log and the incident record, then recomputes each metric with the population it was measured over written next to it. Out comes a sheet with one row per metric: the number your dashboard reports, the number the underlying events support, the share of merged work the calculation could see, and the gap between them. Then a document explaining every gap and what it would take to close it.
Page one for this query is a metric catalogue. Twelve metrics, three frameworks compared in a table, a benchmark band, and a paragraph warning you not to use any of it for performance reviews. All of that is fine advice and none of it is a measurement. What is missing everywhere is the denominator: nobody says what share of a team's work the four numbers can see, because the honest answer is usually a bit over half and no vendor wants that on their pricing page.
Built for the director asked to show delivery is improving, the engineering manager whose team is about to be compared to another, and whoever has to put a number in a board deck and defend it. Run it once to find out what your dashboard is actually counting, then quarterly. The narrative that goes around the numbers is an engineering status update, and the plan they are measuring is a technical program plan.
DORA changed a metric for exactly this reason
In 2023 the metric everyone still calls mean time to recover was renamed and redefined as failed deployment recovery time. DORA's own account of the change gives the reason plainly: the old definition did not distinguish a failure caused by a software change from one caused by something external, such as a data centre outage. The definition was not vague. Its denominator was wrong, it was mixing two populations, and the median of the mixture described neither of them.
That is the same error most dashboards are still making in four other places, and it is always in the flattering direction. A deploy log counts events, not changes, so redeploying the same artifact raises the deployment frequency and lowers the change fail rate at the same time, from one inflated denominator. Cycle time is computed over tickets that carry both a start and a done timestamp, which is never all of them. Every one of these is checkable in an afternoon.
There is also a fifth metric almost nobody computes. DORA now publishes five, and the newest is the deployment rework rate: the share of deployments that were unplanned and happened because of a production incident. It is the one number on the list that gets worse when you game the others, which is exactly why it belongs on the same page as them. A team shipping more and rolling back more is visible in it and invisible in the rest.
How it works
Paste the exports
Ticket history, git log, deploy events and incidents, for one quarter or two.
River joins them
Each metric is recomputed from events, and the rows that fall out of the join are counted.
Read both numbers
One row per metric with the reported figure, the measured figure and the coverage.
Take the caveats
The document says what each number cannot see, in language a board deck can carry.
What you get
- Each metric restated with the population it was computed over, beside the reported figure
- Deploy events separated from deployments of new code, which moves two metrics at once
- Recovery time split into change-caused and everything else, the way DORA now defines it
- The deployment rework rate, computed from your hotfix history rather than left blank
- The share of merged work that carries a ticket, which is what cycle time can see
- A gaming table: the cheapest action per metric and how far it moves the number
Common questions
What data does it need?
Four exports, and it runs on three. The deploy or CI event log, the git history, the ticket export, and the incident record. Anything missing becomes a stated gap rather than a guess, and the review says which metric went uncomputed and what would be needed to compute it.
Our dashboard already shows all of this.
It shows the numerators. The question is what each one was divided by, and a dashboard that joined your deploy log to your incident tool on a sixty-minute timestamp window will not tell you that six of the nineteen matches were something else. That join is a choice somebody made once, and it is worth reading.
Why compute the rework rate if nobody asks for it?
Because it is the only one of the five that gets worse when the others are gamed. Deployment frequency, lead time, fail rate and cycle time all respond to how work is recorded. The share of deploys that were unplanned responses to an incident responds only to whether the last release held. Whether the fixes promised after those incidents landed is a separate postmortem follow-through.
Is the gaming table not a manual for gaming?
Every engineer already knows all five, which is the point of writing them down. What they usually do not have is the arithmetic showing that a nightly redeploy job more than doubles a headline number. Put it in front of the person setting the target and the target changes, which is cheaper than discovering it a year in.
Can we use these for individual performance?
No, and the review says so with a number rather than a caveat. Eight of Tarrant's forty-eight engineers produced eleven of its seven hundred and thirty-one tickets, because they spent the quarter on a migration. Ranking on ticket counts puts the hardest work at the bottom, and the coverage column is what makes that visible.
What about the metrics that are not in DORA?
They get the same treatment. Cycle time, throughput, review latency and pull request size are all computed over a population smaller than the work, so each comes back with its coverage beside it. Reviewing is the largest blind spot: 3,214 comments appear in none of the five, and a chunk of them are reviewers asking what a diff does, which is a change summary problem.
What comes out of it?
A sheet with one row per metric carrying the reported figure, the measured figure, the population and the coverage. Then a document explaining each gap, the gaming table, and a short section of caveats written to be pasted into whatever deck the numbers are going into. The status update is where they usually land.
Engineering Metrics and Delivery Review
Fill in the form and your workspace opens with the work already underway.