Thistledown Systems, a subscription billing platform, ran six months of its own deployment history back through its release pipeline's five stages rather than trusting the stage list on its own. Of 1,240 deployment attempts, every automated stage blocked a real, consistent share: 15.1 percent at the automated test suite, 3.9 percent at the staging smoke test, 2.8 percent at canary analysis. Its Change Advisory Board approval, the one human sign-off in the pipeline, blocked zero of 991 requests over the same six months, deciding in a median of 47 seconds.
A zero-block rate alone could mean the gate is simply redundant with the checks around it. Thistledown's own approval charter names a specific, checkable requirement instead: confirm a rollback plan is attached before signing off. 214 of the 991 requests, 21.6 percent, were approved with no plan attached, meaning the one item the approver's own checklist asks for went unchecked on more than one release in five. DORA's 2019 Accelerate State of DevOps Report found this pattern industry-wide: an external approval body carries no measurable gain in stability and a real cost in speed.
The gap is not abstract. Nine of the window's shipped releases needed an actual rollback rather than a forward-fix hotfix. The six with no plan attached took a median of 52 minutes to execute against 11 minutes for the three that had one, a 4.7x difference on the identical operation. That is the change failure rate DORA's own metric guide defines, split by the one variable a pipeline diagram never shows: whether the approval that let the release ship actually checked anything on the way through.