What this is, and which numbers are real
A discrete-event simulation of one team's delivery pipeline. Changes are authored, queue in the yard, are batched into a deployment, and either land or are rolled back — taking every change aboard with them. Nothing on this page is a lookup table: every figure is a counter incremented by that loop, and the seed is shown so any of it can be reproduced.
The fidelity ledger
| Bucket | What is in it |
|---|---|
| computed live | Change fail rate, deployments per month, times production broke, defects reaching users, changes reverted, blameless changes reverted, median recovery time, the quiet defects currently sitting in production, the month-by-month strip, and the batch-size curves. All emergent counters of the simulation, re-run whenever a control moves (1,200 simulated months per projection, 800 per sweep point). |
| scaled down | The animation runs a second, short simulation so a month passes in seconds, and its clock is retuned with the batch size so roughly one deployment happens per second whether the team ships forty times a month or once. Every duration inside it is stretched to match, so the animated sim's durations carry no data — its counts still do, because no duration enters any counter. Its live counters are noisy on purpose: that noise is the quantisation point, not a bug. Every stable figure in the panel comes from the long run. |
| assumed | 40 changes merged per month. 65% of user-visible defects severe enough to force immediate intervention — no source for this, and the whole "quiet bug" argument scales with it. Batch size 12 as the default, chosen to land the team on the 2025 median deployment frequency; DORA publishes no median batch size. Review degradation κ — DORA hypothesises the direction (2025, p.57), the magnitude is invented, so it ships off. All five durations (pipeline, detect, isolate, fix, latent-surface). Culprit isolation as a log₂(B) bisect, charged as time and not as deployments. |
| calibrated | The per-change escape rate q = 0.0189 is solved, not
typed — scripts/calibrate.mjs bisects on the simulation
until the default team sits on both 2025 DORA medians at once:
3.8 deploys/month and 12.0% change fail rate. The 12% target is the
midpoint of DORA's 8–16% modal bucket and is my choice;
DORA publishes the bucket, not a median value. |
| faked for legibility | Trains longer than 14 wagons are drawn truncated with a "+n" marker — 40 will not fit at a readable size — so the engine also carries an "n changes aboard" badge. Wagons still behind the departure signal are not drawn: the rest of the train has not left the yard. The isometric world's distances, speeds, building shapes and window count carry no data. Crate colour marks defect class; that is real in the model, but a real team cannot see it before the deploy lands, and that asymmetry is the whole reason batching is a gamble. |
The cited figures, with their edition year
Benchmarks move between report editions, so each carries its year. All DORA figures are self-reported survey estimates, not measured telemetry.
| Figure | Source |
|---|---|
| Change failure rate — Elite 0–15%, High 0–15%, Medium 0–15%, Low 46–60%. High vs Medium not significantly different (Tukey post-hoc). | Accelerate State of DevOps 2019, p.18 |
| Three clusters only — no Elite. High 0–15%, Medium 16–30%, Low 46–60%. | Accelerate State of DevOps 2022, p.11 |
| Elite 5%, High 10%, Medium 15%, Low 64%. Six ordinal buckets replaced by a 0–100% slider this year. | Accelerate State of DevOps 2023, p.12 and Appendix p.92 |
| Elite 5%, High 20%, Medium 10%, Low 40% — the metric is non-monotonic across clusters. | Accelerate State of DevOps 2024, p.13 |
| No per-metric performance-level table. Distribution instead: 36.2% of respondents at or below 8%, 62.2% at or below 16%, 5.9% above 64%. Median respondent deploys between once a week and once a month. | State of AI-assisted Software Development 2025, Figs 12 and 14, pp.20–21 |
| "A change that is stopped on its way to production is a successful demonstration of the deployment process's ability to detect errors." | DORA 2024, footnote 1, p.16 |
| "Change fail rate: the ratio of deployments that require immediate intervention following a deployment." | dora.dev guide, updated January 2026 |
| In DORA's first study (2014), change fail rate did not correlate with the other variables well enough to join the construct — "IT performance" that year was defined on three metrics. | dora.dev, A history of DORA's software delivery metrics, 2026 |
What this page does not show
It does not implement DORA's rework rate. It counts forced redeploys — a rolled-back batch going out again — which is well defined here; DORA's metric counts unplanned deploys made to fix a user-facing bug, which is degenerate under this model. It also assumes defects are independent across changes. Integration defects are plausibly worse than that in big batches, which would make small batches win by more, not less — so the base case is deliberately the one least flattering to the argument. Turn review degradation up to see the other world.
Full working, claims table, sources and the list of everything that could not
be verified: knowledge/change-failure-rate-ratio-trap.md.
Press A to close.