A Lower Change Failure Rate Can Mean Nothing Improved

I used to read change failure rate like every other rate in engineering: lower is better.

But CFR is a fraction, and its denominator is a decision, not a measurement. Ship the same work one change at a time instead of in batches of forty, and the rate falls twenty-three-fold — while the number of times you actually break production drops by 22%, and the number of bugs reaching users does not move at all. Same code, same people, same defects. The metric moved because the batch size did.

The column that does transform is the one no DORA metric contains: at batches of forty, one bad change drags an average of 38 blameless ones out of production with it.

So I built the simulation. Move the batch-size slider yourself.