Your AI Code Passed Review. Production Sent the Bill.

A change lands on a Tuesday. An agent wrote it, a senior engineer approved it in eleven minutes, CI was green, and the review thread has one comment: good tests. The pull request closes. The dashboard records a merge, a cycle time, and a cost per AI-assisted PR that looks better than it did last quarter.

Six weeks later the same change comes back. A hotfix on a Friday evening. A rollback. A schema assumption that only misbehaves under production traffic. Two pages to the on-call engineer, and the better part of a day from a staff engineer’s week spent reconstructing what the agent was doing.

Both events are real engineering costs. Only the first one is in the ledger.

Most AI measurement programs are built around the merge event, because that is where the data is clean. Tokens, runs, seats, review time, cycle time and acceptance all resolve at or before merge. What happens next resolves slowly, lands in different systems, and is rarely joined back to the change that caused it. The result is an accounting boundary that no longer matches where the money goes.

New Relic’s 2026 State of AI Coding Report, an online survey of 200 US technology decision-makers at manager level and above conducted by Hanover Research, found that 94% of respondents rate AI-generated code as higher quality than human-authored code at review time. Thirty-three percent called it much higher quality.

The same respondents also reported what happened afterwards. Seventy-eight percent saw measurable spikes in production incidents tied to AI-generated code. Eighty-two percent had at least one major production failure attributable to AI-generated code in the previous six months. Seventy-four percent said that at least a quarter of their AI-generated code required significant rework after deployment. Eighty-six percent reported more senior engineer time spent on emergency fixes.

A separate survey of 200 site reliability and DevOps leaders in the US, UK and EU, published by Lightrun as the 2026 State of AI-Powered Engineering Report and administered by Global Surveyz in January and February 2026, puts a number on the gap: 43% of AI-generated code changes needed manual debugging in production after passing QA and staging. In the same study, no respondent reported being able to verify an AI-suggested fix in a single redeploy cycle.

Both are leader surveys, so they measure perception and recollection rather than instrumented outcomes. That caveat matters, but the direction of the error is consistent with the one controlled study available. METR’s randomized trial of experienced open-source developers, published in July 2025, found that participants completed tasks 19% slower with AI assistance while estimating afterwards that AI had made them 20% faster. Judgments made close to the change are optimistic. Review approval is a judgment made close to the change, usually by someone with minutes of context on work that took an agent seconds to produce.

That does not make review worthless. It makes review a poor proxy for durability.

Four mechanisms move cost past the merge boundary. None of them requires anyone to behave badly.

Volume outruns verification. More changes arrive per reviewer per week, so the average change gets less scrutiny even when total review effort rises. We have written about the review load this creates; the part that matters here is that thinner review lets more marginal changes through, and marginal changes are the ones that resurface later.

Local correctness, global drift. GitClear’s Maintainability Gap research, published in January 2026 from 623 million analyzed code changes between 2023 and 2026, found duplicated blocks rising from 40.3 to 73.0 per million changed lines, an 81% increase since 2023. Over the same period the share of changes that move existing code, the signature of refactoring, fell from 21% in 2022 to 3.8% year to date in 2026, while copy and paste rose from 9.4% to 15.7%. Code that works and does not consolidate passes review comfortably. It bills later, in the form of the next change to the same area.

Security defaults fail quietly. Veracode’s 2026 GenAI Code Security Report, published in July 2026, found an average security pass rate of 56% across the models tested, barely moved from 55% the year before, meaning roughly 44% of code generation tasks introduced a risky vulnerability. Pass rates were far worse for classes that reviewers rarely catch by eye: 15% for cross-site scripting and 12% for log injection.

Defect density is higher per change. CodeRabbit’s State of AI vs Human Code Generation report analyzed 470 open-source pull requests, 320 AI-coauthored and 150 human-only, and reported roughly 1.7 times more issues in the AI-coauthored set. This is a small sample scored by a review tool’s own taxonomy rather than by production outcomes, so treat it as directional rather than settled.

None of this is an argument for less AI. It is an argument that the cost of an AI-assisted change is not knowable on the day it merges.

The useful unit is not the change. It is the change plus the period during which it is still capable of generating work. Call that period the durability window.

A durability window has five observation points. Each one answers a different question, and each one is measurable from systems most engineering organizations already have.

The 90-day point is where most of the financial argument sits, because reliability shows up in money faster than maintainability does. DORA’s ROI of AI-Assisted Software Development report, published by Google Cloud’s DORA team in May 2026, models a 500-person engineering organization and projects a 39% first-year return with a payback period of roughly eight months. The same model shows what a small reliability regression does to that figure: a change failure rate moving from 5% to 6% appears as a negative $344,000 downtime impact. One point of change failure rate consumes a visible share of the modeled gain.

That is the practical case for measuring past merge. Not that AI-assisted delivery is unprofitable, but that the profit is decided in a window nobody is currently watching.

Cost per merged pull request is a useful number and an incomplete one. It divides real cost by a generous denominator.

Cost per durable change uses the same cost basis and a stricter denominator: the changes that pass their durability window without corrective work, with the cost of corrective work added back in.

The absolute figures will differ in every organization. The ratio between the two numbers is what to track, because it is comparable across teams, tools, models, agents and change types, and it moves before anything shows up in an incident review. A tool whose cost per merged change is 20% cheaper and whose cost per durable change is 40% more expensive is not a saving. It is a transfer.

Two cautions. Attribution of corrective work to an originating change is imperfect, so measure the trend rather than the decimal, and keep the same attribution rule from month to month. And a rising correction rate on a team doing hard migration work is not automatically a problem, which is why the metric belongs next to what the team was asked to deliver rather than on its own.

Applying a 90-day window to every merge would cost more than it returns. Route it by blast radius.

An organization that cannot name the agent behind a change cannot measure that change’s tail, which is a variant of a problem we have described before: you can list your AI tools, but not your agents.

Measurement that nobody reads is a cost, not a control. Three rhythms are enough.

Weekly, look at reverts and hotfixes on AI-assisted changes, split by team and by tool. This is the cheapest early warning available and it needs no new instrumentation.

Monthly, compare cost per merged change with cost per durable change for each tool and model. The gap between the two lines is the leading indicator. A widening gap is a signal to change model guidance or review routing, not necessarily to reduce usage.

Quarterly, make renewal and expansion decisions on the cohort that has cleared its 90-day window, not on the last four weeks of adoption. This is the single change that most improves the quality of AI tooling decisions, because it stops rewarding tools that generate volume and penalizing tools that generate durable work.

Ownership follows the same split. The platform team owns the measurement window and the joins. Engineering leadership owns the routing policy. Finance and FinOps should be shown the tail explicitly, because the correction cost of AI-assisted code behaves less like a purchase and more like a liability accrued at merge and settled later. Presenting only the invoice understates both the cost and, in the cases where AI-assisted work holds up well, the return.

Milestone is built on the join between AI spend and engineering delivery, and the post-merge period is where that join does the most work.

AI Quality measures Post-Review Change Rate, how often code keeps changing after formal approval, alongside review depth signals and rework patterns, and benchmarks these across GitHub Copilot, Cursor and Claude Code from repository history rather than from surveys.

Human & Agent Spend calculates cost per feature including review effort and rework effort, and classifies how work was built, from human written through AI assisted and AI generated to reworked. That classification is what makes a durability comparison between collaboration modes possible.

Agent Observability attaches identity, run cost, retries and outcomes to agent activity, so an agent-authored change in a load-bearing area can be traced back to the run that produced it.

AI Spend Hub holds the single cost ledger across tools, seats, tokens, models and agents that the denominator of any unit-cost metric depends on, and Milestone Insights explains what a moving KPI means and what to do about it.

A no to several of these does not mean AI-assisted code is failing in your organization. It means you would not currently be able to tell.

The merge event is a convenient place to stop measuring and a poor place to stop accounting. Review approval says a change looked right to one person at one moment. Production says what it costs. The organizations that get the most out of AI-assisted engineering over the next few years will not be the ones with the highest adoption or the lowest token bill. They will be the ones that can price a change six weeks after it shipped, and act on the answer.

If you are still deciding where AI spend is going before you decide whether it worked, our earlier pieces on AI cost attribution and what AI waste actually looks like cover the ledger side of the same problem.

1. What is a durability window?

The period after a change merges during which it can still generate engineering work. In practice it is a set of observation points at seven, 30, 90 and 365 days, each joined to signals such as reverts, churn, incidents and duplication.

2. Why is review approval a weak quality signal for AI-generated code?

Because it is a judgment made at the moment of the change, on work produced far faster than it can be verified. Survey evidence from 2026 shows leaders rating AI code highly at review while simultaneously reporting more post-deployment incidents and rework from the same code.

3. What is cost per durable change?

Total cost of producing changes, including AI usage, human effort and review time, plus the cost of corrective work, divided by the number of changes that clear their durability window without correction. It is a stricter version of cost per merged pull request.

4. Does this mean teams should slow down AI adoption?

No. It means expansion decisions should use post-release evidence. Some teams and some workflows produce durable output at lower cost with heavy AI use, and a merge-boundary metric cannot distinguish those from the ones that do not.

5. Which existing systems supply the data?

Git history, CI and deploy records, incident and on-call systems, issue trackers, and AI tool and agent billing. The difficulty is rarely collection. It is keeping a change identifier stable from the AI tool through merge and into the incident record.