Seat counts tell you who has access. Token bills tell you what you burned. Neither tells you whether the work landed. The session outcome ladder does.
Most engineering organisations now have a reasonable answer to “what are we spending on AI?” Invoices arrive, dashboards aggregate them, and someone in finance can produce a monthly figure per tool. What far fewer organisations can answer is the question that follows immediately: of everything those agents did last month, how much of it ended up in the product?
That gap is not a reporting inconvenience. It is a measurement error at the level of the unit itself. Seats and tokens are both inputs. They describe capacity and consumption, not delivery. An organisation optimising either one in isolation can quite easily make its AI programme look healthier while shipping less.
The missing unit is the agent session: one continuous run of an agent working on a task, from the first prompt to the point it stops. A session has a cost, a duration, a set of tool calls, a repository it touched, and – critically – an outcome. Once you can classify that outcome, the economics of agentic engineering become legible for the first time.
Why seats and tokens both mislead
Seat-based measurement treats access as adoption. It answers “how many developers could use Cursor this month”, which was a reasonable proxy in the autocomplete era, when a licence roughly corresponded to a steady, low-variance level of usage. Agentic tools broke that correspondence. Two developers on identical licences can differ by an order of magnitude in what they consume and what they produce, because one is running long autonomous sessions against a large codebase and the other is asking occasional questions.
Token-based measurement fixes the variance problem and introduces a different one. Tokens are a faithful record of consumption and a poor record of value. A session that burns a large context window exploring a codebase, produces a careful patch, and gets merged is expensive and excellent. A session that burns the same context, produces nothing, and is abandoned is expensive and worthless. On a token dashboard the two are indistinguishable. Worse, the common instinct to reduce token spend penalises the first kind of session as readily as the second, because the dashboard has no way to tell them apart.
The point is not that tokens are the wrong thing to track. Milestone has written at length about AI token spend management and cost attribution precisely because consumption data is necessary. The point is that consumption data is not sufficient. You need a numerator and a denominator drawn from the same unit.
The session outcome ladder
Every agent session ends somewhere on a ladder of progressively stronger evidence that work actually landed. Milestone classifies sessions into five mutually exclusive outcomes, assigning each session the highest rung it reached.
Figure 1. The agent session outcome ladder. Each session is classified once, at the highest rung it reaches.The precedence rule matters. A session that produced three commits, opened a PR, and saw it merged is classified once, at rung 5. This prevents the double-counting that makes activity dashboards so flattering and so useless.
What makes the ladder analytically useful is that each rung has a different diagnosis attached to it, and the diagnoses are not interchangeable.
A heavy concentration at rung 1 usually indicates a scoping or context problem rather than a model problem. Agents that end without touching code have typically been asked questions, been pointed at the wrong part of the repository, or hit permission and tooling walls early. This rung is also where exploratory and research usage lives, which is legitimate and should not be driven to zero.
A heavy concentration at rung 2 is the signature of work that was attempted and then abandoned. The agent edited files, the developer looked at the result, and nothing survived. At scale, this rung is the clearest proxy for wasted spend in the entire system.
A heavy concentration at rung 3 should prompt a governance conversation, not a productivity one. Code is reaching the repository without passing through review. Whether that is acceptable depends on the repository, but it should be a decision rather than an accident. It also connects directly to the review-capacity problem: as AI writes more code and review becomes the bottleneck, one way organisations relieve that pressure is by quietly routing around it.
A heavy concentration at rung 4 suggests the agent is producing plausible but unacceptable work. The output was good enough to propose and not good enough to accept. This is the rung most worth correlating against code quality signals, because it frequently moves together with rework and review depth.
Rung 5 is the only rung that evidences delivery – and even then, only that the code landed, not that it was good. More on that below.
The two numbers worth reporting upward
The ladder produces two derived metrics that travel well to a CTO, a CFO, or a board.
The first is session yield: the share of sessions that reach a given rung. Yield to rung 5 is the strictest reading and the most useful headline. Yield to rung 3 or above is a looser reading that captures work landing outside the review path.
The second is cost per merged PR. This figure converts AI spend into something a finance function already understands, because it has the same shape as a unit-economics number. It is computed by attributing session cost to the pull requests the work landed in. Milestone supports two attribution methods with different trade-offs: an exact method that follows git history from commits to pull requests, and an approximate session-based method that provides coverage where the git linkage is incomplete. Both roll up to repository, developer, team, and PR.
Figure 2. Three units of AI engineering measurement, and what each one can and cannot answer.
| Unit | Answers | Blind to | Failure mode when used alone |
| Seats |
Who has access |
Intensity and output |
Access is reported as adoption |
| Tokens |
What was consumed |
Whether anything shipped |
Productive and abandoned sessions look identical |
| Sessions |
What was attempted and what landed |
Whether the landed work was sound |
Merge velocity gets optimised at the codebase's expense |
Where the ladder will mislead you
Any measurement framework an engineering leader will act on should come with its failure modes attached. This one has three worth stating plainly.
The commit-based rungs are lower bounds. Attribution relies on linking a session to commits and those commits to pull requests. Where that chain is broken – squash merges that lose authorship, work that moves between branches, commits made outside the session’s visibility – the session will be classified lower than it deserves. Treat rung 5 yield as a floor, not a point estimate.
“Local changes only” is an upper bound. A session is credited with local changes when it successfully edited files. Some of those edits were immediately reverted, overwritten, or thrown away. The rung captures attempted work, not retained work.
Signal coverage is not uniform across tools. Session-level telemetry depends on what each vendor emits. In practice, Claude Code currently emits the richest signal set, including tool calls, MCP calls, sub-agents, context compaction, peak context, errors, and hooks. Codex emits a narrower set, without compaction or session-context data. Cursor’s admin feed currently provides feature-level adoption rather than session-level telemetry. Cross-tool comparison of outcomes and cost is valid; comparisons of context behaviour should be read as “reported where the tool sends the signal” rather than as a like-for-like benchmark. Absence of a signal is not a zero, and any platform that renders it as a zero is misleading you.
Merged is not the same as good
A merged PR is evidence that work landed. It is not evidence that the work was sound, and any framework that stops at rung 5 will eventually produce a team optimising for merge velocity at the expense of the codebase.
This is why the ladder needs to sit next to quality and stability signals rather than replace them. The ones that matter most here are rework after review, review depth, test coverage on the pull request, and – the slowest and most revealing – code stability over time: what share of merged code remains untouched, what share is reworked within thirty days, and what share is refactored later. A team whose rung 5 yield is climbing while its thirty-day rework rate climbs alongside it is not getting more productive. It is moving its costs downstream, a pattern Milestone has described before in the context of AI code that passes review and bills you in production.
Putting the ladder to work
For a team starting from token dashboards, the sequence that produces an answer fastest is:
- Classify a full month of sessions before changing anything. The baseline distribution is the diagnosis.
- Look at rung 2 first. Abandoned local work is usually the largest recoverable cost and the easiest to investigate, because the sessions are individually inspectable.
- Segment by repository before segmenting by developer. Outcome distributions vary far more between codebases than between people, and starting with people invites the wrong conversation.
- Pair rung 5 yield with thirty-day rework. Track them on the same chart, so a gain in one cannot hide a loss in the other.
- Re-check coverage before drawing conclusions. If a tool stopped reporting mid-period, its yield will look like a collapse when it is a telemetry gap.
How Milestone measures this
Milestone classifies every agent session into the five outcomes above and attributes cost to the pull requests the work reached, across the AI coding tools an organisation actually runs. Agent observability provides the session-level view: outcomes, cost, duration, tool and MCP calls, and context behaviour where the tool reports it. AI cost attribution rolls those costs up by team, repository, and feature. GenAI code quality analytics supplies the rework, review-depth, and stability signals that keep a yield number honest.
The underlying argument is simple enough to state in a sentence: if you cannot tell a session that shipped from a session that did not, you are not managing an AI programme, you are paying for one.
See what your agent sessions actually produced
Book a walkthrough of Milestone's agent observability and we will classify a month of your own session data against the ladder.
Book a walkthrough