Stage 2 · Choose the instrument
What an AI use case actually costs
The token bill is the smallest line. The costs that decide whether a use case pays back are evaluation, context maintenance, revalidation after every model change, and the human time spent repairing wrong output.
Scope, honestly: The cost structure below comes from running my own AI work under a hard budget, and from delivery-side cost and KPI ownership. I have not owned a corporate AI budget line.
Ask what an AI use case costs and you will usually get a number derived from price per million tokens. That number is real, and it is rarely the one that determines whether the thing pays back.
I learned this the direct way, by running my own AI practice under a hard budget where overspending means the work stops for the day. Constraints teach faster than theory.
The costs a business case usually omits
- Building the evaluation. Before you can ship, someone assembles a reference set that represents reality, agrees what counts as correct, and encodes it. This is real work and it comes before any value is realised.
- Running the evaluation, repeatedly. Not once at launch — on every prompt change, every model version, every schema change upstream.
- Context maintenance. The prompt is not the system. The system is the prompt plus the context it is given, and that context has to be curated, versioned and pruned as the business changes. Neglect it and quality drifts without a single line of code changing.
- Revalidation after model changes. Providers deprecate and upgrade on their schedule, not yours. Each move means re-running the acceptance bar. Budget for it or be surprised by it.
- Repairing wrong output. The dominant cost in most use cases. If a person has to check every result, you have not automated the task — you have moved it and added a review step. The economics only work when the error rate is low enough, and known well enough, that spot-checking is defensible.
- Supervision and escalation. Someone has to notice when it goes wrong, and there has to be a path when it does.
An operating rule I actually enforce
In July I ran a multi-round exploratory analysis across many parallel agents. It produced an answer. It also consumed roughly half a day’s budget for something whose value was, in hindsight, bounded — and I hit the ceiling twice that day, mid-task.
The rule that came out of it is now enforced in my own tooling: before any fan-out, state the expected scale and get it approved; cap exploratory work at one round; prefer one cheap probe over a parallel verification army; and reuse cached results instead of re-running. Exploratory AI work has unbounded cost and bounded value. That asymmetry has to be handled by policy, because in the moment it always feels like one more round will settle it.
This is the same class of discipline an enterprise needs, at a different scale. The failure mode — spend accumulating across functions with no outcome attached — is identical.
Measuring return without lying to yourself
The measurement mistake I see most often is reconstructing return after the fact. Once a change has shipped into a live process, the AI effect and everything else that moved that quarter are no longer separable, and what gets reported is a story rather than a measurement.
The alternative is to instrument before launch:
- Pick the value stream where the outcome is real — productivity, differentiation, or new revenue. Not a proxy metric.
- Embed the AI in the flow and wire the data at the same time. Retrofitting the telemetry is how attribution dies.
- Map the whole flow, separating AI-touched from non-AI steps, so a change can be located.
- Tag the usage — which part was AI, what share of the outcome it touched.
- Run a control. A comparable area without the AI, for as long as it takes to be believable.
None of this is exotic. It is the same instinct as a controlled test in any other discipline, and it is what separates a defensible number from a confident one.
What I would put in front of a sponsor
A single view with the assumptions visible and adjustable: volume, unit time saved, loaded hourly cost, error rate, cost of repairing a wrong output, build cost, and monthly run cost — with payback period falling out of it. No fixed percentage claims, because those are borrowed from someone else’s context and do not survive contact with yours.
The point of showing the model rather than the conclusion is that it can be argued with. A number you cannot interrogate is a number nobody will defend six months later when it is wrong.
What does it actually pay back?
Set your own assumptions. The default values are neutral placeholders, not figures from any engagement of mine — the point of the model is that you can argue with it.
How this transfers
This is the artefact that decides funding. In an enterprise portfolio I would expect it to sit next to each candidate use case, so that prioritisation is a conversation about assumptions rather than enthusiasm — and so that “this does not pay back” is a finding the process can produce, rather than an embarrassment it has to avoid.