Method

Five questions, in this order

The order is the point. Nearly every expensive mistake I have seen — including my own — comes from starting at question three.

Find the bottleneck

Problems choose tools. Tools do not get to choose problems.

Half the constraints an organisation wants to solve with AI are not constraints — they are habits, missing data, or symptoms of something upstream. Ask why five times and the compliance question often turns into an integration question.

Where is a faster competitor serving the same customer better?
Which high-volume decisions cost money through delay or error?
Which two departments are measured on metrics that contradict each other?
Is this bottleneck real, or a misread of where the time goes?

Finding the real bottleneck (and the fake ones)

Choose the instrument

The first job is not to sell AI.

A language model is one option among five: do nothing, delete the step, buy what exists, write the rule, or use a model. The fourth wins more often than anyone expects — cheaper, faster, auditable, testable.

Not could we — should we? What new risk does AI introduce here?
Would a deterministic rule be cheaper and more testable?
Does this build something a competitor cannot buy next quarter?
What does a wrong output cost, and who absorbs it?

Deciding when AI is the wrong tool What an AI use case actually costs

Redesign the process

Automating a bad sequence gives you a faster bad sequence.

Most processes are sequential for historical reasons. When the executor stops being one person, most of those dependencies dissolve — but only if someone goes looking for them.

Which dependency is a requirement, and which is just history?
What would this look like if waiting time were zero?
Is this manual check a control, or a comfort?
What does the current shape cost in customers who quietly give up?

From sequential to parallel

Prove it works

Eighty-six percent correct is not a result. It is a starting point.

Measuring business return tells you whether something created value. It does not tell you whether the output is good enough to release. That needs an acceptance criterion agreed before the build, a reference set that represents reality, regression on every change, and root-cause discipline that is allowed to conclude "the source data is wrong".

What is the acceptance criterion, written down before anyone builds?
Who agreed that the reference set represents reality?
When it fails — is it the model, the input, the retrieval, or the logic?
How would we notice a silent regression after a model change?

When six of seven runs failed Generating at scale, and validating every piece

This is the stage most frameworks skip, and the one I came up through.

Make it stick

A pilot is a question. It needs an answer that kills it or scales it.

Pilot purgatory is not a technology failure. It happens when nobody wrote the exit criteria, the data foundation was treated as a cathedral, roles were replaced instead of redesigned, and the team that proved the idea had no incentive to spread it.

What kills this pilot? Write it down before the pilot starts.
Which decisions are reversible enough to delegate?
Are we retraining people into new roles, or quietly replacing them?
Whose job is it to make this work somewhere else?

Escaping POC hell

Credit where it is due

What is mine, and what is not

Stages one, two, three and five are not original to me, and I would distrust anyone claiming otherwise — they are roughly where serious practitioners have landed. The reframing I have found most useful this year came from an AWS re:Invent 2025 leadership session by Helena Yin Koeppl: that organisational silos persist because incentives are misaligned rather than because the org chart says so, and that return has to be instrumented during operation, since afterwards the AI effect can no longer be separated from everything else that moved.

Stage four is the part I add. That material, like most leadership-level work, moves from "is this worth doing" straight to "did it pay back". Between them sits the release decision — is the output good enough to put in front of a customer — and answering it is ordinary quality engineering applied to a probabilistic system. I spent twelve years doing exactly that for software that had to ship.

Scale note: I have applied all five stages end to end on systems I built and owned, not on a transformation programme across thousands of people. Every case page says which.