The Wrong Question About AI: Why the Pilots Multiply and Nothing Changes
Most organisations are asking where they could use AI. That produces a list of pilots and very little else. The question that changes anything is what you would do differently if the work cost a tenth as much.
Almost every organisation I speak to has AI pilots running. Very few have changed anything about how they actually work.
That is not usually a technology failure. The pilots mostly do what they were meant to do. Something else is going wrong, and it happens before any of the technology is chosen.
The Question Everyone Is Asking
The question on the table is nearly always some version of “where could we use AI?”
It is a fair question, and it produces a tidy answer — a ranked list of use cases, a couple of them funded, a working prototype inside a quarter. It is answerable, it is safe, and it creates visible momentum quickly. There are consultancies whose entire proposition is helping you produce that list.
The trouble is that it starts from the tool and reasons backwards to the business. What you get is a set of improvements bolted onto processes that were designed around the cost of doing the work by hand.
The process is untouched. It is just slightly faster in one place.
The Question Worth Asking
The better question is harder to answer and considerably more useful.
What would we do differently if this work cost a tenth of what it costs today?
That is a business question. It does not ask where to insert a tool; it asks what the organisation would look like if a constraint it has designed around for thirty years stopped applying.
Most of the structure in a large organisation exists because something used to be expensive. Work is batched because handling it individually cost too much. Approvals exist because checking everything was impractical. Teams are shaped around volumes of manual effort. Change the economics of the work and the reason for a good deal of that structure quietly disappears — but only if somebody goes back and asks.
You see it plainly in regulated industries. A billing exception, a meter read that will not reconcile, a case that needs a second pair of eyes — each has a process wrapped around it, and most of that process exists because human attention was the scarce thing. Make attention cheap and the interesting question is not how to automate the checking. It is how much of the checking needs to exist at all.
Why the Honest Answer Is Uncomfortable
The answer usually involves fewer steps, fewer handoffs, fewer approvals and a different distribution of who decides what.
Which is exactly why AI stalls after the pilot. The pilot proves the technology works. Acting on it means redesigning how work flows, moving accountability, and accepting that some roles change shape. No pilot budget covers that, and no vendor demonstration mentions it.
It is also, unavoidably, a leadership decision rather than a technical one. Somebody has to be willing to say that a process the organisation has run for a decade is the wrong shape now, and to have that conversation with the people who run it.
So the pilot succeeds, the operating model does not move, and eighteen months later there are eleven pilots, some genuine improvement inside individual teams, and a cost base that looks much as it did before.
Data Matters, But It Is Rarely What Stops You
Data quality gets blamed for this, and it is a real constraint — you cannot automate a judgement on information you do not trust.
But in most organisations I see, the data problem is understood and someone is working on it. What nobody owns is the decision to change how the work is organised once the tool does what it promised. That is not a data gap. It is an accountability one.
The distinction matters, because a data problem comes with an owner, a budget and a plan. An accountability gap has none of those, which is why it survives so much longer and attracts so much less attention.
What to Ask For Instead
If you are commissioning this work, four things change the conversation:
- Start from a business outcome — a cost, a wait time, a service level — rather than a use case.
- Name whoever is accountable for the end-to-end process, not for the pilot.
- Decide in advance what will stop happening if it works.
- Measure the change in that outcome, not model accuracy or how many people logged in.
The third one is the test. If nobody can say what the organisation will stop doing, the work is an experiment rather than a change — which is fine, as long as everyone knows that is what has been funded.
The technology question already has an answer, and it improves every few months without your help. The management question — what should this organisation stop doing, now that this has become cheap — is the one nobody has been asked, and it is the only one that changes anything.