Every AI pilot starts the same way. A conference. A vendor demo. A board member who saw something on LinkedIn. The kickoff is energetic — there's a Slack channel, a roadmap, executive buy-in. The team builds something in two weeks that demos beautifully. And then…
Nothing. The pilot ships into a slide deck and stays there. Twelve months later, leadership asks where the AI strategy went, and nobody has a clean answer.
We've sat in dozens of these post-mortems. The reasons cluster into four buckets. None of them are technical.
1. The pilot scoped a demo, not a deployment
The team built something that worked on five test cases. Production needs it to work on 5,000. The gap between those two looks small in the demo and is enormous in reality. It includes auth, permissions, edge cases, retry logic, monitoring, rate limits, fallback flows, an admin panel, and someone on call when it breaks.
None of that fits in a two-week sprint. So the pilot ships looking great and the production version costs four times what was budgeted. Rather than honestly re-budget, the project quietly dies.
2. There was no human to hand it to
You can't deploy an AI workflow into a vacuum. Someone needs to own the daily operation: review the logs, handle the escalations, maintain the prompts, retrain the model when the world changes. If that owner doesn't exist before launch, the system rots.
We won't take an AI project on if there isn't a named human who'll own the system on day 31.
3. Success was never defined in money
"Reduce manual effort." "Improve customer experience." "Modernise operations." None of these survive contact with a CFO. The successful pilots we see all started with a number — pounds saved, hours reclaimed, revenue retained, errors caught. Vague success criteria mean any failure can be re-framed as success, which sounds nice and means the project loses momentum because nobody can prove it's working.
4. The org wasn't ready to change
This is the quiet one. The pilot worked, the metrics were real, and the team still didn't roll it out — because rolling it out meant restructuring a department, ending a vendor contract, or admitting that a senior manager's process was redundant. The AI was the easy part. The org change was the hard part. The org change didn't happen.
What we do differently
Before we start building, we ask four questions. They're unglamorous and have killed several engagements before they began.
- What number, in pounds or hours, defines success?
- Who is the named human who will own this on day 31?
- What organisational change must happen for this to deploy?
- What's the cheapest version of this that proves the value?
If a project can't answer those, we don't say no — we ask the leadership team to answer them with us, before any code is written. Sometimes that takes two weeks. It always saves four months.
The AI is the easy part now. Boring discipline around scope, ownership, and measurement is what separates the pilots that ship from the ones that don't.