Enterprise AI & Strategy · Project approach
Why Do So Many AI Projects Fail Before They Reach Production?
Many AI projects impress in the pilot and never make it into daily work. Often the reason is that they are planned like classic IT projects: fixed requirements, an acceptance date and the idea that the work ends at go-live. AI behaves differently.
Julia Rose
Key takeaways
- According to IDC, 88% of AI proofs of concept do not reach wide-scale production, and only four of 33 pilots go live.
- The often-quoted Gartner figure of 85% describes a 2018 prediction about erroneous outcomes caused by bias, not a failure rate.
- AI results depend on data. How good a model becomes only shows with real data.
- After go-live, data and conditions change. AI therefore needs ongoing monitoring and adjustment.
What the numbers say
The most-quoted current figure comes from IDC: 88% of the AI proofs of concept studied do not reach wide-scale production. Of 33 pilots launched, only four went into production.
Source: CIO.com on the IDC study with Lenovo
You also often read that, according to Gartner, 85% of all AI projects fail. The source does not say that. In February 2018 Gartner predicted that through 2022, 85% of AI projects would deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them. The prediction still points to where the risk lies: in data and approach.
Source: Gartner press release, 13 February 2018
How AI projects differ from IT projects
In classic software, the result of every program step can be predicted. AI models learn patterns from data and produce probabilities. That changes planning fundamentally:
| Classic IT project | AI project | |
|---|---|---|
| Behaviour | defined by rules | learned from data, with uncertainty |
| Requirements | can be fully specified upfront | target values that can only be confirmed with real data |
| Testing | function works or not | quality measured across many real cases |
| After go-live | maintenance when things change | ongoing monitoring, because data changes |
| Risk | bugs in the code | data that does not match reality |
Data and expectations
A model is as good as the data it learns from and how closely that data reflects reality. If training data is collected only under controlled conditions, the model often performs much worse in daily use. On top of that come edge cases missing from the dataset.
A common mistake is expecting very high metrics before it is clear how well the task can be solved today and whether reliable data exists. That is why the start should be: understand the business problem, check the data and agree realistic target values with everyone involved.
After go-live: data changes
With classic software, a project often ends with acceptance and a support phase. With AI, that is where a new stage begins. Over time the distribution of the data shifts, known as data drift: customers behave differently, products and forms change, new terms appear. A model that does not react to this slowly gets worse.
The same applies to applications built on large language models. They only know their training data, and a model change at the provider or new documents change the results. That is why every AI application in production needs:
- Monitoring: quality, errors and cost are measured continuously.
- Test sets: changes to model, prompt or data are checked before use.
- Adjustment: models are retrained or prompts and knowledge base updated.
- Ownership: one team owns it and has time for it.
How this is organised is covered in our articles on MLOps and LLM observability.
An approach that leads to production
- Analysis: clarify process, data and value before development starts.
- Proof of concept with real data: show that the task can be solved, including edge cases.
- MVP in production: a first version with real users, connected to existing systems.
- Expansion: further cases and areas only once the metrics are right.
- Operation: monitoring, adjustment and clear ownership planned from the start.
What this path looks like in our projects is described in Building AI that works: our MVP-first approach. For AI agents specifically, read The pilot worked, production didn’t.
Want your AI project to reach production?
We review process, data and goals with you and plan the path from first test to operation.
Request a process analysisFrequently asked questions
Common reasons are data that does not match reality, high expectations without first checking the data, missing integration with existing systems and the assumption that the work ends at go-live.
Not quite. In 2018 Gartner predicted that through 2022, 85% of AI projects would deliver erroneous outcomes due to bias in data, algorithms or the teams responsible for managing them.
The gradual change in data after go-live, for example through new customer behaviour, new products or new terms. A model that is not adjusted delivers worse results over time.
With an analysis of process and data, a proof of concept with real data, an MVP in production, step-by-step expansion and monitoring and adjustment planned from the start.
Julia has been part of theBlue.ai since 2019 and has accompanied the development of AI applications in the enterprise environment since the company’s early days. In her role as Marketing Lead, she works closely with the engineering and consulting teams and makes complex technical topics understandable and accessible for decision-makers.
In her articles, she writes about practical experience from enterprise AI projects, as well as the challenges and opportunities of using AI in companies.