5 rules to avoid the 95% AI project failure rate

Here’s the hard truth: 95% of corporate AI pilots fail to deliver ROI.

That’s not consultant hype—it’s from MIT’s NANDA initiative, reported by Fortune. And it should wake up every board and C-suite that’s been promising a productivity revolution.

So why does AI project success remain stuck at just 5%?

Because AI isn’t just a technology problem. It’s a transformation problem. Transformation from a business sense means you need to reconsider how you operate differently. Transformation is about rethinking every part of your business. From how you make decisions to how you manage logistics to how you deliver a product or service.

For example, if you think about a call center, could you replace it with an AI powered chatbot? What would that mean for your business? Would it succeed or fail?

After working on dozens of transformation programs, I’ve seen a clear pattern: the winners all have 5 things in common. Miss even one, and odds are your project joins the 95%.

5 non-negotiables for AI-driven transformation

1. Quantify success

If your AI-enabled project doesn’t connect directly to revenue growth, cost reduction, or profitability, don’t start it. If it does, how does it drive your AI solution choices? Vague value = guaranteed failure. Specific expected outcomes require specific technology choices.

In many engagements, I’ve often been confronted with the following answers when I ask, “what does success look like to your company?”  

• More uptime
• Higher profits
• Bigger share
• Faster processing
• The thing to be delivered on time and on budget  

Focusing a particular client in health care, if I asked for a quantifiable or measurable outcome I would inevitably get, “More uptime than we have now.”

I eventually got from the operations person, “We want 99.9% uptime.”

Then I asked, why 99.9%? And with a straight face, they said because 100% uptime is impossible.

When I asked what 1% downtime actually cost the business, they had no idea.

When we finally did the analysis, we realized the business didn’t really suffer until they hit 3% downtime, and it cost them about 2% of their business.

Finally, we were able to ask if 2% of the business was ~$5M annually, what 5-year ROI would they need to have in order to make the project viable. That number became our criteria for success.

2. Lead agile from the top

Agile and product principles aren’t just for the CIO. The CEO, COO, and CFO have to run on them too—or the org stays stuck in old patterns.

In more cases than I care to remember, the CIO would tell us that they were an agile shop and the company supported it. I would then get a list of features and functions, a timeline for a production release, and a budget.

I get it. It’s human nature to inject certainty into a process, but it creates an expectation around a false sense of control. That’s why waterfall is no longer prevalent in software development.  So it’s important to ask …  

• Why these features and functions?
• Why this deadline?
• How did you arrive at this budget?

The response typically went something like this. “Well, the business defined the features, the COO (or CFO) defined the deadline, and the CFO defined the budget.”  When I asked based on what? Crickets.

When I pointed out that none of those things supported an agile process … crickets, again.

I would pivot the discussion to techniques like Lean Value Tree and MVP testing to identify and test quantifiable bets the company is making about value.

The intent should be to validate each of those bets to focus on the features and functions that drive value. 

I would often get pushback that the C-suite just doesn’t do things that way.  This way of thinking guarantees missing expectations around delivering on time, on budget, and with the expected ROI. 

3. Make the C-suite adapt

Most projects fail not because of bad tech, but because leaders resist new ways of working and funding. MVPs aren’t about testing features—they’re about testing ideas.

On a particular project for a health care tech company, I had access to some of the C-suite, but not all.

It was clear that the C-suite didn’t have confidence in agile. They thought it was just an excuse to not have deadlines, and the freedom to burn through budgets.

When I finally got in front of the entire C-suite, I was able to be clear about how they were failing the company and their shareholders.

Yes, it was a very tense meeting.

I pointed out to the Chief Product Officer (CPO) that by not making a bet on the outcome, they were guaranteeing failure.  

• The CEO cannot tell the shareholders why they are investing in this product
• The CFO has no way of determining what a reasonable budget is
• The COO has no way of understanding how much they should invest in making organizational changes
• The CIO had no way of being successful, because they couldn’t actually leverage agile to drive speed to value
• Finally, they were setting themselves up for failure by not testing the collection of ideas that make up a product

I pointed out to the CFO that by holding the team to an annualized budget, they were dooming the project to failure.

• The CPO couldn’t plan for a product budget without the ability to put together a multi-year plan—they simply threw as much at the wall as they could and hoped that something good happened
• The CIO over-indexed on optimizing the capital budget, without worrying about how all the technical debt would drive up their operating expenses over time
• Finally, they set themselves up for failure because the budget constraint became the overriding factor in the project, not driving ROI

And I did the same thing for each person in the C-suite.

While initially dubious, they ultimately agreed and aligned on an approach to continuously drive ROI while lowering their expected operating expense. This resulted in a significantly higher ROI than the initial estimate they wanted.

4. Change the org, not just the tech

If you drop AI into broken structures, it will only reinforce your inefficiencies. Transformation requires rethinking the organization, not just the software.

Any transformational project is not going to deliver to expectations if the organization doesn’t adapt to it.

As a simple example, my first job after college was conducting cost studies at a telephone company. For years, they did all the calculations by hand on large spreadsheets. After they did the first calculations, the cost analyst checked their math with a calculator, put blue dots by each calculation, and a copy of the paper tape got stapled to it. Then a clerk rechecked the math, doing the same things but using a red dot.

When I got there, they introduced the first desktop computer and I had access to Lotus 123 (yes, I am that old). When I finished my first cost study on the computer and printed it out, they asked me to recheck the math (which I told them was unnecessary), and then hand it to a clerk to be checked again. It may have taken two years before they began to trust Lotus 123.

Obviously, this desire to keep old systems and processes in place was a waste of time and effort. Eventually the phone company figured it out, but it took quite a while.

5. Fix your fundamentals

Even the best AI will not overcome communication gaps, data issues, and inconsistent decision-making approaches. Getting these fixed and aligned is the hardest problem to solve, but drives the quantum leap everyone is looking for.

When implementing a sophisticated analytic system at a large grocery chain, it became apparent that their attempts to build a data dictionary did not serve them.  When we got into the system, simple things like sales, which were officially documented in one place, did not match all the places where it was used.

In some cases, it was total sales, and in others it was total sales minus promotions and coupons. 

If you are creating sales projections to review expected revenue for a C-suite meeting, and two executives produce two different projections using the same calculations, how much confidence does the CEO have in those forecasts? How can you even project quarterly revenue to your board, much less make more tactical decisions?  

Extending it further, what if you are looking at out-of-stock metrics? How you calculate out-of-stock matters:

• Are you including expected spoilage, damage, and theft with item sales?
• Are you including the expected change in sales due to an upcoming promotion?
• Did anyone tell the store that there was an expected promotion coming?
• If you are expecting an unseasonable cold snap, do you have enough cans of soup?

Knowing what data is used and how it's used is table stakes for enabling AI.  

Understanding existing, or more crucially missing, communication and how that impacts decisions, including who, where, and how decisions are made are critical to actually enabling AI to do its thing.  

A simple truth

At the end of the day, a company is a decision-making machine. Networks, data, and heuristics keep it running.

If you try to “bolt on” AI without changing those fundamentals, you’re just recreating a faster version of the Titanic. But if you rebuild those foundations and apply the rest of the non-negotiables, your odds of success increase dramatically.

Ed Frank is VP of Customer Experience & Operations at Test Double, and has led software implementations for 30 years as a consultant.  He has experience in leading consulting organizations and working closely with C-Suite executives in a variety of industry verticals.