The $4M Question
A regional bank spent $4M on enterprise AI tooling. Eighteen months in, the CIO ran a dashboard query and discovered weekly active users sat at 11% of the licensed seats. He called me and asked the question every CIO in this position eventually asks: "Did the technology fail, or did the organization fail?"
The technology hadn't failed. The licenses were active. The integrations worked. The training had been delivered. The vendor's reference architecture was implemented to spec.
The organization had failed at something most AI strategies don't even measure: adoption.
Two Axes, Not One
Most AI strategy conversations conflate two completely independent things:
Capability is what the technology can do.
- Models deployed
- Integrations live
- Licenses purchased
- Features enabled
- API call volume
Adoption is what humans actually do with the technology.
- Weekly active users in the target population
- Workflows redesigned around AI
- Decisions accelerated
- Outcomes attributable to AI-influenced work
- Time-to-result on AI-eligible tasks
These are independent axes. You can be high capability / low adoption (the $500K shelfware problem). You can be low capability / high adoption (a small team doing brilliant work with free tools). You can be high on both, or low on both.
The AI Usage Maturity Model (AI-UMM) treats this as a 2x2. Most enterprise programs cluster in the high-capability / low-adoption quadrant. That is the most expensive quadrant to be stuck in, because the operating budget keeps charging the licenses regardless of the workflow change.
Why Capability Metrics Are Easier (And Misleading)
If you go back through the last three quarterly business reviews at most large enterprises, the AI section reads like a procurement report:
- "We deployed Model X in Q3."
- "We integrated AI Tool Y with Salesforce in Q4."
- "We rolled out training to 5,000 employees."
These are capability metrics. They are easy to measure. They are easy to defend. They are also nearly worthless as predictors of business outcome.
A capability metric tells you what's possible. An adoption metric tells you what's happening. The difference between possible and happening is where most enterprise AI value gets stuck.
Four Adoption Metrics That Actually Matter
If your AI dashboard only shows capability metrics, you are flying blind on the half of the strategy that actually drives business outcome. Add these four:
1. Weekly active users in the target population. Not licensed seats — that's a capability metric. The denominator is "people whose job is supposed to change because of this tool." The numerator is "people who used it productively this week." If the ratio is below 30%, you are in the Pilot Plateau regardless of how the rest of the dashboard looks.
2. Workflow change rate. Pick the top 10 workflows the AI was supposed to influence. For each one, measure the percentage of work units that now flow through the AI tool versus the legacy path. If this number is not moving quarter-over-quarter, your investment is not changing how work gets done — it's just adding a parallel system.
3. Time-to-result delta. For AI-eligible tasks, what is the median completion time today versus six months ago? If this number is flat or worse, you have an integration problem (the AI is being used but is not faster) or a usage problem (the AI is being used wrong).
4. Quality drift. Quality at the same speed is fine; quality drop at the same speed is a hidden failure. Audit a sample of AI-influenced outputs against pre-AI baselines. Catch the regressions before customers do.
The Pilot Plateau
Stage 2 in AI-UMM is "Productive Pilots." It is where most enterprise AI programs go to die. Why? Because Stage 2 is comfortable.
- Executives can point to a working pilot at the next board meeting.
- Innovation teams can claim progress without organizational disruption.
- IT can manage risk by keeping AI in a controlled sandbox.
- The pilot team feels like rockstars.
No one in this configuration has a strong incentive to push to Stage 3 (Scaled Capability), because Stage 3 means actual organizational change: procurement decisions across business units, workflow redesign in functions that didn't run the pilot, performance metrics tied to AI-influenced outcomes, and operating model adjustments.
The Pilot Plateau is not a technology problem. It is an organizational design problem. The leaders who break out of it do three things:
Set Stage 3 success criteria at the start of the pilot, not after. "If this pilot works, here is what we will scale, who will own the scaling, and what budget is pre-approved." If you can't write that paragraph at pilot kickoff, your pilot will plateau.
Identify the Stage 3 sponsor on day one. This is usually NOT the pilot sponsor. The pilot sponsor is rewarded for innovation; the Stage 3 sponsor is rewarded for operational adoption. Different incentives, often different people. If you don't name them on day one, you don't have a path to Stage 3.
Treat the pilot as a hand-off exercise, not a proof-of-value exercise. A successful pilot ends with the operations team saying "we'll take it from here," not with the innovation team writing a celebration deck.
What This Means for Your Roadmap
Go pull your current AI roadmap. Count the milestones that are capability milestones (model deployed, integration shipped, training delivered). Count the milestones that are adoption milestones (workflows changed, weekly active users hit X, time-to-result improved by Y).
If the ratio is heavily skewed toward capability, your next quarterly review is going to be uncomfortable. The CFO will ask "what did we get?" and your roadmap will answer "we deployed things." That is not the answer the CFO is looking for.
The fix is not more capability investment. The fix is to reframe at least half the milestones around adoption and outcome. Some of those milestones will require organizational change that the IT function alone cannot deliver. That is the point. AI value at enterprise scale is an organizational design challenge, not a procurement challenge.
The Bottom Line
The bank in the opening recovered. We mapped 12 high-frequency workflows to specific AI use cases, identified non-IT champions inside each function, and tied 30% of digital transformation OKRs to adoption metrics. Twelve months later, weekly active users hit 64%. Same tools. Same training material. Different organizational design.
If your enterprise AI program feels stuck, the diagnostic is simple: pull up your dashboard and ask "is this measuring capability or adoption?" If it's capability, you don't have a strategy yet — you have a procurement plan.
Capability without adoption is shelfware. And shelfware shows up in the operating budget every single month.
This article is adapted from a LinkedIn series on the AI Usage Maturity Model.
Top comments (0)