A VP of Engineering posted this to Reddit last month:
"Last quarter I rolled out Microsoft Copilot to 4,000 employees. $30 per seat per month. $1.4 million annually. Three months later I checked the usage reports. 47 people had opened it. 12 had used it more than once."
The post got 1,100 upvotes. Not because it was unusual — because it was relatable.
The Shadow AI Problem Nobody Talks About
While that VP was getting 47 active Copilot users out of 4,000, something else was happening: engineers on the same team were using ChatGPT and Claude to write code, review PRs, and draft documentation.
They just were not using the approved tool.
This is shadow AI — unsanctioned AI use that happens because the approved tool was not introduced well enough to become habit.
You paid for Copilot. Your engineers are using a competitor product instead. And you have no visibility into either.
Why This Happens
From the same Reddit thread, the top comment (5,600 upvotes):
"I do not think they convinced anyone what the use cases are for Copilot. Most people do not ask many questions when using their computer — they just click icons, read, and scroll."
That is the diagnosis. Employees were not given a reason to change behavior. They got an IT email and a recorded webinar. That is not training. That is notification.
The default behavior — click icons, read, scroll — does not change unless someone shows you exactly which tasks to do differently and why the AI version is better for those specific tasks.
The Measurement Gap
Here is what makes the 47-of-4,000 problem hard to solve: most companies did not measure before rollout.
They do not know:
- How long engineers spent on PR reviews before Copilot
- How much time went into first-draft documentation
- What the baseline was for code generation tasks
So when Finance asks "is this working?" — there is nothing to compare against. Only 18% of companies establish a utilization baseline before rollout. The other 82% are flying blind.
The Fix Is Not Complicated
The companies that hit 65%+ utilization within 30 days do three things differently:
1. They pick one workflow. Not "use it for everything" — one specific, high-frequency task where the time savings are obvious. For developers, pre-PR review is usually the winner. For ops teams, it is meeting summaries.
2. They train by role, not by tool. Generic "here is how Copilot works" demos do not stick. Role-specific sessions that answer "what does this replace in your actual workday?" do.
3. They share wins visibly. Teams that sustain usage post weekly wins. Isolated wins do not compound. Public wins create culture.
What This Actually Costs
Back-of-envelope for a 100-person engineering team:
- Copilot licenses: $30/seat × 100 = $3,000/month
- At ~1% utilization: you are getting value from roughly $36/month of that spend
- At 65% utilization with training: value from $1,950/month
That is a $1,914/month gap. A one-time training investment closes it permanently.
We built a free calculator that lets you run this math for your actual team size:
👉 askpatrick.co/roi-calculator.html
Ask Patrick helps engineering teams close the gap between "we deployed this tool" and "everyone actually uses it." Flat-fee co-work sessions, no per-seat pricing. askpatrick.co
Top comments (0)