Six months after your Copilot deployment, your CTO asks: "Is it working?"
If your answer is a shrug, you're not alone. Most engineering orgs have no structured way to evaluate AI tool adoption. They know seats purchased and licenses active — but they have no idea if the tools are changing how engineers work.
Here's a 90-day retrospective format we run with teams after a training engagement. It takes 90 minutes and produces a clear read on where you stand.
Why 90 Days?
- Week 1–2: Novelty phase. Engineers try it, get inconsistent results, form early opinions.
- Week 3–6: Plateau. Enthusiasm drops. Usage falls unless engineers found specific workflows that stuck.
- Week 7–12: Either adoption is happening or it's not. At 90 days, the pattern is set.
This is when the data is meaningful.
The 4 Questions Your Retrospective Should Answer
1. What's our actual utilization rate — and how does it compare to our baseline?
If you didn't set a baseline before rollout, you can't answer this. If you did: compare completion rates, feature usage, and time-in-tool from your analytics dashboard.
Benchmark: Teams with structured training typically hit 60–75% weekly active usage at 90 days. Teams with email-only rollouts typically sit at 20–35%.
2. Which workflows changed — and which didn't?
Survey your team (5 questions, anonymous, 3 minutes):
- What tasks do you now use Copilot for regularly?
- Where did you try it and stop? Why?
- What's one thing you wish it did better?
- Did your PR review time change?
- Would you recommend it to a peer at another company?
The answers tell you where the training gaps are.
3. Are the skeptics converting or cementing?
In most teams there's a 30/50/20 split at day 1: 30% enthusiastic adopters, 50% neutral, 20% resistant. At day 90, check where that 50% landed. If they moved toward adoption, your rollout worked. If they drifted toward resistance, you have a training problem that's getting harder to fix.
4. What's the ROI story you could tell leadership?
You don't need perfect data. You need a story. Time saved per engineer per week × loaded hourly cost × team size = monthly value. Compare to licensing + training cost. If you can't construct this, you can't justify renewal.
The 30-Minute Exercise
Split your team into groups of 3–4. Give each group 15 minutes to answer:
- "What's one workflow that's genuinely faster now?"
- "What's one thing we tried and abandoned?"
- "If we were starting over, what would we do differently in week 1?"
Debrief as a full group. Record the answers. This surfaces the actual state of adoption faster than any survey.
What to Do With the Results
If utilization is 60%+: You're in good shape. Focus on deepening skill — find the 3–4 prompting patterns that are working and document them for the whole team.
If utilization is 30–60%: The plateau is real. The gap is almost always training — engineers learned basic prompting but never learned the workflows that make the tool essential. A targeted 3-hour session focused on their actual use cases typically breaks through this.
If utilization is below 30%: You have an adoption failure. The tool isn't integrated into workflows. Engineers have reverted to defaults. This is recoverable but requires a structured re-launch, not just another nudge email.
Wondering where your team actually stands? The free ROI calculator at askpatrick.co/roi-calculator.html runs the utilization math for your team in about 2 minutes. If you want to run this retrospective with a facilitator, that's what we do: askpatrick.co/contact.html
Top comments (0)