Suppose leadership rewards teams for increasing the percentage of “AI-assisted pull requests.” The dashboard rises. Did productivity improve, or did people learn which box to tick?
Before launching that metric, I would run a consequence-mapping session:
| Intended behavior | Plausible adaptation | Counter-metric |
|---|---|---|
| try useful assistance | label trivial PRs as assisted | retained task outcome |
| ship faster | split work into tiny PRs | lead time per task |
| share adoption | avoid difficult non-AI work | task-mix distribution |
| accept suggestions | reduce review scrutiny | rollback and defect rate |
The metric card should make disagreement possible:
name: ai_assisted_pr_share
purpose: detect workflow adoption, not productivity
owner: developer-experience
known_game: self-label inflation
counter_metrics: [task_mix, review_minutes, rollback_rate]
review_date: 2026-08-19
retire_when: classification cannot be audited
Then interview both high and low scorers without treating the score as performance. Ask what work disappeared, what new verification appeared, and what behavior the dashboard encouraged. Include an anonymous channel: a metric cannot reveal pressure if challenging it carries career risk.
The SPACE framework argues that developer productivity cannot be captured by one dimension. That is especially relevant when AI telemetry is easy to count but verification and rework are harder to observe.
My launch gate is not “the metric is accurate.” It is: teams can inspect its definition, challenge its interpretation, and show where it changes behavior. If the counter-metrics diverge, pause incentives before refining the chart.
What behavior would your current AI dashboard accidentally reward?
Top comments (0)