What 1 million retro cards say dev teams actually complain about 👨‍💻👩‍💻

#ai #agile #scrum #productivity

Every agile retrospective produces a small pile of sticky notes, and then the pile disappears. Multiply that by 86,977 retros and you get something more interesting: a dataset of what software teams say is wrong, in their own words, when nobody is performing for an audience.

I work on Kollabe, so we happen to be sitting on exactly that pile: 1,071,253 retro cards written by real teams between October 2023 and July 2026. Last week I finally ran the analysis I'd been putting off. I expected communication to top the complaint list, because I have personally written that think piece.

It came in eleventh. This post is the league table, the methodology (SQL + pgvector, nothing exotic), and the three numbers that changed how I run my own retros.

The methodology (the part you'd actually ask about)

Everything is aggregate analysis on our production Postgres replica. No card text leaves the database; nothing below quotes an individual card.

The approach was deliberately boring. Retro boards have named columns ("What Went Well", "Glad", "Anchors", "What Went Wrong"), so classifying column names splits the corpus into 254,653 complaint cards (23.8%), 314,630 positive cards (29.4%), and a big neutral middle: Start, Learned, and Lean Coffee columns that don't lean either way. Twenty keyword patterns then bucket the complaints into themes in a single pass. Keyword matching is conservative and English-centric, so every share below is a floor.

To sanity-check the buckets, we leaned on the pgvector embeddings we already keep for AI features (512 dimensions, HNSW-indexed). Embed a probe sentence like "pull requests sitting too long waiting for code review" and pull its nearest neighbours: the 1,000th nearest card is still unmistakably about slow reviews. There are at least a thousand ways to phrase that one complaint, and teams have used all of them.

The league table

Share of all 254,653 complaint cards:

Rank	Theme	Share
1	Testing and QA	10.2%
2	Tickets and requirements	8.1%
3	Deploys and releases	4.9%
4	Estimation and planning	4.5%
5	Bugs and incidents	4.0%
6	Code review and PRs	3.0%
7	Meetings and ceremonies	2.7%
8	Environments, CI, tooling	2.6%
9	Documentation	2.2%
10	Dependencies and blockers	2.2%
11	Communication	1.9%
12	Deadlines and delays	1.9%

Teams complain about testing more than five times as often as they complain about communication. Jira, as a named tool, gets 1.2% of all complaints, nearly tying communication. So does "people being on vacation".

I don't think this means communication is fine. I think when people describe problems in their own words, they name the concrete stage where work got stuck: the ticket with no acceptance criteria, the release that slipped. "Communication" is the abstraction we reach for afterwards, once the details have faded. The retro card, written the same week, still has the details.

What teams write is not what they vote for

Kollabe retros let people vote on cards, which gives you a second signal: not what's on people's minds, but what they want fixed. The two disagree.

The average complaint card gets 0.64 votes. Code review complaints get 0.80, the highest of any theme, 25% above baseline. Workload and burnout cards get 0.79, despite being only 1% of complaints by volume. Scope creep and tech debt both sit at 0.77.

Testing, the volume champion, gets 0.63. Below average.

My read: testing complaints are frequent but routine, the background radiation of software work. Review bottlenecks and overload are the ones people are quietly waiting for permission to escalate. When a workload card appears, the room piles on: someone said the thing.

Also worth knowing: 76% of complaint cards get zero votes, and teams vote on problems 36% more than on wins (0.64 vs 0.47 votes per card). Retros are for fixing things, whatever the icebreaker says.

Complaints are sticky

We paired 9,250 consecutive retros from the same team and asked: if a theme appears this retro, what are the odds it appears next retro?

Every theme repeats above its base rate. Communication is the stickiest of all: a 9.1% base rate, but 21.3% if the previous retro mentioned it, a 2.3x multiplier. Workload is 1.9x. Meetings and documentation are 1.8x.

This is where the "it's always communication" camp gets to be right, and I want to be fair to them, since I was in it. Few teams complain about communication, but the ones that do tend to stay stuck with it. Process problems like testing recur because they're everywhere; communication recurring at 2.3x its base rate looks less like a process gap and more like something structural the retro alone can't fix.

The uncomfortable part was checking whether my own team's boards had a recurring theme. We did. Three retros in a row, same complaint, slightly different wording each time. Nobody had noticed because each retro felt like a fresh conversation.

The action item numbers are grim, and the fix is embarrassing

Only 19.6% of retrospectives produce even one action item. Of the action items that do get created, 68.7% are still sitting in pending. 13.2% ever reach done.

But the completion data has one loud signal in it. Action items with an assigned owner complete at 23.2%. Without one: 8.6%. That's 2.7x, for the cost of saying a name out loud. Due dates are rarer still, 1.1% of items have one, but those complete at 37%.

I'd love to tell you the fix for retro follow-through is sophisticated. It's a name and a date, typed into the card before the call ends.

Honest limitations

Keyword bucketing undercounts non-English cards and clever phrasing. Column-name classification puts roughly 47% of cards in a neutral bucket that never gets sentiment-analyzed. The vote data only exists for teams that use voting. And this is one tool's user base, which skews toward teams that already care enough about retros to use a dedicated app. Your dysfunction may vary.

About half of complaint cards matched none of my twenty themes. Those are the genuinely local problems: a specific outage, a specific person's last sprint before parental leave. No dataset fixes those for you.

The Monday-morning version

Three things I've changed from staring at this data:

Sort by votes, discuss top-down, always. The most-written theme and the most-voted theme are usually different, and volume is the worse signal.
No retro ends without the action item ritual: owner and date on every item, out loud. The 8.6% vs 37% spread is too big to ignore.
Grep your own boards for repeats. If a theme shows up two retros running, stop filing it as feedback. It's a project now.

The full analysis, with the recurrence table, the anonymity myth-busting (anonymous cards are slightly less negative, it turns out), and the cadence data, is in the long-form write-up: what 1 million retro cards reveal. If you'd rather poke at the numbers yourself, the live retrospective statistics page is public.

Grep your last three retro boards before your next one. If the same card keeps coming back, you don't have a feedback problem. You have a follow-through problem, and now there's data on exactly how cheap the fix is.