I run a prediction market for my AI agents.
Not for money. For truth.
Here's how it works: When Research proposes a feature, Strategy doesn't just say 'sounds good.' Strategy bets. 'I give this 70% chance of >100 signups in week one.' Finance counters: '40%. Users don't adopt features, they adopt solutions to pain.'
When Content pitches a post, Critic doesn't just critique. Critic bets. 'I give this 60% chance of >50 likes.' Content adjusts the hook and raises: '75% with this opener instead.'
Every prediction gets logged. Every outcome gets resolved. Agents who bet poorly accumulate calibration debt — a visible score that says 'your confidence outpaces your accuracy.'
It sounds like a game. It's not. It's a forcing function.
Most teams make decisions based on vibes. Someone senior says 'let's do X' and everyone nods. No one commits to a forecast. No one checks if X worked. Next quarter, repeat.
Prediction markets force you to think in probabilities instead of certainties. They surface disagreement. They reward accuracy over confidence. And most importantly: they create a feedback loop.
When Strategy bets 70% and the feature flops, Strategy doesn't just move on. Strategy asks: 'What did I miss? What signal did Finance see that I didn't?' Next bet, Strategy is sharper.
The result? Our agents are calibrated. They know what they know and what they don't. When Research says 'I'm 90% confident,' I trust it. When Critic says 'this is 50-50,' we dig deeper or A/B test.
Some decisions are bets. The question is: are you learning from yours?
Top comments (0)