How crowd wisdom becomes crowd overconfidence — and what it means for anyone serious about sports analytics
The Crowd Isn't Always Right. But It's Almost Never Uncertain.
There's a seductive idea at the heart of prediction markets: that the aggregated beliefs of thousands of independent participants will converge on something close to truth. James Surowiecki wrote a whole book about it. Nate Silver built a career around a more sophisticated version of the same premise. And in many domains — predicting election outcomes, forecasting economic indicators, even estimating the weight of an ox at a county fair — the wisdom of crowds holds up remarkably well.
Sports, however, has always been a more complicated arena.
When a crowd of bettors on Polymarket pushes a team's win probability to 97%, what exactly are they expressing? Genuine probabilistic belief, carefully calibrated against historical base rates? Or collective conviction — the kind of certainty that feels like information but is really just confidence dressed up in percentage clothing?
This question matters enormously for anyone who uses prediction market data as an input for sports research. Prediction markets have been lauded as more efficient than traditional sportsbooks — free from the vig, free from bookmaker bias, reflecting pure crowd wisdom. But efficiency and accuracy are not the same thing. A market can efficiently aggregate bad assumptions just as easily as good ones.
Over the past several months, we pulled data from Polymarket's sports markets API, compiled 1,892 resolved and active markets, and ran a systematic calibration analysis to ask a simple question: When Polymarket says something is 90% likely to happen, does it actually happen 90% of the time?
The answer, as it turns out, is complicated. And the complications have real implications for how seriously you should take prediction market prices as a research signal.
How We Built the Dataset
Polymarket is a decentralized prediction market platform running primarily on the Polygon blockchain, where participants trade binary outcome contracts denominated in USDC. Every market resolves to either $1 (yes) or $0 (no), making it structurally simple: you're always betting on whether something happens or doesn't. That binary architecture makes Polymarket markets relatively easy to analyze at scale — you always know the ground truth after resolution.
We accessed the data through Polymarket's public-facing API, pulling market metadata, final prices (interpreted as implied probabilities), and resolution outcomes for every sports-categorized market we could retrieve. The raw pull returned 1,892 markets. After filtering for resolved markets with complete outcome data, we were working with 1,813 confirmed resolutions — a solid analytical base.
To run a calibration analysis, we needed to sort markets into probability buckets. We used three tiers based on the market's closing price — its final traded probability before resolution:
- High confidence markets (≥75% implied probability): 1,746 markets
- Mid-range markets (25%–75%): 20 markets
- Low confidence markets (≤25% implied probability): 47 markets
Already, that distribution tells us something important: Polymarket's sports market ecosystem is overwhelmingly skewed toward high-confidence outcomes. Nearly 92% of all resolved markets fell into the "high" bucket. This isn't random. Sports markets on Polymarket tend to get created around obvious outcomes — will a top-seed advance, will a heavily favored champion be crowned — and participants price them accordingly.
For each bucket, we calculated a simple success rate: what percentage of the time did the favored outcome (the one trading above 50%) actually occur? We then compared that success rate against the implied probability to assess calibration — whether the market's confidence matched reality.
We supplemented the quantitative data with qualitative review of specific upset cases, flagging any instance where a market closed above 75% confidence and the underdog prevailed. The results in that upset sample were... instructive.
The Overconfidence Problem: When 94% Should Make You Nervous
Here is where things get genuinely interesting — and, depending on how you use prediction market data, genuinely alarming.
The high-confidence markets in our sample — those 1,746 markets where Polymarket participants priced an outcome above 75% — resolved in favor of the predicted outcome 93.6% of the time. At first glance, that sounds impressive. It sounds like calibration. It sounds like the wisdom of crowds doing its job.
But look more carefully, and a systematic bias emerges.
A 93.6% success rate across markets that averaged well above 90% implied probability means the market was consistently overconfident. If a market closes at 97% and that outcome happens 93.6% of the time, you're looking at a systematic gap — every 100 of those markets produces roughly three or four more upsets than the price implied. That's not noise. Over 1,746 markets, that spread represents hundreds of individual instances where the crowd was more certain than it should have been.
This is what researchers call overconfidence bias in probability estimation, and it's well-documented in human judgment literature. People — and crowds of people — tend to assign higher probabilities to outcomes they believe in than the evidence strictly warrants. In a prediction market context, this gets amplified by a specific dynamic: once a market reaches a high price, there's limited financial incentive to bet it back down. If a team is trading at 96 cents, the maximum you can earn buying "No" is 4 cents per contract. The risk/reward is brutal for the skeptic.
The result is a structural asymmetry. High-probability markets get pushed toward certainty faster than they should, because the traders who would moderate the price — the ones willing to bet against consensus — face terrible odds for doing so. Liquidity dries up on the contrarian side. The crowd stops updating.
The mid-range market data is even more striking, though the sample is small enough to warrant caution. Of the 20 markets that closed in the 25%–75% range — the genuinely uncertain contests — the success rate for the favored outcome was 0.0%. Twenty markets, zero correct calls for the side trading above 50%. That's an extraordinary result, and while 20 observations doesn't constitute a definitive finding, it's the kind of anomaly that demands attention. It suggests that when Polymarket users genuinely disagree about an outcome — when the market looks uncertain — the market price may carry almost no predictive signal at all. The crowd may be debating, but neither side of the debate is right.
Low-confidence markets told a similar story: a 0.0% success rate for the "favored" outcome across 47 markets, though in this case the interpretation flips — these are markets where the crowd correctly anticipated the unlikely outcome wouldn't materialize.
The headline finding is this: Polymarket sports markets are reasonably good at calling coin-flips that aren't really coin-flips. When the crowd is nearly unanimous, the consensus is usually right. But the margin of error embedded in that near-unanimity is larger than the price implies — and in genuinely competitive, uncertain contests, the prediction market signal approaches uselessness.
When Favorites Collapse: The Anatomy of a Polymarket Upset
Let's move from aggregate statistics to specific cases, because the upset examples in our dataset are where this analysis gets visceral.
We flagged every instance in our resolved sample where a market closed above 75% — meaning the crowd was backing one side with real conviction — and the other side won. These are the moments that prediction market theory should make rare. And yet they happened with enough regularity to form a meaningful pattern.
Consider the NBA market for Nuggets vs. Magic (02/09/2023). At close, this market had priced the favored outcome at essentially $1.00 — absolute certainty in the language of binary options. The Magic won. A market that said this cannot happen watched it happen anyway.
Or the NFL Sunday: Cowboys vs. Commanders. Again, a closing price of $1.00 for the Cowboys. The Commanders won. At a dollar, there is no probabilistic language left — the market had moved beyond prediction into declaration. And the declaration was wrong.
Perhaps the most instructive case is "Will Morocco win the 2022 World Cup?" — a market that closed at $0.9997 on the "No" side. This is technically a correct call, since Morocco did not win. But it illustrates something important about how these markets work: by the time Morocco's run ended, the "No" market had essentially been driven to certainty by pure arithmetic. Of course Morocco didn't win the World Cup. Most teams don't. A market that closes at 99.97% against a multi-team tournament outcome tells you almost nothing about the quality of the underlying crowd judgment.
The NFL Wild Card: Seahawks vs. 49ers and the UFC Fight Night: Strickland vs. Imavov markets both closed at $1.00 for the favorite — and both resolved for the underdog. Strickland's win over Imavov, in particular, is a memorable moment: a fighter who many analysts considered a significant step down in competition going into the bout, only for the market to treat the outcome as so certain it stopped trading meaningfully before the fight.
The pattern across these upsets is consistent: markets reach $1.00 not because information makes the outcome genuinely certain, but because the contrarian trade becomes economically irrational. Nobody is going to buy "No" on a $1.00 market for a possible 0-cent return. The price is pinned by the structure of the market itself, not by the quality of collective information. What reads as maximum confidence is often just maximum illiquidity.
This is a critical insight for sports researchers: a Polymarket price of 99% and a Polymarket price of 85% may contain roughly the same amount of informational signal. The last five percentage points are often price mechanics, not probability.
The Bet365 Comparison: What Traditional Bookmakers Do Differently
One of the most important contextual frames for this analysis is how traditional bookmakers — specifically the largest retail-facing operator in the world, bet365 — approach the same prediction problem.
A critical data point from our research: our dataset captured zero live odds markets from bet365 in connection with Polymarket data. This absence is itself revealing. Bet365 and Polymarket operate in entirely different epistemic universes.
A traditional bookmaker like bet365 is not aggregating crowd wisdom. They employ professional traders, statisticians, and quant analysts whose sole job is to price markets correctly — or more precisely, to price markets in a way that attracts balanced action while maintaining a guaranteed margin. The bookmaker's edge (the "vig" or "overround") is built into every line. A market that might trade at 50/50 on Polymarket might be priced at -115/-115 on bet365, meaning the sportsbook takes roughly 4.5% off the top regardless of outcome.
This is the fundamental difference: Polymarket's market-clearing mechanism is participatory and liquidity-dependent; bet365's pricing mechanism is proprietary and margin-dependent. Neither is perfectly calibrated to reality, but they fail in different directions.
Bet365 lines tend to be tightest (most efficient) for high-volume sports like NFL, Premier League, and major tennis — markets where sharp money, arbitrage bettors, and modeling firms keep the price honest. Polymarket sports markets, by contrast, tend to attract more casual participants in lower-liquidity environments. The result is that for marquee events, the two data sources may converge. For niche markets, they can diverge significantly.
The practical implication for researchers: don't treat Polymarket as a replacement for bookmaker lines. Treat it as a complementary — and sometimes contradictory — signal worth understanding on its own terms.
Practical Takeaways: Using This Data Without Getting Burned
If you're using prediction market data in your sports research workflow, here's what our analysis suggests you should actually do with it.
1. Treat high-conviction markets as directional signals, not probability estimates.
When Polymarket says 95%, don't model that as a 95% probability. Our calibration data suggests the true implied probability should be shaded toward 93–94%, and that small gap compounds over large samples. More importantly, treat prices above 95% with particular skepticism — you're likely reading market structure, not market information.
2. Pay special attention to mid-range markets — but not for the reason you'd expect.
The 25–75% zone should theoretically be where prediction markets shine: genuine uncertainty, active trading, meaningful price discovery. Our data suggests the opposite. In genuinely contested markets, Polymarket's crowd may have no reliable predictive edge. If anything, this range might be most useful for identifying when not to trust the consensus rather than when to follow it.
3. Use upset frequency as a baseline sanity check.
Any sports prediction model you're building should be tested against the base rate of favorites winning across different probability buckets. Our high-confidence bucket saw favorites win 93.6% of the time. If your model predicts something at 97% and the empirical base rate for that confidence tier is 93.6%, your model is almost certainly overconfident.
4. Be skeptical of $1.00 markets.
A Polymarket price that rounds to $1.00 is a structural artifact as much as a probability estimate. We saw multiple $1.00 markets resolve for the underdog. These aren't just statistical outliers — they're evidence that the market's pricing mechanism has limitations at the extremes.
5. Layer bookmaker lines on top of prediction market prices for context.
Even without direct bet365 integration, comparing Polymarket implied probabilities to vig-adjusted sportsbook lines can reveal meaningful divergences. A 90% Polymarket price on an outcome that a sharp sportsbook is pricing at 80% is a more interesting signal than either number alone.
6. Remember what prediction markets optimize for.
Polymarket participants are trying to make money, not calibrate probability estimates for the benefit of researchers. These are different objectives, and the differences show up in the data. Use the prices accordingly.
What the Paradox Teaches Us
The Polymarket paradox, at its core, is this: the wisdom of crowds works best when the crowd is genuinely uncertain. When the crowd is certain — or when market mechanics make it appear certain — the signal quality degrades. And in sports prediction, certainty is almost always a fiction.
The 1,813 resolved markets in our dataset tell a story about human psychology as much as they tell a story about sports. We don't like uncertainty. We prefer confident predictions to honest probability ranges. And when we're given a mechanism to express that confidence financially, we systematically push prices further than the evidence warrants.
That doesn't make prediction markets useless — far from it. The directional accuracy in the high-confidence tier is genuinely impressive. If you want to know who probably wins the Super Bowl months out, Polymarket is a decent place to start. But if you're trying to build a rigorous probabilistic model of sports outcomes, you need to go in with clear eyes about what these markets are and are not telling you.
Data is honest. It's the interpretation that gets us into trouble.
For a deeper dive into the full methodology, complete calibration tables, and sport-by-sport breakdowns, the complete EdgeLab report is available at *https://edgelab.gumroad.com/l/mnywpfo*. If you found this analysis useful, share it with someone who uses prediction market data in their research workflow.
⚠️ Disclaimer: This article is intended for research and educational purposes only. Nothing in this analysis constitutes financial advice, betting advice, or a recommendation to participate in prediction markets or sports wagering of any kind. Sports betting involves substantial risk of financial loss. All data analysis reflects historical patterns and is not predictive of future outcomes. Please consult applicable laws and regulations in your jurisdiction before participating in any form of wagering.
EdgeLab publishes data-driven sports analytics research. Follow for more.
For the full dataset on soccer match prediction: The 87th-Minute Soccer Edge — $19
Top comments (0)