NOT BETTING ADVICE
This is a paper-only forecasting analysis. No wagers were placed, no stake sizing is discussed, and this story contains no betting tips.
Reveal Wire EP17 tested six AI personas against a bookmaker using English Premier League matches from January 1 to May 24, 2026. In total, the test covered 60 matches and 348 forecasts.
Watch the episode here: https://youtu.be/vWjgbjsyWsg
The result
The bookmaker's closing odds won on Brier score, with a Brier score of 0.2129 and accuracy of 43.3%.
The most accurate persona, Hot-Take Pundit, had 45.8% accuracy — higher than the bookmaker — but finished 5th on Brier score, at 0.2341.
| Forecaster | Brier score | Accuracy | Brier ranking |
|---|---|---|---|
| Bookmaker closing odds | 0.2129 | 43.3% | 1st |
| Hot-Take Pundit | 0.2341 | 45.8% | 5th |
The average vig was 5.58%.
Accuracy is not calibration
On Brier score, lower is better. Accuracy counts how often the top pick was right. Calibration asks how well the stated probabilities matched what actually happened. That distinction is the story of this test: the most accurate persona did not have the best-calibrated probabilities, so it finished 5th on Brier despite leading on accuracy.
Why this matters for evaluating AI forecasts
A leaderboard that only reports accuracy can reward confident top picks while hiding poorly calibrated probabilities. Scoring forecasts with a proper scoring rule such as the Brier score gives a fuller picture: it penalizes probabilities that are far from the outcome, not just the wrong winner.
The headline result is therefore twofold. The bookmaker's closing odds were the best-calibrated forecasts in this test, and the highest-accuracy AI persona still lost on calibration.
NOT BETTING ADVICE — paper-only analysis only.
Top comments (0)