We publicly grade every AI football prediction — here's the pipeline

Most football prediction sites show you their winners after the fact. If a model goes 4/10 on a weekend, those four hits end up in a screenshot and the six misses vanish.

We built Footalytics the other way around: every official prediction is published before kickoff, and graded automatically after the match — hit or miss, plus a Brier score per row. The full record is a free, no-signup CSV.

This post is the engineering write-up: what the pipeline looks like and what the first 127 graded matches say.

The pipeline

1. Data in. Fixtures, recent results, league tables and head-to-head come from official sports data APIs.
2. Model. An LLM pipeline running on Cloudflare Workers (behind the AI Gateway, with a provider fallback chain) turns the structured match data into full 1X2 probabilities for every game in the Big 5 leagues + the Champions League.
3. Publish before kickoff. The three probabilities and the resulting pick go live on the match page before the game starts. Nothing is retro-edited.
4. Mechanical settlement. When the official final score is in, a job settles the row: hit_1x2 compares the pick to the outcome, and a multiclass Brier score is computed over the three probabilities. No manual overrides, no quietly retiring bad picks.
5. Public dataset. Every settled row lands in a free CSV, mirrored to GitHub by a scheduled Action.

The dataset

One row per graded prediction:

column	meaning
kickoff_utc	scheduled kickoff (ISO 8601, UTC)
league	eng.1, esp.1, ita.1, ger.1, fra.1, uefa.champions
home, away	teams
p_home, p_draw, p_away	model probabilities
pick	official pick (home/draw/away)
home_goals, away_goals	official final score
hit_1x2	did the pick land
brier	multiclass sum over the three outcomes

The repo (github.com/footalytics/footalytics-data) re-fetches the CSV daily via a GitHub Action and commits when it changes, CC BY 4.0.

What 127 matches say

• 1X2 hit rate: 57.5% — vs 47.2% for "always pick the home team" and 33.3% for random.
• Mean Brier (3-class sum): 0.581 (uniform guessing would be ≈0.667).
• The model never picks draws. 0 of 127 picks. It treats a draw as never the best expected outcome, and the grades show the cost — that's a known, published weakness, not a surprise.
• 127 matches is statistically nothing. This is the start of a public record, not a conclusion.

What's next

Calibration curves and per-league breakdowns as the sample grows, grading more markets than 1X2, and a longer discussion of the draw problem once there's enough data to measure it properly.

If you poke at the data and find an error in the grading, open an issue on the repo — grading is mechanical, but we want errors on the record.

Links: live grading board (footalytics.farvor.com/accuracy) · free CSV · GitHub repo