How one scheduled Python job and a single Cloudflare Worker serve football stats for hundreds of matches a week

Xpect — Tue, 10 Nov 2026 11:32:36 +0000

I'm a solo developer. This year I shipped Xpect, a football statistics app: for each upcoming professional match it shows probabilities for the usual markets (1X2, over 2.5 goals, both teams to score, corners, cards, offsides) next to the numbers they come from.

This post is about the plumbing: a SQLite file, a Python script on a schedule, one Cloudflare Worker, KV and D1, and the parts that went wrong.

The shape of the system

History lives in one SQLite database: about 1.4 million football matches with scores and, where the source has them, corners, cards and offsides.
The pipeline is a folder of plain Python scripts. No framework, mostly the standard library.
The API is one Worker with KV for precomputed JSON, two D1 databases, a rate limiter and an hourly cron trigger. Live scores come from a third-party API and are cached per edge node for 15 seconds.
The client is an Expo app (SDK 57, React Native 0.86, expo-router, TypeScript). The same code is exported with react-native-web and served by the same Worker as static assets.

The "nightly" job that runs in the morning

The orchestrator is called nightly.py, although it actually starts at 08:00 from a plain OS scheduler. It is about 150 lines and does this, in order:

Waits until the separate results import has finished for the day. It looks for a summary file that the importer writes as its last action.
Builds probabilities for the next seven days and writes upcoming.json. It also stores a snapshot of every number in a local database, so accuracy can be checked later against what was really shown before kick-off.
Refreshes extras (logos, match details). If these fail, the probabilities are still uploaded.
On Mondays, re-runs validation.
Uploads the JSON files to KV, takes a backup and runs a data health check.

One lesson from this file: wait for a marker, not for a process. The first version checked whether the importer process was still running. One morning the importer had not started yet, the check passed, and the whole day was computed on yesterday's data. Now the job waits for the summary file, for up to four hours, and if it gives up it says so loudly in the log.

The model is deliberately boring

For goals: each team gets an attack and a defence strength from its last 30 matches, measured against the league average and shrunk halfway back toward that average. Those feed two Poisson distributions, one per team, and every goal market is read off the resulting score grid. That window model is the baseline. On top of it sits an incremental rating: every team has attack and defence ratings that move after each match, and every league has its own strength term, so a European cup game between clubs from two countries shifts both leagues, and a newly promoted side does not carry its lower-division numbers upward unchanged. Corners, cards and offsides follow the same scheme with their own shrink factors.

The interesting part is validation, not the model. Everything is walk-forward: a match is only ever scored by a model that has seen earlier matches. On 445,280 matches since July 2023 the baseline's Brier score for over 2.5 goals was 0.2359, against 0.2381 for the naive "average match of this league" baseline. For the match result it was 0.612 against 0.642.

That is a small edge, and I would rather show it than hide it. Calibration is what matters for a stats product: when the model said 60–70% for over 2.5 goals, it happened 63.3% of the time. The weekly validation output feeds an "Accuracy" screen inside the app.

Serving: one JSON blob and a very defensive read

This week upcoming.json holds 406 matches from 46 competitions and weighs about 2 MB. The Worker reads it from KV at most once every five minutes per isolate and keeps the parsed object in module scope.

The read path got more defensive after a real incident: a KV read failed and users saw "could not load". Now:

if KV returns nothing or unparsable JSON, the Worker serves the last good copy;
that copy lives both in isolate memory and in the Cache API, so a fresh isolate on the same edge node can still find it;
KV is retried a minute later instead of on every request.

The loader never throws. The endpoint returns 503 only if no copy has ever existed.

Lock on the server, not in the client

Without a subscription the app shows a limited set of numbers. The important detail is that locked values are stripped from the response on the server. They are not sent at all, rather than sent and hidden. Anything that reaches the client can be read out of the client.

Entitlements are verified server-side as well. The app sends the purchase token, the Worker checks it with the Google Play Developer API (the App Store Server API path is written for iOS) and stores "open until" per installation in KV. Renewals are re-checked with the store at most once an hour. Because responses differ per user, they go out with cache-control: private.

The step that pays for itself

The last step of the job is a data health check: are all finished fixtures present in the database, do recent matches have corners and cards where that league normally has them, does every team in next week's list have enough history, are there team names in the schedule that the database does not know. It writes OK / WARN / FAIL per section.

Most of what it catches is naming. The same club arrives as "Club Brugge" from one source and "Club Brugge KV" from another, and without a mapping its domestic and European matches never meet. I should have written this step first.

Where it is now

The app is on Google Play with the interface translated into about two dozen languages. The subscription costs $0.99 for the first week, then $19.99 a month. If you want to see what the pipeline produces, the public match pages are at xpect.app.

Questions about the architecture are welcome in the comments.

DEV Community: Xpect