I built a hockey analytics pipeline that scrapes, models, and publishes itself every week — no server, no database

agesatony — Fri, 04 Sep 2026 10:29:38 +0000

Kenya Hockey Union publishes match results on its website. That's it — no API, no downloadable dataset, no structured feed. If you want to know how the 2026 domestic season is going, you read HTML pages by hand.

I didn't want to do that every week, so I built a pipeline that does it for me: it scrapes the season, runs a full analytics suite over it, builds a dashboard, publishes it, and posts a summary to LinkedIn — completely unattended, once a week, forever. No server, no database, no manual step. Here's how it's put together and a few of the decisions that made it actually trustworthy rather than just automated.

The shape of it

Five Jupyter notebooks, chained together by a GitHub Actions workflow:

01_scraping — pulls match results, events, and player stats across all 8 competitions in the domestic season (two Premier League divisions, two Super League divisions, four National League zones).
02_league_analytics — builds league tables, form guides, scorer and discipline breakdowns.
03_advanced_analytics — 44 analytical modules: Elo ratings, playing-style clustering, giant-killer detection, squad reliance, predictive tables, and the dashboard builder itself.
04_international — Kenya's national-team record pulled from the FIH's own systems: world rankings, ranking history, continental results.
05_worldcup_live — a lightweight standalone refresh used only during the FIH World Cup window, so standings can update every few hours without re-running the whole domestic scrape.

nbconvert --execute --inplace runs each notebook in order inside GitHub Actions, the workflow commits the refreshed data and the rebuilt dashboard straight back into the repo, and then deploys the dashboard to GitHub Pages. A separate step posts a real-data digest to LinkedIn. All of it triggers on a Monday-morning cron, with a manual "run it now" button for whenever I don't want to wait.

Catching data the incremental scraper structurally can't see

The scraper defaults to incremental mode — skip any match URL already saved, only fetch what's new. Efficient, but it has a blind spot: KHU sometimes backfills goal-scorer and card detail onto matches that were already saved with just a final score. Incremental mode will never re-check those, because as far as it's concerned they're already "done."

So there's a second mode — a full re-scrape, every match fetched fresh, ignoring anything already saved — meant to be run occasionally specifically to catch that backfilling. Running it once found that goal-level detail coverage had actually improved from ~19% to 22.2% since the last full pass, concentrated almost entirely in the men's Premier League. That's not a bug fix, it's the pipeline finding real information a purely incremental design would have missed forever.

Clustering playing styles without letting divisions distort each other

One of the 44 modules groups teams into playing-style clusters (high-press, possession-heavy, counter-attacking, etc.) using k-means on standardized efficiency, net-performance, and per-match-points features. The detail that mattered: it's fit separately per competition, not pooled across all eight. Feed clustering un-normalized cross-competition metrics and you get clusters that mostly just reflect "which division is this team in" rather than actual playing style — a distortion that crept back in more than once as new features were added upstream, and had to be caught and fixed explicitly. Below a minimum team count per competition (six), clustering is skipped entirely rather than run on a sample too small to mean anything.

Making the predictions answer for themselves

There's an Elo rating system, and it doesn't just produce a leaderboard — it's checked against a naive baseline using a Brier score (0 = perfect, 0.25 = no better than a coin flip, 1 = perfectly wrong). The report explicitly states whether the model is beating that baseline or not, rather than presenting the numbers with an implied credibility they haven't earned yet. That's the difference between a model that produces numbers and a model that's been checked against outcomes it didn't see coming.

A dashboard with no backend

dashboard.html is a single self-contained file — every chart is a matplotlib figure baked in as a base64 PNG at build time, every table is fully rendered HTML, no client-side data fetching. It's ~3.5MB and it's the literal file GitHub Pages serves. Nine sections (season overview, league tables, coach intelligence, scout intelligence, federation intelligence, predictive ratings, international, an interactive team comparison tool, and a full match log), all built from the same in-memory data in one pass, so nothing can drift out of sync between tabs — because there's only ever one source of truth per run, not nine separately-cached ones.

Not hiding the gaps

Roughly 22% of matches with a recorded goal have full goal-by-goal detail (scorer, minute, cards); the rest only have a final score, because that's genuinely all KHU's own site publishes for them. Rather than quietly limiting scope or overstating what the data supports, the notebook computes that percentage itself, states it in a self-generated data-quality table, and every metric that depends on event-level detail (top scorers, player influence, squad reliance) is explicit that it describes only that subset — not the season as a whole.

Where it stands

186 matches, 63 teams, 8 competitions, one Monday-morning workflow run, zero manual intervention. Live dashboard: https://agesatony.github.io/hockeyiq-kenya/ — repo: https://github.com/agesatony/hockeyiq-kenya

Feedback, issues, and stars all genuinely welcome — this is a side project built to solve my own problem (there was no other way to follow the season with real data), and I'd like to know if it's useful or interesting to anyone else building similar pipelines for data sources that don't want to be scraped.

DEV Community: agesatony

I built a hockey analytics pipeline that scrapes, models, and publishes itself every week — no server, no database