How to Backtest a Sports Betting Model with Historical Data (Python)

Building a model that predicts match outcomes is the easy part. Knowing whether it would have actually made money — after accounting for the bookmaker's margin, realistic bet sizing, and variance — is the part most tutorials skip. This post walks through a proper backtest: pulling historical data, generating predictions, sizing bets with the Kelly criterion, and simulating a full bankroll to see if an "edge" survives contact with reality.

Why Backtesting Is Not Optional

A model that's 55% accurate sounds good until you check whether that 55% beats the bookmaker's implied probability after their margin. As the penaltyblog backtesting library docs show in their own worked example, a real backtest framework needs three things: a trainer that fits your model on a lookback window, a decision function that only bets when there's positive expected value, and an account object that tracks bankroll bet by bet — not just a final accuracy score.

Step 1: Pull Historical Data

You need real historical fixtures, results, and closing odds — not just today's data. Using Orbistats' Historical Sports Data API:

python
import requests
import pandas as pd

API_KEY = "YOUR_API_KEY"
BASE_URL = "https://api.orbistats.com/v1"

def get_historical_matches(league_id, season):
res = requests.get(
f"{BASE_URL}/football/historical/results",
headers={"Authorization": f"Bearer {API_KEY}"},
params={"league_id": league_id, "season": season},
)
res.raise_for_status()
return pd.DataFrame(res.json()["data"])

matches = get_historical_matches(league_id=39, season=2024)
matches.head()

You want at minimum: date, team_home, team_away, goals_home, goals_away, and — critically — the closing odds (the last price before kickoff), since that's your fairest comparison point.

Step 2: Fit a Model on a Rolling Window

The mistake most backtests make is training on the whole dataset and then testing on the same data — that's not a backtest, that's memorization. Use a rolling/expanding window so the model only ever sees data before the match it's predicting:

python
def rolling_backtest(matches, train_window=200):
results = []

for i in range(train_window, len(matches)):
    train = matches.iloc[i - train_window : i]
    fixture = matches.iloc[i]

    model = fit_model(train)  # your model — Poisson, Dixon-Coles, ELO, whatever
    pred = model.predict(fixture["team_home"], fixture["team_away"])

    results.append({
        "date": fixture["date"],
        "fixture": f"{fixture['team_home']} vs {fixture['team_away']}",
        "pred_home_win": pred.home_win,
        "actual_result": fixture["result"],
        "odds_home": fixture["closing_odds_home"],
    })

return pd.DataFrame(results)

This is the same walk-forward pattern used in penaltyblog's Dixon-Coles example, where a model is refit on each rolling lookback window before predicting the next fixture — no peeking ahead.

Step 3: Only Bet When You Have an Edge

Compare your model's probability to the bookmaker's implied probability. You only place a bet when your estimate is meaningfully higher than what the odds imply:

python
def implied_probability(decimal_odds):
return 1 / decimal_odds

def find_value_bets(results_df, min_edge=0.02):
results_df["implied_prob"] = results_df["odds_home"].apply(implied_probability)
results_df["edge"] = results_df["pred_home_win"] - results_df["implied_prob"]
return results_df[results_df["edge"] > min_edge]

This is the same core logic — model probability minus market-implied probability — that underlies the implied probability and no-vig calculations used across the industry to spot value.

Step 4: Size Bets with the Kelly Criterion

Don't bet a flat amount on every "edge" — size the bet to the strength of your edge. The Kelly formula:

f = (bp - q) / b

where b = decimal odds − 1, p = your estimated win probability, q = 1 − p.

python
def kelly_fraction(prob, decimal_odds, fraction=0.25):
b = decimal_odds - 1
q = 1 - prob
f = (b * prob - q) / b
return max(0, f * fraction) # fractional Kelly — never full Kelly

Note the fraction=0.25 default. As research on Kelly staking in sports betting points out, full Kelly is too aggressive in practice — it assumes you know the true probability, which you never do with a model built on estimation. Most practitioners run quarter or half Kelly specifically to survive the estimation error your model inevitably has. Topendsports' Kelly calculator has a good visual explanation if the formula alone isn't clicking.

Step 5: Simulate the Bankroll

This is the step most people skip, and it's the one that actually tells you whether your model is viable:

python
def simulate_bankroll(value_bets, starting_bankroll=1000):
bankroll = starting_bankroll
equity_curve = [bankroll]

for _, bet in value_bets.iterrows():
    stake_fraction = kelly_fraction(bet["pred_home_win"], bet["odds_home"])
    stake = bankroll * stake_fraction

    if bet["actual_result"] == "H":
        bankroll += stake * (bet["odds_home"] - 1)
    else:
        bankroll -= stake

    equity_curve.append(bankroll)

return equity_curve

Step 6: Don't Trust One Run — Monte Carlo It

A single backtest run shows you one path through variance. As OddsPapi's risk-of-ruin guide puts it: two bettors with the identical edge can end a season one up big and the other flat broke, purely from variance — a point estimate hides the distribution, and that's exactly where ruin lives. Run the same edge through thousands of simulated orderings to see the real spread:

python
import random

def monte_carlo_bankroll(value_bets, runs=1000, starting_bankroll=1000):
final_bankrolls = []

for _ in range(runs):
    shuffled = value_bets.sample(frac=1).reset_index(drop=True)
    curve = simulate_bankroll(shuffled, starting_bankroll)
    final_bankrolls.append(curve[-1])

return pd.Series(final_bankrolls)

outcomes = monte_carlo_bankroll(value_bets)
print(f"Median: {outcomes.median():.2f}")
print(f"5th percentile: {outcomes.quantile(0.05):.2f}")
print(f"% of runs that went broke: {(outcomes <= 0).mean() * 100:.1f}%")

If your 5th-percentile outcome is bankruptcy, your edge isn't real, your staking is too aggressive, or both — and you'd never know that from a single backtest run.

Step 7: Check ROI Against the Vig, Not Zero

Don't just check if you're profitable — check if you're beating the bookmaker's margin. If a market's overround is 5%, being profitable at 2% ROI isn't actually beating a sharp market; it may just mean your sample got lucky. Cross-reference your bet's closing odds against margin/vig calculations to see how your edge compares to the actual cost of betting that book.

Common Mistakes This Catches
Look-ahead bias — training and testing on the same window (fixed by the rolling window in Step 2)
Ignoring the vig — comparing raw win rate to 50% instead of to implied probability (fixed in Step 3)
Full Kelly overbetting — one bad losing streak wipes the bankroll (fixed with fractional Kelly in Step 4)
Single-path illusion — one backtest run looking good by luck (fixed with Monte Carlo in Step 6)
Wrapping Up

A backtest that skips bankroll simulation and variance analysis isn't really a backtest — it's a curve-fit story you're telling yourself. Pull real historical data (the Historical Sports Data API docs are a good reference for the endpoint shape), walk-forward your model instead of training on the whole set, size bets fractionally, and always run the Monte Carlo step before you trust an equity curve.