Why one-off latency tests mislead
Run a speed test once and you learn how one API behaved for 20 seconds from your laptop. That is not reliability. Reliability is what happens over days: how often a call succeeds, how often it times out, what happens when you hit a rate limit, and whether the slow tail (the p95 and p99) stays acceptable.
So instead of a single run, we build a small monitor that probes each provider on a schedule, stores every result, and produces a report. By the end you will have two scripts and a SQLite file you can chart, share or attach to a vendor negotiation.
Read this first: I work with Orbistats, which is one of the ten providers. I have not run this monitor over time, so there are no results in this post. Vendor facts come from public pages as of 12 October 2026, and you should verify them. Publish your own numbers, with your region and dates, once you have them.
The ten providers and what they let you test
Not every vendor offers free access, so plan which ones you can actually probe:
Sportradar: enterprise, quote-based, a trial is advertised for many APIs.
Stats Perform (Opta): premium analytics, tailored licensing, sales contact.
Genius Sports: official data and integrity, custom quote.
Sportmonks: self-serve football, entry plan at €29 per month.
SportsDataIO: US leagues and fantasy, product-specific pricing.
Orbistats: 13 sports, Free pass of 1 hour after request and approval.
OddsPapi: odds-focused, details unverified.
SportsGameOdds: free tier, billed by returned objects.
API-Football: free plan with 100 requests per day.
The Odds API: free tier with a quota by sport, market and region.
A fair note on the free passes: Orbistats’ Free pass lasts 1 hour from the moment you open your activation link, so a multi-day monitor needs a longer pass. Its pricing is pass-based: Starter is €19 for 24 hours, Growth €79 for 5 days, and Unlimited €1,999 for 30 days. For API-Football, 100 requests per day limits how often you can probe, which we handle below.
What we will measure
For every probe we record four things: the HTTP status, the response time, whether the body was valid JSON, and an outcome label. The outcome label is the important part. A timeout, a 429 rate limit and a 503 are three different problems, and lumping them together as “failed” hides what is really going on.
ok: a 2xx response with valid JSON.
rate_limited: HTTP 429. This is your quota, not the vendor’s outage.
client_error: other 4xx, usually a bad key or wrong endpoint.
server_error: 5xx, the vendor’s problem.
timeout: no answer within the limit.
connection_error: DNS, TLS or network failure.
bad_json: a 2xx whose body is not valid JSON.
Step 1: Setup
bash
mkdir sports-reliability && cd sports-reliability
python -m venv .venv && source .venv/bin/activate
pip install requests
Set your keys as environment variables so they never appear in code:
bash
export ORBISTATS_KEY="your_key_here"
export APIFOOTBALL_KEY="your_key_here"
export ODDSAPI_KEY="your_key_here"
Step 2: Describe your targets
Treat each vendor as data. Orbistats’ base URL and bearer authentication come from its documentation and quickstart. For every other vendor, confirm the endpoint and auth header in its own docs. The two extra entries below are examples to verify, not guaranteed values.
Create targets.py:
python
import os
TARGETS = {
"orbistats": {
"url": "https://api.orbistats.com/v1/football/matches/live",
"headers": {"Authorization": "Bearer " + os.getenv("ORBISTATS_KEY", "")},
"params": {},
"interval": 60,
},
"api_football": {
"url": "https://v3.football.api-sports.io/fixtures",
"headers": {"x-apisports-key": os.getenv("APIFOOTBALL_KEY", "")},
"params": {"live": "all"},
"interval": 900,
},
"the_odds_api": {
"url": "https://api.the-odds-api.com/v4/sports",
"headers": {},
"params": {"apiKey": os.getenv("ODDSAPI_KEY", "")},
"interval": 900,
},
}
Each target has its own interval in seconds. That is how you respect a 100-requests-per-day free plan: at one probe every 900 seconds you use 96 requests a day.
Step 3: The probe and the database
Create monitor.py:
python
import sqlite3
import time
import requests
from targets import TARGETS
DB_FILE = "reliability.db"
TIMEOUT = 10
def init_db():
conn = sqlite3.connect(DB_FILE)
conn.execute(
"CREATE TABLE IF NOT EXISTS probes ("
"ts INTEGER, provider TEXT, status INTEGER, "
"latency_ms INTEGER, outcome TEXT, size_bytes INTEGER)"
)
conn.commit()
return conn
def probe(cfg):
start = time.perf_counter_ns()
try:
r = requests.get(
cfg["url"],
headers=cfg["headers"],
params=cfg["params"],
timeout=TIMEOUT,
)
except requests.Timeout:
return None, TIMEOUT * 1000, "timeout", 0
except requests.RequestException:
return None, (time.perf_counter_ns() - start) // 1_000_000, "connection_error", 0
latency = (time.perf_counter_ns() - start) // 1_000_000
size = len(r.content)
if r.status_code == 429:
return r.status_code, latency, "rate_limited", size
if 400 <= r.status_code < 500:
return r.status_code, latency, "client_error", size
if r.status_code >= 500:
return r.status_code, latency, "server_error", size
try:
r.json()
except ValueError:
return r.status_code, latency, "bad_json", size
return r.status_code, latency, "ok", size
def record(conn, provider, result):
status, latency, outcome, size = result
conn.execute(
"INSERT INTO probes VALUES (?, ?, ?, ?, ?, ?)",
(int(time.time()), provider, status, latency, outcome, size),
)
conn.commit()
def run_forever():
conn = init_db()
next_due = {name: 0 for name in TARGETS}
while True:
now = time.time()
for name, cfg in TARGETS.items():
if now >= next_due[name]:
record(conn, name, probe(cfg))
next_due[name] = now + cfg["interval"]
print("probed", name)
time.sleep(1)
if name == "main":
run_forever()
Run it with: python monitor.py
Leave it running for hours or days, ideally on a small always-on machine or server in the region where your real users are. A laptop that sleeps will give you gaps that look like outages.
Step 4: The report
Create report.py:
python
import sqlite3
import statistics
conn = sqlite3.connect("reliability.db")
def percentile(values, p):
values = sorted(values)
if not values:
return None
index = round((p / 100) * (len(values) - 1))
return values[index]
providers = [row[0] for row in conn.execute("SELECT DISTINCT provider FROM probes")]
for provider in providers:
rows = conn.execute(
"SELECT latency_ms, outcome FROM probes WHERE provider = ?", (provider,)
).fetchall()
total = len(rows)
ok_latencies = [lat for lat, outcome in rows if outcome == "ok"]
counts = {}
for _, outcome in rows:
counts[outcome] = counts.get(outcome, 0) + 1
counted = total - counts.get("rate_limited", 0)
availability = (len(ok_latencies) / counted * 100) if counted else 0
print(provider)
print(" probes:", total)
print(" availability (excluding 429s):", round(availability, 2), "percent")
print(" median ms:", round(statistics.median(ok_latencies)) if ok_latencies else None)
print(" p95 ms:", percentile(ok_latencies, 95))
print(" p99 ms:", percentile(ok_latencies, 99))
print(" outcomes:", counts)
print()
Run it with: python report.py
Two choices here are worth explaining. Latency percentiles use only successful calls, because a timeout recorded as 10,000 ms would distort the median in a misleading way. And availability excludes 429s, because hitting your own quota is not the vendor being down. Report both numbers if you publish results, so readers can see how you counted.
Step 5: Measure live streaming separately
Request and response APIs are only half the picture. For live scores and odds, the question is whether a stream stays connected. Orbistats offers a WebSocket API, where the company cites sub-50ms, and webhooks. Take the exact connection URL and subscribe message from the documentation and never guess them. This script counts how long a connection survives and how many messages arrive:
python
import json
import time
import websocket
WS_URL = "PASTE_WS_URL_FROM_DOCS"
SUBSCRIBE = {"action": "subscribe", "sport": "football"}
connected_at = time.time()
messages = 0
try:
ws = websocket.create_connection(WS_URL, timeout=60)
ws.send(json.dumps(SUBSCRIBE))
while True:
ws.recv()
messages += 1
except Exception as e:
uptime = round(time.time() - connected_at)
print("disconnected after", uptime, "seconds,", messages, "messages, reason:", repr(e))
Install the extra dependency with: pip install websocket-client. The SUBSCRIBE message above is illustrative. Log how often the connection drops, and how long reconnecting takes, across a full match window.
Step 6: Read the results honestly
Numbers are easy to misuse. Before you draw conclusions, check these:
Sample size. A few dozen probes tell you almost nothing about p99. Collect hundreds, ideally thousands.
Time of day. Probe during live matches and quiet hours, since load differs.
Region. Latency from India is not latency from Frankfurt. Always state where the monitor ran.
Compare like with like. A live-scores endpoint and a static standings endpoint are not the same workload.
Caching. A response served from a vendor cache can look unrealistically fast.
Plan differences. A free tier can be throttled differently from a paid one.
Empty live endpoints. A live endpoint may legitimately return an empty list when no match is on, so label those periods.
For vendors that publish a status page, compare your own results with it. Orbistats lists a public status page, and checking it against your own logs is a useful sanity test of both.
A results template to publish
When you have real data, present it like this, one block per provider, with your own measurements:
Provider: name
Plan tested: free, trial or paid
Region and dates: where the monitor ran and for how long
Probes: total number
Availability: percent, excluding 429s
Median and p95 latency: in milliseconds
Dominant failure: timeout, 5xx or rate limit
Keep the method next to the numbers: interval, timeout, endpoint, and whether a match was live.
Which provider should you probe first?
Enterprise vendors (Sportradar, Stats Perform, Genius Sports): ask sales for a trial key and written SLA terms, then probe the same way.
Football products: Sportmonks and API-Football, probed on the leagues you will actually show.
Odds apps: SportsGameOdds, OddsPapi and The Odds API, probed on the markets and bookmakers you need.
Multi-sport apps with odds and streaming: Orbistats covers 13 sports (football, basketball, American football, cricket, tennis, baseball, esports, combat sports, volleyball, handball, ice hockey, golf and horse racing), so probe your own sports, for example cricket or basketball, in the sandbox before committing.
Backtesting and machine learning: probe the history endpoint separately, for example the Historical Data API, and verify retention per sport. The company says some seasons go back to 2010.
Common mistakes
Counting 429s as vendor downtime, which blames them for your own quota.
Probing too often on a request-metered plan and burning the free tier in an afternoon.
Running the monitor on a laptop that sleeps.
Measuring from one region and generalising to everywhere.
Publishing a table with no method, so nobody can trust or reproduce it.
Next steps
Chart the SQLite data with a notebook, add an alert when availability drops below a threshold, and run the same monitor against your own production traffic so you compare vendor claims with what your users actually experience. If you want an access pass to add a multi-sport feed to the test, follow the quickstart and request access here. The glossary is useful if the odds terms are new, and the API reference lists the exact response shapes.
Top comments (0)
Some comments may only be visible to logged-in visitors. Sign in to view all comments.