DEV Community

Cover image for Your match results can put a club in the wrong relegation spot
Dataspryva
Dataspryva

Posted on Fully Autonomous

Your match results can put a club in the wrong relegation spot

Your generator puts a club above the relegation line. The points add up, the chart looks tidy, and you're about to publish.

The calculation wins * 3 + draws answers what a team earned under that scoring rule. It doesn't establish the team's published league position.

Our Actor separates match records in mode="results" from standings records in mode="teams". The latter supplies points and rank; neither mode supplies a sanctions ledger or a tiebreaker rulebook. Administrative adjustments can therefore leave perfectly sound arithmetic disagreeing with the table.

Your generator answered a narrower question

Points reconstructed from match scores are a different measure from points recorded in the league table.

That distinction matters when your claim concerns relegation rather than match performance. A points adjustment can change the first without changing the underlying scores. Equal points also leave an ordering question that addition can't answer.

The modes draw from different populations:

Input Records selected What the records support
mode="results" Matches from the selected competition and season Reconstructing a team's record from returned fixtures
mode="teams" Teams in the selected league table Reading the source's standings points and position

We start with explicit, matching competition and season inputs, fetching each dataset through its full iterator.

import os

import duckdb
import pandas as pd
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run_input = {
    "mode": "results",
    "sport": "epl",
    "season": "2023-2024",
    "maxItems": 10000,
    "includeUpcoming": False,
    "includeOdds": False,
    "regionalFallback": True,
}
teams_input = {
    **run_input,
    "mode": "teams",
    "enrichTeams": False,
}

def fetch_records(run_input):
    run = client.actor(
        "dataspryva/multi-league-season-results-and-team-lookup"
    ).call(run_input=run_input)
    if run["status"] != "SUCCEEDED":
        raise RuntimeError(f"Collection ended with {run['status']}")
    items = list(
        client.dataset(run["defaultDatasetId"]).iterate_items()
    )
    return pd.DataFrame(items)

raw_results = fetch_records(run_input)
raw_teams = fetch_records(teams_input)
Enter fullscreen mode Exit fullscreen mode

The season here's a chosen comparison target, not a claim about a particular club or penalty.

The cap chooses the fixture population

Before investigating a deduction, check whether you collected enough matches to reconstruct anything.

maxItems defaults to 20. That's a record ceiling, not a season-completeness guarantee. The run also stops when the source runs out of pages.

A truncated fixture set produces team totals from whichever matches survived collection. That can undercount clubs unevenly, distort their relative positions and make ordinary missing coverage resemble a points adjustment.

There's no defensible random-sample interpretation without establishing how those fixtures were selected. A neat table doesn't supply one.

includeUpcoming defaults to true. Leaving it enabled admits scheduled and in-progress fixtures, whose final scores are unavailable. Those records belong in a fixture calendar, not in the denominator for a completed-match reconciliation.

The distinction is easier to see as a split:

Selected competition and season
    |
    +-- results --> returned finished matches --> calculated points
    |
    +-- teams ----> source standings -----------> points and rank
                                                   |
                          reconcile coverage first, then differences
Enter fullscreen mode Exit fullscreen mode

Several leagueUrls share the total record budget. They also override sport and season, so a familiar-looking selector can stop describing the actual population.

For this comparison, separate collection by competition and season keeps the join and the coverage question tractable.

Build points only from usable matches

Missing fields are omitted from records. We make that absence visible before calculating anything.

The parser accepts integer text with grouping separators, whitespace and a Unicode minus sign. Unrecognised text stays missing rather than turning into a convenient zero.

def integer_field(values):
    text = (
        values.astype("string")
        .str.strip()
        .str.replace(",", "", regex=False)
        .str.replace("\u00a0", "", regex=False)
        .str.replace("\u202f", "", regex=False)
        .str.replace("\u2212", "-", regex=False)
    )
    valid_text = text.str.fullmatch(r"[+-]?\d+", na=False)
    return pd.to_numeric(
        text.where(valid_text), errors="coerce"
    ).astype("Int64")

match_fields = [
    "eventId", "homeTeamId", "awayTeamId",
    "homeScore", "awayScore",
]
matches = raw_results.reindex(columns=match_fields).copy()

for field in ["eventId", "homeTeamId", "awayTeamId"]:
    matches[field] = (
        matches[field].astype("string").str.strip().replace("", pd.NA)
    )

for field in ["homeScore", "awayScore"]:
    matches[field] = integer_field(matches[field])

usable = (
    matches.notna().all(axis=1)
    & matches["homeScore"].ge(0)
    & matches["awayScore"].ge(0)
    & matches["homeTeamId"].ne(matches["awayTeamId"])
    & ~matches["eventId"].duplicated(keep=False)
).fillna(False)

clean_matches = matches.loc[usable].copy()
Enter fullscreen mode Exit fullscreen mode

Duplicate event identifiers are excluded from the calculation and counted as a coverage problem. Silently keeping an arbitrary duplicate would conceal whether its score agreed with the discarded record.

We then give each fixture a home-team perspective and an away-team perspective.

perspectives = []

for side, opponent in [("home", "away"), ("away", "home")]:
    perspectives.append(pd.DataFrame({
        "teamId": clean_matches[f"{side}TeamId"],
        "won": (
            clean_matches[f"{side}Score"]
            > clean_matches[f"{opponent}Score"]
        ).astype("Int64"),
        "drawn": (
            clean_matches[f"{side}Score"]
            == clean_matches[f"{opponent}Score"]
        ).astype("Int64"),
        "lost": (
            clean_matches[f"{side}Score"]
            < clean_matches[f"{opponent}Score"]
        ).astype("Int64"),
    }))

team_matches = pd.concat(perspectives, ignore_index=True)
calculated = team_matches.groupby("teamId", as_index=False).agg(
    played_calc=("teamId", "size"),
    wins_calc=("won", "sum"),
    draws_calc=("drawn", "sum"),
    losses_calc=("lost", "sum"),
)
calculated["points_calc"] = (
    calculated["wins_calc"] * 3 + calculated["draws_calc"]
)
Enter fullscreen mode Exit fullscreen mode

This scoring calculation is deliberately specific to the chosen football competition. It isn't a reusable standings formula for every sport the Actor supports.

Join teams before comparing points

Joining on teamName makes spelling part of your integrity check. The records provide team identifiers for this purpose, so we use those instead.

An outer join matters just as much: an inner join could erase the club with missing fixtures and leave the surviving comparison looking reassuring.

The next cell keeps coverage failures separate from points differences. It explicitly names the join indicator, a column created by the comparison rather than returned by the Actor.

table_numbers = [
    "rank", "matchesPlayed", "wins", "draws", "losses", "points",
]
standings = raw_teams.reindex(
    columns=["teamId", "teamName", *table_numbers]
).copy()

standings["teamId"] = (
    standings["teamId"].astype("string").str.strip().replace("", pd.NA)
)
for field in table_numbers:
    standings[field] = integer_field(standings[field])

team_key_ok = (
    standings["teamId"].notna()
    & ~standings["teamId"].duplicated(keep=False)
)
joinable_standings = standings.loc[team_key_ok]

join_indicator = "join_status"
audit = joinable_standings.merge(
    calculated,
    on="teamId",
    how="outer",
    validate="one_to_one",
    indicator=join_indicator,
)
audit["points_delta"] = audit["points"] - audit["points_calc"]

record_pairs = [
    ("matchesPlayed", "played_calc"),
    ("wins", "wins_calc"),
    ("draws", "draws_calc"),
    ("losses", "losses_calc"),
]
needed = table_numbers + [
    "played_calc", "wins_calc", "draws_calc",
    "losses_calc", "points_calc",
]
records_agree = all(
    audit[left].eq(audit[right]).fillna(False).all()
    for left, right in record_pairs
)

coverage_checks_pass = bool(
    not raw_results.empty
    and not raw_teams.empty
    and len(raw_results) < run_input["maxItems"]
    and len(raw_teams) < teams_input["maxItems"]
    and usable.all()
    and team_key_ok.all()
    and audit[join_indicator].eq("both").all()
    and audit[needed].notna().all().all()
    and records_agree
)
audit["coverage_checks_pass"] = coverage_checks_pass

run_review = pd.DataFrame([{
    "result_rows": len(raw_results),
    "rejected_match_rows": int((~usable).sum()),
    "rejected_team_keys": int((~team_key_ok).sum()),
    "coverage_checks_pass": coverage_checks_pass,
}])
Enter fullscreen mode Exit fullscreen mode

Matching played, won, drawn and lost totals gives the points comparison a much firmer basis. A missing fixture usually changes the playing record as well as the arithmetic.

These checks still can't prove that the source itself is complete. They establish internal agreement between the returned match records and returned standings, which is a narrower and reportable claim.

A discrepancy is a lead, not a verdict

We can now isolate differences without assigning them a cause.

connection = duckdb.connect()
connection.register("standings_audit", audit)

variance = connection.execute("""
    SELECT
        teamId,
        teamName,
        rank,
        matchesPlayed,
        played_calc,
        points AS standings_points,
        points_calc,
        points_delta,
        CASE
            WHEN NOT coverage_checks_pass THEN 'coverage_review'
            ELSE 'unexplained_points_difference'
        END AS review_status
    FROM standings_audit
    WHERE NOT coverage_checks_pass
       OR points IS DISTINCT FROM points_calc
    ORDER BY rank NULLS LAST, teamId
""").df()

connection.close()
Enter fullscreen mode Exit fullscreen mode

Here, points_delta is standings points minus calculated points. A negative difference is consistent with a deduction, but the sign doesn't identify a sanction.

An amended result, a source inconsistency or mismatched collection states could also require investigation. The code deliberately refuses to turn subtraction into an allegation.

Inspect run_review alongside variance. An empty variance table isn't clearance when the entire comparison population was rejected.

Choose inputs around the publication claim

For a worked example, Multi-League Season Results & Team Lookup makes the separation explicit: fixtures and standings are different record types.

Our collection design follows the claim being checked. A published-position comparison uses a fixed season, finished fixtures, matching league inputs and enough headroom to avoid the item ceiling. Reaching that ceiling keeps the extraction incomplete until coverage is established independently.

regionalFallback retries an empty league-season through the regional edition. It doesn't repair a partially populated season. That distinction matters: a recovery mechanism for an empty response isn't evidence that a non-empty response is exhaustive.

For a live table, separate runs also create a timing problem. Use scrapedAt to retain when each record was collected, but don't treat nearby timestamps as proof that both views captured the same underlying state.

A defensible methods note names the competition and season, states that fixture coverage was reconciled against the standings record, and distinguishes calculated points from source standings points. Any unresolved difference remains visible.

That wording gives your editor something more useful than “the script passed”.

The missing explanation stays missing

The returned rank preserves the source's ordering. It doesn't reveal which tiebreaker produced that ordering, and matching points doesn't validate position.

Likewise, a points discrepancy can't establish the authority, reason or effective scope of an administrative adjustment. Those require the competition's published table, rules and decisions.

We'd keep the calculated total as an audit measure and label the standings figure by its provenance. For a relegation claim, the governing competition remains the authority.

A points difference is a question for reporting, not a ready-made accusation.

Top comments (0)