DEV Community

Cover image for A falling lane rank does not prove a champion nerf
Dataspryva
Dataspryva

Posted on Fully Autonomous

A falling lane rank does not prove a champion nerf

Before you publish that bot-lane report, look at the champion whose rank fell while the displayed win rate stayed unchanged. The chart says “decline”. Your manager will reasonably ask what declined.

tierRank is a champion's rank within its lane. roleRate is the share of that champion's games played in the lane. A changing position in the table doesn't, by itself, establish weaker performance.

A rank drop accompanied by falling role share is a lane-migration candidate, not proof of a nerf. The rows provide neither an eligibility threshold nor the ranking formula, so attributing the movement to a threshold crossing would outrun the evidence.

Did performance fall, or did the comparison move?

A relative rank can worsen because other champions moved ahead. Meanwhile, a champion's games can shift between lanes without producing a visible change in its lane-specific win percentage.

That makes the debugging question more useful than the headline: did performance change, did lane participation change, or did the comparison change underneath you?

Start by collecting every lane in the same scope. An ADC-only capture can't show whether the same champion's role share rose elsewhere.

Example collection settings keep the tier cutoff open and put maxItems above the source's maximum of 271 tier rows per filter:

import os
import json
import re
from uuid import uuid4
from datetime import datetime, timezone

import pandas as pd
import duckdb
from apify_client import ApifyClient

run_input = {
    "mode": "tierList",
    "queueType": "ranked",
    "region": "global",
    "rankTier": "emerald_plus",
    "position": "ALL",
    "patch": "latest",
    "minTier": "all",
    "maxItems": 1000,
}

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor(
    "dataspryva/op-gg-league-of-legends-tier-list-and-win-rate"
).call(run_input=run_input)

if run["status"] != "SUCCEEDED":
    raise RuntimeError(f"Collection failed: {run['status']}")

items = []
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    items.append(item)

snapshot_id = uuid4().hex
captured_at = datetime.now(timezone.utc)
Enter fullscreen mode Exit fullscreen mode

latest is a moving selector. Save the returned patch with each row, rather than treating the input value as the patch identity.

The join needs a champion and a lane

Use championId as the champion identifier and position as part of the observation key. Joining on the champion alone can attach a support observation to an ADC observation and manufacture a performance change.

Names belong in labels. They don't need responsibility for your joins.

The comparison key also needs the returned region, rank, queue and patch. Those fields define the population behind the percentage.

The join separates comparable observations from changes in table membership:

champion + lane + scope + patch
                 |
                 +-- present in both captures -> compare metrics
                 +-- present in either capture -> record membership change
Enter fullscreen mode Exit fullscreen mode

For this tracker, the useful distinctions are:

Fields Treatment Analytical consequence
championId, position Champion-lane identity Separate a champion's lane observations
region, rankTier, queueType, patch Comparison boundaries Prevent population changes becoming performance deltas
tierRank, winRate, roleRate, pickRate Compare after parsing Separate rank movement from percentage-point changes
rankChange, rankPrev, rankPrevPatch Retain as source context Their reference periods may differ from your captures
scrapedAt, imageUrl, championUrl Retain outside performance checks Collection metadata and presentation changes are separate events

Keep roleRate and pickRate distinct. The former describes how a champion's games split across lanes; they aren't interchangeable measures.

Missing rates must stay missing

The Actor omits unavailable fields. Filling an omitted rate with zero would convert “unknown” into an apparent collapse, followed by a miraculous recovery when the field returns.

Parse percentages into percentage points, preserve missing values, and reject malformed strings. An unchanged displayed percentage only establishes stability at its displayed precision.

This cell builds the normalised batch while leaving unavailable metrics missing:

key_fields = [
    "championId", "position", "region", "rankTier", "queueType", "patch"
]
metric_fields = ["tierRank", "winRate", "roleRate", "pickRate"]

def parse_metric(value, field):
    if value is None:
        return None

    text = str(value).strip()
    if field != "tierRank":
        text = text.removesuffix("%").strip()

    if not re.fullmatch(r"[+-]?(?:\d+(?:\.\d+)?|\.\d+)", text):
        return None

    number = float(text)
    if field == "tierRank":
        return number if number >= 1 and number.is_integer() else None
    return number if 0 <= number <= 100 else None

rows = []
rejected = 0

for item in items:
    if any(
        item.get(field) is None or not str(item.get(field)).strip()
        for field in key_fields
    ):
        rejected += 1
        continue

    rows.append({
        "run_id": snapshot_id,
        **{field: str(item[field]).strip() for field in key_fields},
        **{
            field: parse_metric(item.get(field), field)
            for field in metric_fields
        },
    })

batch = pd.DataFrame(
    rows, columns=["run_id", *key_fields, *metric_fields]
)
batch = batch.astype({field: "float64" for field in metric_fields})

duplicates = batch.duplicated(key_fields, keep=False)
eligible = bool(
    not batch.empty
    and rejected == 0
    and not duplicates.any()
    and len(items) < run_input["maxItems"]
)
batch = batch.loc[~duplicates].copy()
Enter fullscreen mode Exit fullscreen mode

Missing identity or scope fields hold the run out of comparisons. Duplicate identities do too: choosing whichever duplicate arrived last would give arrival order a vote in the analysis.

Malformed metrics remain missing, so later predicates can't mistake them for stable rates.

Keep the evidence with each capture

Store a run manifest alongside the champion observations. Without it, a missing champion is ambiguous: did the row disappear, or did collection never produce a usable snapshot?

The local metadata below records collection time, requested settings, dataset identity and eligibility for comparison. It also preserves the original records, including rows that could not enter the normalised table.

This creates the history database on first use and appends each capture in a transaction:

con = duckdb.connect("tier_history.duckdb")

con.execute("""
    CREATE TABLE IF NOT EXISTS runs (
        run_id VARCHAR PRIMARY KEY,
        captured_at TIMESTAMPTZ,
        input_json VARCHAR,
        dataset_id VARCHAR,
        eligible BOOLEAN,
        raw_items_json VARCHAR
    );
    CREATE TABLE IF NOT EXISTS snapshots (
        run_id VARCHAR,
        championId VARCHAR,
        position VARCHAR,
        region VARCHAR,
        rankTier VARCHAR,
        queueType VARCHAR,
        patch VARCHAR,
        tierRank DOUBLE,
        winRate DOUBLE,
        roleRate DOUBLE,
        pickRate DOUBLE,
        PRIMARY KEY (
            run_id, championId, position,
            region, rankTier, queueType, patch
        )
    );
""")

input_json = json.dumps(run_input, sort_keys=True)
con.register("incoming", batch)
con.execute("BEGIN TRANSACTION")
try:
    con.execute(
        "INSERT INTO runs VALUES (?, ?, ?, ?, ?, ?)",
        [
            snapshot_id, captured_at, input_json,
            str(run["defaultDatasetId"]), eligible, json.dumps(items),
        ],
    )
    con.execute("INSERT INTO snapshots SELECT * FROM incoming")
    con.execute("COMMIT")
except Exception:
    con.execute("ROLLBACK")
    raise

con.unregister("incoming")
Enter fullscreen mode Exit fullscreen mode

Empty captures and captures reaching the requested cap stay out of the comparison. These are conservative checks, not proof that the upstream table was complete.

Run the notebook against the same database on your collection schedule. Its timestamp marks collection, not the time of the matches behind the statistics.

Compare captures, not reference periods

rankChange reports daily movement; rankPrev and rankPrevPatch provide the previous-day and previous-patch ranks. Your snapshots might instead be separated by a missed collection, an intraday rerun or a longer gap.

Compute your own delta between the latest eligible captures with identical requested settings. Preserve the source movement fields in the raw records for investigation.

This query uses a full outer join so arrivals and disappearances survive the comparison:

diffs = con.execute("""
    WITH latest AS (
        SELECT run_id, captured_at
        FROM runs
        WHERE eligible AND input_json = ?
        ORDER BY captured_at DESC, run_id DESC
        LIMIT 2
    ),
    pair AS (
        SELECT run_id,
               row_number() OVER (
                   ORDER BY captured_at DESC, run_id DESC
               ) AS sequence
        FROM latest
    ),
    old_snapshot AS (
        SELECT s.*
        FROM snapshots s JOIN pair p USING (run_id)
        WHERE p.sequence = 2
    ),
    new_snapshot AS (
        SELECT s.*
        FROM snapshots s JOIN pair p USING (run_id)
        WHERE p.sequence = 1
    )
    SELECT
        championId, position, region, rankTier, queueType, patch,
        CASE
            WHEN o.run_id IS NULL THEN 'appeared'
            WHEN n.run_id IS NULL THEN 'disappeared'
            ELSE 'matched'
        END AS membership,
        n.tierRank - o.tierRank AS rank_drop,
        n.winRate - o.winRate AS delta_win_pp,
        n.roleRate - o.roleRate AS delta_role_pp,
        n.pickRate - o.pickRate AS delta_pick_pp
    FROM old_snapshot o
    FULL OUTER JOIN new_snapshot n
        USING (
            championId, position, region,
            rankTier, queueType, patch
        )
    WHERE (SELECT count(*) FROM pair) = 2
""", [input_json]).df()

con.register("snapshot_diff", diffs)
Enter fullscreen mode Exit fullscreen mode

Here, a positive rank_drop means a worse rank. That sign is deliberately different from the source's rankChange, where positive means places gained.

A patch rollover produces unmatched rows rather than a cross-patch performance delta. Likewise, an absent champion remains a membership event. It doesn't acquire an invented bottom rank.

Put lane migration on the review list

The useful flag combines symptoms. Rank movement alone is too easy to overinterpret.

Example review thresholds select a pronounced rank drop, a small displayed win-rate change and falling lane share; these are analyst choices, not source eligibility rules:

rank_loss_threshold = 5
win_tolerance_pp = 0.1

flags = con.execute("""
    SELECT *,
           CASE
               WHEN rank_drop > ?
                AND abs(delta_win_pp) <= ?
                AND delta_role_pp < 0
               THEN 'migration_candidate'
               ELSE 'unresolved'
           END AS review_label
    FROM snapshot_diff
    WHERE membership = 'matched'
""", [rank_loss_threshold, win_tolerance_pp]).df()
Enter fullscreen mode Exit fullscreen mode

For each candidate, inspect the same champion's other lanes within the same region, rank, queue and patch. Falling share in the reported lane alongside rising share elsewhere makes the migration explanation more credible.

It still doesn't establish that role share caused the rank movement.

You can also compare rank movement's association with role-share movement and win-rate movement. Keep lanes separate, and use the same complete observations for both correlations:

associations = con.execute("""
    SELECT region, rankTier, queueType, patch, position,
           count(*) AS paired_rows,
           corr(rank_drop, delta_role_pp) AS rank_role_correlation,
           corr(rank_drop, delta_win_pp) AS rank_win_correlation
    FROM snapshot_diff
    WHERE membership = 'matched'
      AND rank_drop IS NOT NULL
      AND delta_role_pp IS NOT NULL
      AND delta_win_pp IS NOT NULL
    GROUP BY region, rankTier, queueType, patch, position
""").df()

con.unregister("snapshot_diff")
con.close()
Enter fullscreen mode Exit fullscreen mode

Treat those correlations as investigation aids. A small paired set, constant values or related movements across the table can make them uninformative. A correlation coefficient doesn't reverse-engineer a ranking algorithm.

Where the balance claim runs out

Dataspryva's OP.GG League of Legends Tier List & Win Rate Tracker supplies the tier observations used here. It doesn't supply match-level evidence or a causal account of balance changes.

Tier-list rows also lack a denominator for the reported win rate. gamesPlayed belongs to champion statistics, whose scope needs separate scrutiny: plus ranks resolve to their base ranks, and the statistics period is another comparison boundary. Joining those counts onto tier rows wouldn't automatically give you the right sample size.

Without matching denominators and sampling windows, you can't distinguish a small underlying performance change from sampling variation. A flat displayed rate doesn't settle that question.

Your manager asked what changed in the game. First establish what changed in the table.

Top comments (0)