DEV Community

Cover image for How to map Steam games in Python without assuming mutual links
Dataspryva
Dataspryva

Posted on Fully Autonomous

How to map Steam games in Python without assuming mutual links

Your competitor map is ready for Friday, until someone asks whether a line means both games recommend each other.

The similarGames name makes that interpretation tempting. Dataspryva's Steam Actor collects recommendations from each source app's More like this carousel, so the link starts on a particular app's page.

A recommendation from one game to another doesn't establish a mutual competitor relationship.

A carousel link has a direction

Keep the page that supplied the recommendation attached to the target:

source app ──recommends──> target app

source app ──recommends──> target app
source app <──recommends── target app
                          separate observation
Enter fullscreen mode Exit fullscreen mode

An undirected line discards that distinction. It makes an observed recommendation look like a shared relationship, then carries that assumption into degree counts, clusters and connectivity.

The chart gets cleaner. The claim gets harder to defend.

We can still build a useful map: which collected apps recommend which targets, which targets receive links, and where a return link is also observed.

Which games can enter the graph?

Your source selection defines where outgoing recommendations can be observed. A target appearing in a carousel doesn't establish that its own page was collected.

These are the boundaries that matter for this analysis:

Boundary Collection behaviour Consequence for the graph
Source selection URL and App ID modes select exact apps. Search collects top matches and removes duplicate App IDs. Browse follows the selected chart. The source set reflects your selection method, rather than the whole catalogue.
Categories appType keeps a selected app type in the collected results. A source filter doesn't establish the type of every recommendation target.
Caps maxItems permits 1–1000 source records. Search also limits candidates through maxResultsPerQuery, which permits 1–100. These limits constrain source selection; neither specifies a carousel length.
Geography country selects the storefront for prices, availability and ratings. Record it as collection context. It doesn't establish where recommendation recipients live.
Time Records can carry scrapedAt; there is no historical recommendation-window input. Treat the export as observations collected over an interval, rather than recommendation history.
Missing fields Fields Steam doesn't publish for an app are omitted. An absent similarGames field must remain distinct from a present, empty list.

Example input: this fixes the source apps and storefront, and preserves the effective settings with the export.

import os
import re

import pandas as pd
import duckdb
from apify_client import ApifyClient

run_input = {
    "mode": "appIds",
    "appIds": ["570", "620"],
    "appType": "game",
    "maxItems": 2,
    "country": "gb",
    "language": "english",
    "includeSimilarGames": True,
}

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor(
    "dataspryva/steam-game-specs-pricing-and-media-extractor"
).call(run_input=run_input)

manifest = {"input": run_input.copy(), "status": run["status"]}
if run["status"] != "SUCCEEDED":
    raise RuntimeError(f"Collection ended with status {run['status']}")

items = []
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    items.append(item)
Enter fullscreen mode Exit fullscreen mode

Iterating the dataset avoids quietly treating a dataset API page as the entire export. A successful run still doesn't prove that every intended source app returned a usable recommendation field.

The switch changes the collection work

includeSimilarGames has a schema default of true, while its form prefill is false. Save the effective input rather than reconstructing it from whichever default you remember.

Similar-game titles require an extra small lookup per recommendation. When you only need App IDs, switching the option off avoids that title-enrichment work.

That lookup describes collection work. It doesn't establish a separate published charge event for each recommendation.

Our example explicitly enables the option. The analysis still checks what actually arrived.

Inspect the payload before naming its edges

We'd rather leave an edge unresolved than turn an arbitrary number into an App ID.

Start by inspecting the actual similarGames values. This cell distinguishes omission from a present value and gives you a bounded preview without guessing nested property names.

inventory = pd.DataFrame([
    {
        "appId": record.get("appId"),
        "field_present": "similarGames" in record,
        "value_type": type(record.get("similarGames")).__name__,
        "preview": repr(record.get("similarGames"))[:240],
    }
    for record in items
])
inventory.head()
Enter fullscreen mode Exit fullscreen mode

The decoder below deliberately accepts a narrow, inspectable case: a list whose entries each contain a Steam app URL identifying a single target.

It walks values without assuming nested keys. Bare numeric values, ambiguous entries and other container shapes remain unresolved. A list is this decoder's requirement, not a claim about every export's shape.

def canonical_app_id(value):
    if isinstance(value, bool) or not isinstance(value, (str, int)):
        return None
    text = str(value).strip()
    return str(int(text)) if re.fullmatch(r"[0-9]+", text) else None

def linked_app_ids(value):
    if isinstance(value, str):
        match = re.fullmatch(
            r"https?://store\.steampowered\.com/app/([0-9]+)(?:[/?#].*)?",
            value.strip(),
        )
        if match:
            yield str(int(match.group(1)))
    elif isinstance(value, dict):
        for child in value.values():
            yield from linked_app_ids(child)
    elif isinstance(value, list):
        for child in value:
            yield from linked_app_ids(child)

def decode_targets(value):
    if not isinstance(value, list):
        return [], None, None, False
    targets, unresolved = [], 0
    for entry in value:
        candidates = set(linked_app_ids(entry))
        if len(candidates) == 1:
            targets.extend(candidates)
        else:
            unresolved += 1
    return targets, len(value), unresolved, unresolved == 0
Enter fullscreen mode Exit fullscreen mode

If your export uses another representation, the inventory shows where this decoder needs adapting. Until then, its recovered graph is explicitly partial.

That restraint matters: a decoder that silently loses targets can manufacture apparent asymmetry.

Keep failed decoding out of the denominator

Build an audit row for every returned source, including rows that can't contribute edges. Preserve titles as attributes, and deduplicate ordered source-target pairs.

audit_rows, recovered_pairs = [], []
for record in items:
    source_id = canonical_app_id(record.get("appId"))
    payload = record.get("similarGames")
    targets, entries, unresolved, complete = decode_targets(payload)
    audit_rows.append({
        "source_id": source_id,
        "title": record.get("title"),
        "scrapedAt": record.get("scrapedAt"),
        "field_present": "similarGames" in record,
        "list_entries": entries,
        "unresolved_entries": unresolved,
        "parse_complete": complete and source_id is not None,
    })
    if source_id is not None:
        recovered_pairs.extend((source_id, target) for target in targets)

audit = pd.DataFrame(audit_rows, columns=[
    "source_id", "title", "scrapedAt", "field_present",
    "list_entries", "unresolved_entries", "parse_complete",
]).astype({
    "source_id": "string", "title": "string",
    "field_present": "bool", "parse_complete": "bool",
    "list_entries": "Int64", "unresolved_entries": "Int64",
})
audit["title"] = audit["title"].where(
    audit["title"].str.strip().ne("").fillna(False)
)
audit["scrapedAt"] = pd.to_datetime(
    audit["scrapedAt"], errors="coerce", utc=True, format="mixed"
)

edges = pd.DataFrame(
    recovered_pairs, columns=["source_id", "target_id"]
).astype("string").drop_duplicates()

nodes = (
    audit.loc[audit["source_id"].notna(), ["source_id", "title"]]
    .rename(columns={"source_id": "app_id"})
    .drop_duplicates("app_id")
)
Enter fullscreen mode Exit fullscreen mode

Here, parse_complete means every entry in a returned list decoded successfully. It does not certify that the export captured every recommendation Steam could display.

A present empty list passes that parsing check. An omitted field doesn't.

Target names remain conservative too: the node lookup uses titles attached to collected source rows. It doesn't guess which nested text belongs to a recovered target.

Measure reciprocity where a return link is observable

A missing reverse edge is interpretable only when the target's outgoing list was collected and decoded. Restrict the reciprocity denominator accordingly, while retaining unresolved coverage separately.

This query counts eligible directed edges, so reciprocal relationships contribute their observed directions. Self-links are excluded from that comparison.

connection = duckdb.connect()
connection.register("edges", edges)
connection.register("audit", audit)

reciprocity = connection.sql("""
    WITH comparable AS (
        SELECT e.source_id, e.target_id
        FROM edges e
        JOIN audit s
          ON e.source_id = s.source_id AND s.parse_complete
        JOIN audit t
          ON e.target_id = t.source_id AND t.parse_complete
        WHERE e.source_id <> e.target_id
    )
    SELECT
        count(*) AS eligible_directed_edges,
        count(r.source_id) AS reciprocated_directed_edges,
        count(r.source_id)::DOUBLE / nullif(count(*), 0)
            AS observed_reciprocity
    FROM comparable c
    LEFT JOIN edges r
      ON r.source_id = c.target_id
     AND r.target_id = c.source_id
""").df()

connection.close()
Enter fullscreen mode Exit fullscreen mode

A null reciprocity value means the denominator is empty. Substituting zero would turn “couldn't evaluate” into “evaluated and found none”.

Now calculate directed degree and the coverage measures needed to explain it.

degrees = (
    pd.concat([
        edges.groupby("source_id").size().rename("out_degree"),
        edges.groupby("target_id").size().rename("in_degree"),
    ], axis=1)
    .fillna(0)
    .astype("int64")
    .rename_axis("app_id")
    .reset_index()
    .merge(nodes, on="app_id", how="left")
)

target_titles = (
    edges[["target_id"]].drop_duplicates()
    .merge(nodes, left_on="target_id", right_on="app_id", how="left")
)
parsed_sources = set(audit.loc[audit["parse_complete"], "source_id"])

coverage = pd.Series({
    "returned_source_rows": len(audit),
    "rows_without_usable_app_id": audit["source_id"].isna().sum(),
    "rows_with_omitted_similarGames": (~audit["field_present"]).sum(),
    "rows_without_list_payload": audit["list_entries"].isna().sum(),
    "unresolved_list_entries": audit["unresolved_entries"].sum(),
    "sources_with_fully_parsed_lists": audit["parse_complete"].sum(),
    "recovered_directed_edges": len(edges),
    "targets_without_parsed_outgoing_lists": (
        ~target_titles["target_id"].isin(parsed_sources)
    ).sum(),
    "targets_without_source_row_title": target_titles["title"].isna().sum(),
    "missing_or_unparseable_timestamps": audit["scrapedAt"].isna().sum(),
    "earliest_observation": audit["scrapedAt"].min(),
    "latest_observation": audit["scrapedAt"].max(),
})
Enter fullscreen mode Exit fullscreen mode

The resulting degrees, reciprocity and coverage objects answer different questions. Keep them separate in your report.

Incoming degree measures recovered recommendation exposure within this collected graph. It isn't sales, market share or total visibility across Steam.

Give the chart a defensible caption

The worked example uses Steam Game Specs, Pricing & Media Extractor.

For a chart caption, describe the edge before describing the pattern: “An arrow indicates a recovered More like this recommendation from the source app's page to the target app.”

Then attach the source-selection method, effective inputs, observation interval and parsing coverage from your own pull. State that reciprocity was evaluated only where both endpoints had fully parsed outgoing lists.

Use labels such as “observed incoming recommendations” and “recovered outgoing recommendations”. “Competitor strength” adds a meaning the collection hasn't measured.

If an undirected layout is necessary, label its construction: “Apps connected by an observed recommendation in either direction.” Keep the underlying directed edges available so that the visual simplification doesn't become the analytical definition.

An arrow can't explain a purchase

These records don't reveal why Steam selected a recommendation, whether a particular person saw it, or whether it changed a buying decision.

A reciprocal link establishes that recommendations were recovered in both directions within your collection context. It still doesn't establish substitutability, shared customers or competitive pressure.

The arrow is evidence of a recommendation. Rivalry is another claim.

Drafted with AI assistance and reviewed for accuracy.

Top comments (0)