DEV Community

Fekri
Fekri

Posted on

Competitor Monitoring on GitHub Actions in Python: Scheduled App Store Snapshots Into a Repo, and the Walls You'll Hit (2026)

When a competitor ships a new version, rewrites their description, or swaps their screenshots, you usually find out weeks later by accident. You can fix that without a server, a database, or a scraper. A scheduled GitHub Actions workflow can pull public App Store metadata once a day, commit it to a repo, and open an issue when something you care about changes.

Git becomes the database. git log -p snapshots/us/297368629.json is the full history of one app's listing, and the diff view on GitHub shows exactly what changed.

This post builds that in about 100 lines of Python with no dependencies, plus one workflow file. I ran the script on Python 3.13 against the live API while writing this, tested the commit logic in a local repo, and checked the workflow with actionlint.

The data source

Apple's public Lookup API takes one or more App Store ids and returns the listing as JSON. No key, no account:

https://itunes.apple.com/lookup?id=297368629,6480417616&country=us&entity=software
Enter fullscreen mode Exit fullscreen mode

The id is the number in an app's store URL (apps.apple.com/us/app/.../id297368629). Pass several ids, comma separated, and you get them all back in one request, which matters later. country picks the storefront, and listings differ per storefront, so treat each one separately.

Configuration

A small apps.json holds what to watch. I'm using three calorie-tracking apps in two storefronts as the example:

{
  "countries": ["us", "gb"],
  "apps": [341232718, 297368629, 6480417616]
}
Enter fullscreen mode Exit fullscreen mode

The script

The script fetches every app per storefront in a single call, keeps the fields worth watching, compares them with yesterday's file, and writes a short report.

#!/usr/bin/env python3
"""Snapshot public App Store metadata for a few apps and report what changed.

Writes one JSON file per app and storefront under snapshots/. Git history
is the database: the workflow commits the folder once a day.
"""
import json
import os
import sys
import time
import urllib.parse
import urllib.request
from pathlib import Path

LOOKUP_URL = "https://itunes.apple.com/lookup"
OUT_DIR = Path("snapshots")

# Fields worth watching. Everything else is noise or changes too often.
WATCHED = [
    "trackName", "version", "currentVersionReleaseDate", "releaseNotes",
    "description", "formattedPrice", "primaryGenreName",
    "averageUserRating", "userRatingCount", "screenshotUrls",
]
# Fields that move almost every day; keep them, but don't alert on them.
QUIET = {"averageUserRating", "userRatingCount"}


def lookup(ids, country, retries=3):
    query = urllib.parse.urlencode(
        {"id": ",".join(map(str, ids)), "country": country, "entity": "software"}
    )
    req = urllib.request.Request(
        f"{LOOKUP_URL}?{query}", headers={"User-Agent": "app-snapshots/1.0"}
    )
    for attempt in range(retries):
        try:
            with urllib.request.urlopen(req, timeout=20) as resp:
                return json.load(resp).get("results", [])
        except Exception as exc:  # 403/429/5xx or a timeout
            wait = 10 * (attempt + 1)
            print(f"[{country}] lookup failed ({exc}), retrying in {wait}s",
                  file=sys.stderr)
            time.sleep(wait)
    raise RuntimeError(f"lookup failed for {country} after {retries} tries")


def normalise(app):
    snap = {k: app.get(k) for k in WATCHED}
    # Drop the host and the size suffix ("/392x696bb.jpg"); keep the asset path.
    snap["screenshotUrls"] = [
        urllib.parse.urlsplit(u).path.rsplit("/", 1)[0]
        for u in app.get("screenshotUrls", [])
    ]
    return snap


def diff(old, new):
    changes = []
    for key in WATCHED:
        if old.get(key) != new.get(key):
            changes.append((key, old.get(key), new.get(key)))
    return changes


def short(value, limit=80):
    text = json.dumps(value, ensure_ascii=False)
    return text if len(text) <= limit else text[: limit - 1] + "…"


def main():
    config = json.loads(Path("apps.json").read_text())
    lines = []
    for country in config["countries"]:
        results = {r["trackId"]: r for r in lookup(config["apps"], country)}
        for app_id in config["apps"]:
            path = OUT_DIR / country / f"{app_id}.json"
            app = results.get(app_id)
            if app is None:
                # Not sold in this storefront, pulled, or a flaky response.
                lines.append(f"- `{country}` {app_id}: missing from lookup (kept old snapshot)")
                continue
            new = normalise(app)
            old = json.loads(path.read_text()) if path.exists() else None
            if old is not None:
                for key, before, after in diff(old, new):
                    if key in QUIET:
                        continue
                    lines.append(f"- `{country}` **{new['trackName']}** `{key}`: "
                                 f"{short(before)} → {short(after)}")
            path.parent.mkdir(parents=True, exist_ok=True)
            path.write_text(json.dumps(new, indent=2, ensure_ascii=False, sort_keys=True) + "\n")
        time.sleep(3)  # be polite between storefronts

    report = "\n".join(lines) if lines else "No watched changes."
    print(report)
    Path("changes.md").write_text(report + "\n")
    # GitHub shows this on the run page; locally the variable is unset.
    if summary := os.environ.get("GITHUB_STEP_SUMMARY"):
        with open(summary, "a") as fh:
            fh.write("## App Store changes\n\n" + report + "\n")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Two design choices are worth explaining.

Watched vs. quiet fields. Rating count and average rating change nearly every day for any popular app. If you alert on them, you'll get an issue every morning and stop reading them within a week. So they're stored (the history is useful) but not reported. Version, release notes, name, description, price and screenshots are the changes a human actually wants to hear about.

One file per app per storefront. Small, stable, sorted JSON makes Git diffs readable. sort_keys=True and a trailing newline stop meaningless reorder diffs.

Running it twice in a row prints No watched changes.. When I edited a snapshot by hand to fake an old version, the next run reported:

- `us` **Lose It! – Calorie Counter** `version`: "0.0.1" → "18.5.702"
Enter fullscreen mode Exit fullscreen mode

I also faked a ratings change in the same file, and it stayed silent, which is what I wanted.

The workflow

name: app-store-snapshots

on:
  schedule:
    - cron: "17 6 * * *"   # 06:17 UTC, off the top of the hour
  workflow_dispatch: {}

permissions:
  contents: write
  issues: write

concurrency:
  group: snapshots
  cancel-in-progress: false

jobs:
  snapshot:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Take snapshots
        run: python snapshot.py
      - name: Commit if anything changed
        run: |
          git config user.name "snapshot-bot"
          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
          git add snapshots
          if git diff --cached --quiet; then
            echo "nothing to commit"
          else
            git commit -m "snapshots: $(date -u +%F)"
            git push
          fi
      - name: Open an issue for real changes
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          if ! grep -q "No watched changes" changes.md; then
            gh issue create --title "App Store changes $(date -u +%F)" --body-file changes.md
          fi
Enter fullscreen mode Exit fullscreen mode

Put it at .github/workflows/snapshots.yml, push, and trigger it once by hand from the Actions tab (that's what workflow_dispatch is for) to create the first snapshots.

The walls you'll hit

1. The schedule is best-effort, and it's UTC. Cron in Actions runs in UTC, so plan your "morning" accordingly. GitHub also says scheduled runs can be delayed when load is high, and the top of the hour is the busiest time. That's why the example uses minute 17. Don't build anything that needs to run at an exact minute on a scheduled trigger.

2. Schedules can switch themselves off. In public repos, GitHub disables scheduled workflows after 60 days without repository activity. Don't assume the bot's own daily commits keep it alive forever. Look at the Actions tab now and then, or you'll find a two-month hole in your history.

3. Missing apps fail silently. The Lookup API doesn't return an error for an id it can't find. It just leaves it out. That happens when an app isn't sold in a storefront, was pulled, or you copied the wrong id (I did, on my first run, and the report flagged it). The script reports a missing app and keeps the old snapshot, because deleting the file would make a flaky response look like a removal.

4. Noise will kill the alerts. Ratings were the obvious noise. Screenshots are the sneaky one. The URLs contain a size suffix like /392x696bb.jpg and a CDN host, so the script strips both and keeps the asset path. If you watch the raw URL, a change in how you request images looks like a redesign. Every rule like this is a judgement call, so start strict and loosen it when an alert turns out to be useless.

5. The token can do less than you think. The default GITHUB_TOKEN needs contents: write to push and issues: write to open issues, so declare them explicitly. Also, commits pushed with GITHUB_TOKEN don't trigger other workflows. If you planned a second workflow "on push to snapshots/", it won't fire. Chain steps in the same job instead.

6. Rate limits and terms. Apple doesn't publish firm numbers for these endpoints. The Search API docs have long suggested staying around 20 calls per minute, and you'll see 403s if you hammer it. Batching ids into one lookup per storefront keeps this workflow to a handful of calls a day. Store pages and private App Store endpoints are a different story: they sit behind bot protection, and scraping them can break Apple's terms. Read the terms before you build something commercial on top of this.

What this doesn't see

This workflow is good at one thing: telling you that a competitor's listing changed, with a Git diff as proof. It can't tell you what they're doing around the listing. That includes which paid ads they're running and for how long, which App Store searches they show up in, or which short-form videos are driving installs.

For that part I use PeekPanda*. It keeps a dated case file per iOS app with paid ads, observed search positions, organic content and download and revenue estimates, and you can put competitors on a board so they're tracked for you. It doesn't keep listing history, which is one reason I still run this little workflow next to it.

Wrap-up

  • One Python file, one workflow, and Git history give you a free daily record of competitor listings.
  • Batch ids per storefront, store everything, and alert only on fields a human cares about.
  • Expect late schedules, silent misses and noisy fields, and design for them from day one.

Run it for a month and read the diffs. Release notes alone will tell you a lot about what your competitors are shipping.

*I build it.

Top comments (0)