Watching GitHub releases with ETags and 304s in Python

If you depend on a handful of open-source projects, you probably want to know when one of them ships a release. When you administer the repository, a webhook is the right tool, and GitHub's own best-practice guide recommends webhook events over polling. For repositories you don't control, polling the REST API on a schedule is the usual answer, and it can be done cheaply.

This post builds a small standard-library Python script that:

1. asks the releases endpoint of each repository once per run,
2. sends the ETag from last time, so an unchanged list comes back as 304 Not Modified with no body,
3. keeps a small JSON state file with the release IDs it has already seen,
4. stays silent on the first run instead of announcing twenty "new" releases,
5. keeps the old state when a request fails, and stops when it is rate limited.

At the end there is a real, unauthenticated run from 27 September 2026 (UTC) and the complete script.

What the documentation says

These points come from GitHub's REST API documentation, read on 27 September 2026:

• Endpoint: GET /repos/{owner}/{repo}/releases. per_page defaults to 30, maximum 100. It works without authentication for public repositories. Draft releases are only listed for users with push access.
• Rate limit: unauthenticated requests get 60 per hour, counted per IP address. A personal access token raises that to 5,000 per hour.
• Conditional requests: most endpoints return an etag header. Send it back in if-none-match; if nothing changed you get 304 Not Modified. The docs say a 304 does not count against the primary rate limit if the request was made with an Authorization header.
• Versioning: requests without an X-GitHub-Api-Version header use version 2022-11-28. The current version is 2026-03-10. Pinning the version keeps a breaking change from reaching a script silently.
• Polling etiquette: poll only as often as you need, on a fixed schedule; make requests one after another, not concurrently.

Step 1: build the request

The If-None-Match header is only sent when there is a saved ETag. The token is optional and read from the environment, never from the code.

Excerpt of release_watch.py, lines 14-26:

API = "https://api.github.com/repos/{}/releases?per_page=20"
USER_AGENT = "release-watch-example/1.0 (tutorial script)"
KEEP_IDS = 200  # enough history to recognise releases we already reported


def build_request(url, etag=None):
    headers = {"User-Agent": USER_AGENT, "Accept": "application/vnd.github+json",
               "X-GitHub-Api-Version": "2026-03-10"}
    if etag:
        headers["If-None-Match"] = etag
    if os.environ.get("GITHUB_TOKEN"):
        headers["Authorization"] = "Bearer " + os.environ["GITHUB_TOKEN"]
    return urllib.request.Request(url, headers=headers)

Step 2: treat 304 as an answer, not an error

urllib raises HTTPError for a 304, exactly as it does for a 404 or a 503. Catching it and returning the status keeps the calling code simple: one status code to branch on.

Excerpt of release_watch.py, lines 29-35:

def fetch(url, etag=None):
    """Return (status, headers, JSON body or None). A 304 is a status here, not an exception."""
    try:
        with urllib.request.urlopen(build_request(url, etag), timeout=30) as response:
            return response.status, response.headers, json.loads(response.read())
    except urllib.error.HTTPError as err:  # urllib raises for 304 and for every 4xx/5xx
        return err.code, err.headers, None

Step 3: decide what is new

This is the core of the script. The rules:

• 304: nothing changed, the saved state stays as it is.
• Anything other than 200 or 304 (a 404, a 5xx, a rate limit): keep the saved state untouched. A failed request must never look like "no releases".
• First run for a repository: save every release ID and report nothing. Otherwise the first run would announce the project's whole history.
• Later runs: a release is new when its ID is not in the saved list. Compare IDs, not the fact that the response was a 200: the ETag can change for reasons other than a new release, as the real run below shows.

Excerpt of release_watch.py, lines 38-58:

def summary(release):
    return {key: release.get(key) for key in ("tag_name", "name", "published_at", "prerelease", "html_url")}


def check(repo, entry, fetch=fetch):
    """Check one repo against its saved state. Returns (new state entry, report)."""
    status, headers, body = fetch(API.format(repo), entry.get("etag"))
    report = {"repo": repo, "status": status, "new": [],
              "remaining": headers.get("x-ratelimit-remaining") if headers else None}
    if status != 200:  # 304: nothing changed. Anything else: keep the old state untouched.
        return entry, report
    releases = [r for r in body if not r.get("draft")]
    if "seen" in entry:
        seen = set(entry["seen"])
        fresh = [r for r in releases if r["id"] not in seen]
        report["new"] = [summary(r) for r in sorted(fresh, key=lambda r: r.get("published_at") or "")]
    else:
        report["baseline"] = len(releases)  # first run: remember everything, alert on nothing
    current = [r["id"] for r in releases]
    older = [i for i in entry.get("seen", []) if i not in set(current)]
    return {"etag": headers.get("ETag"), "seen": (current + older)[:KEEP_IDS]}, report

The saved list keeps the newest IDs first and is capped at 200 entries. per_page=20 means the script sees the 20 newest releases per request. If a project can publish more than 20 releases between two runs, raise per_page (up to 100) or follow the pagination links.

Step 4: the run loop and the state file

Requests go out one per second, in order. On a 403 or 429 the run stops rather than pushing on. The state file is written to a temporary file first and then moved into place with os.replace, so an interrupted run cannot leave half a JSON file behind.

Excerpt of release_watch.py, lines 74-97:

def main(repo_file, state_file):
    with open(repo_file, encoding="utf-8") as handle:
        repos = [line.strip() for line in handle if line.strip() and not line.startswith("#")]
    try:
        with open(state_file, encoding="utf-8") as handle:
            state = json.load(handle)
    except FileNotFoundError:
        state = {}
    print("run at", datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"))
    for number, repo in enumerate(repos):
        if number:
            time.sleep(1)  # serial requests, one per second
        state[repo], report = check(repo, state.get(repo, {}))
        print(describe(report))
        for release in report["new"]:
            print(f"  NEW {release['tag_name']}  published {release['published_at']}"
                  f"{'  (pre-release)' if release['prerelease'] else ''}  {release['html_url']}")
        if report["status"] in (403, 429):  # primary or secondary rate limit: stop, do not push on
            print("rate limited; stopping this run (state for earlier repos is still saved)")
            break
    temporary = state_file + ".tmp"
    with open(temporary, "w", encoding="utf-8") as handle:
        json.dump(state, handle, indent=2)
    os.replace(temporary, state_file)  # never leave a half-written state file

A network error that isn't an HTTP status (DNS failure, timeout) stops the whole run with a traceback before the state is written, so the previous state file stays intact.

Tests

Nine unittest tests use a fake API object instead of the network. The fake answers 304 when it receives its current ETag, so the conditional-request logic can be tested without sending a single request:

Excerpt of test_release_watch.py, lines 13-26:

class FakeApi:
    """Stands in for fetch(): answers 304 when the caller sends the current ETag."""

    def __init__(self, releases, etag='W/"v1"', status=200):
        self.releases, self.etag, self.status, self.calls = releases, etag, status, []

    def __call__(self, url, etag=None):
        self.calls.append(etag)
        headers = {"ETag": self.etag, "x-ratelimit-remaining": "59"}
        if self.status != 200:
            return self.status, headers, None
        if etag == self.etag:
            return 304, headers, None
        return 200, headers, list(self.releases)

Excerpt of test_release_watch.py, lines 36-41:

    def test_unchanged_repo_returns_304_and_keeps_state(self):
        api = FakeApi([release(1, "v1", "2026-01-01T00:00:00Z")])
        entry, _ = check("o/r", {}, fetch=api)
        again, report = check("o/r", entry, fetch=api)
        self.assertEqual((report["status"], report["new"], again), (304, [], entry))
        self.assertEqual(api.calls, [None, 'W/"v1"'])

Excerpt of test_release_watch.py, lines 60-64:

    def test_errors_keep_previous_state(self):
        previous = {"etag": 'W/"x"', "seen": [1]}
        for status in (404, 403, 429, 502):
            entry, report = check("o/r", previous, fetch=FakeApi([], status=status))
            self.assertEqual((entry, report["new"], report["status"]), (previous, [], status))

Excerpt of test-log.txt, lines 11-14:

----------------------------------------------------------------------
Ran 9 tests in 0.002s

OK

A real run

Four public repositories, chosen because they differ: three publish GitHub releases, one does not.

# owner/repo, one per line
pallets/flask
psf/black
astral-sh/ruff
python/cpython

The runs below were made without a token. GITHUB_TOKEN was removed from the environment for each run.

Run 1, with no state file:

$ python release_watch.py repos.txt state.json   # run 1, no state yet
run at 2026-09-27T13:13:50Z
pallets/flask        HTTP 200  baseline: 20 releases saved, nothing reported  (rate limit left: 51)
psf/black            HTTP 200  baseline: 20 releases saved, nothing reported  (rate limit left: 50)
astral-sh/ruff       HTTP 200  baseline: 20 releases saved, nothing reported  (rate limit left: 49)
python/cpython       HTTP 200  baseline: 0 releases saved, nothing reported  (rate limit left: 48)

python/cpython answered 200 with an empty list. That repository publishes no GitHub release objects, so a release watcher sees nothing there; the releases endpoint lists releases, not Git tags.

Run 2, eleven seconds later, with the ETags saved by run 1:

$ python release_watch.py repos.txt state.json   # run 2, nothing touched
run at 2026-09-27T13:14:01Z
pallets/flask        HTTP 304  not modified  (rate limit left: 47)
psf/black            HTTP 304  not modified  (rate limit left: 46)
astral-sh/ruff       HTTP 304  not modified  (rate limit left: 45)
python/cpython       HTTP 304  not modified  (rate limit left: 44)

Four 304s, no bodies, nothing reported.

Run 3. Waiting for a real release could take days, so to exercise the alert path, a small helper edits the state file: it removes the newest astral-sh/ruff release ID and the saved ETag, as if the last check had happened before that release. This is the only manipulated input in this post.

"""Demo helper: make the saved state look as if the last check ran before the newest release."""
import json
import sys

state_file, repo = sys.argv[1], sys.argv[2]
with open(state_file, encoding="utf-8") as handle:
    state = json.load(handle)
dropped = state[repo]["seen"].pop(0)  # the API lists the newest release first
del state[repo]["etag"]  # without this, the next request would just get a 304
with open(state_file, "w", encoding="utf-8") as handle:
    json.dump(state, handle, indent=2)
print(f"{repo}: forgot release id {dropped} and the saved ETag")

$ python forget_newest.py state.json astral-sh/ruff
astral-sh/ruff: forgot release id 396061666 and the saved ETag
$ python release_watch.py repos.txt state.json   # run 3
run at 2026-09-27T13:14:09Z
pallets/flask        HTTP 304  not modified  (rate limit left: 43)
psf/black            HTTP 304  not modified  (rate limit left: 42)
astral-sh/ruff       HTTP 200  1 new release(s)  (rate limit left: 41)
  NEW 0.16.9  published 2026-09-24T20:38:52Z  https://github.com/astral-sh/ruff/releases/tag/0.16.9
python/cpython       HTTP 304  not modified  (rate limit left: 40)

The watcher found exactly the one release that had been removed from its memory (published on 24 September 2026) and nothing else.

What the run shows

Unauthenticated 304s still cost a request. Look at the "rate limit left" column in run 2: 47, 46, 45, 44. Every 304 took one request from the 60-per-hour budget. That matches the documentation, which only promises free 304s for requests with an Authorization header. Without a token, conditional requests save bandwidth and parsing, not rate limit. With a token, they save both.

The ETag can change without a new release. The same four endpoints were also queried ten minutes earlier, at 13:03 UTC. Comparing those ETags with the ones saved at 13:13 to 13:14 UTC:

ETag at 13:03:03Z (earlier run) vs ETag in state.json after the 13:13:50Z-13:14:09Z runs
pallets/flask    same  W/"a15e35fe2fa... -> W/"a15e35fe2fa...
psf/black        CHANGED  W/"be5a64cb397... -> W/"eb174407a1d...
astral-sh/ruff   CHANGED  W/"804e242aa83... -> W/"50f608ba2e0...
python/cpython   same  "74d1db57f0fec... -> "74d1db57f0fec...
psf/black        newest release id at 13:03:03Z 324430137, now 324430137: same
astral-sh/ruff   newest release id at 13:03:03Z 396061666, now 396061666: same

Two ETags changed although the newest release ID of both repositories was the same at both times. Something else in the response changed. The run doesn't show what; release objects include fields that change without a new release, such as the download_count of each release asset, and an edited release note would also do it. That is why the script compares release IDs and never treats a 200 as proof of a new release.

Weak and strong ETags both work as-is. Three repositories returned weak ETags (W/"...") and python/cpython a strong one. The script stores the header value unchanged and sends it back unchanged.

Limits and common mistakes

• Announcing on the first run. Without the silent baseline, the first run reports every release the project ever made.
• Treating a failed request as "no releases". A 404, 5xx or 429 must leave the saved state alone.
• Treating any 200 as news. Compare IDs.
• Assuming 304s are free. Only with authentication, per the docs, and the run above confirms that unauthenticated 304s are counted.
• Pagination. With per_page=20, a burst of more than 20 releases between runs is partly missed.
• Polling too often. 60 requests per hour unauthenticated is shared by everything on your IP address. Pick a schedule that fits the number of repositories, and use a token if you need more.
• Retrying while rate limited. GitHub answers 403 or 429 for both the primary and the secondary rate limits. Its docs say to wait until x-ratelimit-reset when x-ratelimit-remaining is 0, to wait retry-after seconds when that header is present, and otherwise to wait at least a minute and back off exponentially; continuing to send requests while limited can get an integration banned. The script simply stops the run. A longer-lived tool should implement those waits.

Official sources

• REST API, releases: https://docs.github.com/en/rest/releases/releases
• Best practices, including conditional requests and polling: https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api
• Rate limits for the REST API: https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api
• API versions: https://docs.github.com/en/rest/about-the-rest-api/api-versions
• Python urllib.request: https://docs.python.org/3/library/urllib.request.html
• Python os.replace: https://docs.python.org/3/library/os.html#os.replace

The complete script and tests

release_watch.py (103 lines)

"""Watch GitHub repositories for new releases, using conditional requests.

Usage: python release_watch.py repos.txt state.json
Optional: set GITHUB_TOKEN in the environment to use your own (higher) rate limit.
"""
import json
import os
import sys
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone

API = "https://api.github.com/repos/{}/releases?per_page=20"
USER_AGENT = "release-watch-example/1.0 (tutorial script)"
KEEP_IDS = 200  # enough history to recognise releases we already reported


def build_request(url, etag=None):
    headers = {"User-Agent": USER_AGENT, "Accept": "application/vnd.github+json",
               "X-GitHub-Api-Version": "2026-03-10"}
    if etag:
        headers["If-None-Match"] = etag
    if os.environ.get("GITHUB_TOKEN"):
        headers["Authorization"] = "Bearer " + os.environ["GITHUB_TOKEN"]
    return urllib.request.Request(url, headers=headers)


def fetch(url, etag=None):
    """Return (status, headers, JSON body or None). A 304 is a status here, not an exception."""
    try:
        with urllib.request.urlopen(build_request(url, etag), timeout=30) as response:
            return response.status, response.headers, json.loads(response.read())
    except urllib.error.HTTPError as err:  # urllib raises for 304 and for every 4xx/5xx
        return err.code, err.headers, None


def summary(release):
    return {key: release.get(key) for key in ("tag_name", "name", "published_at", "prerelease", "html_url")}


def check(repo, entry, fetch=fetch):
    """Check one repo against its saved state. Returns (new state entry, report)."""
    status, headers, body = fetch(API.format(repo), entry.get("etag"))
    report = {"repo": repo, "status": status, "new": [],
              "remaining": headers.get("x-ratelimit-remaining") if headers else None}
    if status != 200:  # 304: nothing changed. Anything else: keep the old state untouched.
        return entry, report
    releases = [r for r in body if not r.get("draft")]
    if "seen" in entry:
        seen = set(entry["seen"])
        fresh = [r for r in releases if r["id"] not in seen]
        report["new"] = [summary(r) for r in sorted(fresh, key=lambda r: r.get("published_at") or "")]
    else:
        report["baseline"] = len(releases)  # first run: remember everything, alert on nothing
    current = [r["id"] for r in releases]
    older = [i for i in entry.get("seen", []) if i not in set(current)]
    return {"etag": headers.get("ETag"), "seen": (current + older)[:KEEP_IDS]}, report


def describe(report):
    status = report["status"]
    if status == 304:
        what = "not modified"
    elif "baseline" in report:
        what = f"baseline: {report['baseline']} releases saved, nothing reported"
    elif status == 200:
        what = f"{len(report['new'])} new release(s)"
    else:
        what = "request failed; state kept"
    return f"{report['repo']:<20} HTTP {status}  {what}  (rate limit left: {report['remaining']})"


def main(repo_file, state_file):
    with open(repo_file, encoding="utf-8") as handle:
        repos = [line.strip() for line in handle if line.strip() and not line.startswith("#")]
    try:
        with open(state_file, encoding="utf-8") as handle:
            state = json.load(handle)
    except FileNotFoundError:
        state = {}
    print("run at", datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"))
    for number, repo in enumerate(repos):
        if number:
            time.sleep(1)  # serial requests, one per second
        state[repo], report = check(repo, state.get(repo, {}))
        print(describe(report))
        for release in report["new"]:
            print(f"  NEW {release['tag_name']}  published {release['published_at']}"
                  f"{'  (pre-release)' if release['prerelease'] else ''}  {release['html_url']}")
        if report["status"] in (403, 429):  # primary or secondary rate limit: stop, do not push on
            print("rate limited; stopping this run (state for earlier repos is still saved)")
            break
    temporary = state_file + ".tmp"
    with open(temporary, "w", encoding="utf-8") as handle:
        json.dump(state, handle, indent=2)
    os.replace(temporary, state_file)  # never leave a half-written state file


if __name__ == "__main__":
    if len(sys.argv) != 3:
        sys.exit(__doc__)
    main(sys.argv[1], sys.argv[2])

test_release_watch.py (90 lines)

import os
import unittest
from unittest import mock

from release_watch import KEEP_IDS, build_request, check


def release(rid, tag, published, draft=False):
    return {"id": rid, "tag_name": tag, "name": tag, "published_at": published,
            "prerelease": False, "draft": draft, "html_url": f"https://example/{tag}"}


class FakeApi:
    """Stands in for fetch(): answers 304 when the caller sends the current ETag."""

    def __init__(self, releases, etag='W/"v1"', status=200):
        self.releases, self.etag, self.status, self.calls = releases, etag, status, []

    def __call__(self, url, etag=None):
        self.calls.append(etag)
        headers = {"ETag": self.etag, "x-ratelimit-remaining": "59"}
        if self.status != 200:
            return self.status, headers, None
        if etag == self.etag:
            return 304, headers, None
        return 200, headers, list(self.releases)


class Check(unittest.TestCase):
    def test_first_run_is_a_silent_baseline(self):
        api = FakeApi([release(2, "v2", "2026-02-01T00:00:00Z"), release(1, "v1", "2026-01-01T00:00:00Z")])
        entry, report = check("o/r", {}, fetch=api)
        self.assertEqual((report["baseline"], report["new"]), (2, []))
        self.assertEqual(entry, {"etag": 'W/"v1"', "seen": [2, 1]})

    def test_unchanged_repo_returns_304_and_keeps_state(self):
        api = FakeApi([release(1, "v1", "2026-01-01T00:00:00Z")])
        entry, _ = check("o/r", {}, fetch=api)
        again, report = check("o/r", entry, fetch=api)
        self.assertEqual((report["status"], report["new"], again), (304, [], entry))
        self.assertEqual(api.calls, [None, 'W/"v1"'])

    def test_new_releases_reported_oldest_first(self):
        entry = {"etag": 'W/"old"', "seen": [1]}
        api = FakeApi([release(3, "v3", "2026-03-01T00:00:00Z"), release(2, "v2", "2026-02-01T00:00:00Z"),
                       release(1, "v1", "2026-01-01T00:00:00Z")], etag='W/"new"')
        entry, report = check("o/r", entry, fetch=api)
        self.assertEqual([r["tag_name"] for r in report["new"]], ["v2", "v3"])
        self.assertEqual(entry, {"etag": 'W/"new"', "seen": [3, 2, 1]})

    def test_drafts_are_ignored(self):
        api = FakeApi([release(2, "v2", None, draft=True), release(1, "v1", "2026-01-01T00:00:00Z")])
        _, report = check("o/r", {"seen": [1]}, fetch=api)
        self.assertEqual(report["new"], [])

    def test_repo_with_no_releases_becomes_a_baseline_of_zero(self):
        entry, report = check("o/r", {}, fetch=FakeApi([]))
        self.assertEqual((report["baseline"], entry["seen"]), (0, []))

    def test_errors_keep_previous_state(self):
        previous = {"etag": 'W/"x"', "seen": [1]}
        for status in (404, 403, 429, 502):
            entry, report = check("o/r", previous, fetch=FakeApi([], status=status))
            self.assertEqual((entry, report["new"], report["status"]), (previous, [], status))

    def test_seen_ids_are_capped(self):
        entry = {"seen": list(range(1000, 1000 + KEEP_IDS))}
        new_entry, _ = check("o/r", entry, fetch=FakeApi([release(1, "v1", "2026-01-01T00:00:00Z")]))
        self.assertEqual(len(new_entry["seen"]), KEEP_IDS)
        self.assertEqual(new_entry["seen"][0], 1)


class Request(unittest.TestCase):
    def test_conditional_header_only_with_etag(self):
        with mock.patch.dict(os.environ, {}, clear=True):
            plain = build_request("https://api.github.com/x")
            conditional = build_request("https://api.github.com/x", 'W/"abc"')
        self.assertIsNone(plain.get_header("If-none-match"))
        self.assertEqual(conditional.get_header("If-none-match"), 'W/"abc"')
        self.assertIsNone(plain.get_header("Authorization"))
        self.assertEqual(plain.get_header("X-github-api-version"), "2026-03-10")

    def test_token_is_sent_only_when_set(self):
        with mock.patch.dict(os.environ, {"GITHUB_TOKEN": "test-value"}, clear=True):
            self.assertEqual(build_request("https://api.github.com/x").get_header("Authorization"),
                             "Bearer test-value")


if __name__ == "__main__":
    unittest.main()

Run the tests with python -m unittest -v test_release_watch.

---

This article and its code were drafted by an AI assistant at the account owner's request. The code was run against the live GitHub API without a token on 27 September 2026, from 13:13:50 to 13:14:09 UTC (the earlier ETag comparison is from 13:03:03 UTC), and the tests passed the same day.