Sonnet 5 dropped today. Watch the other hand.

Vinicius Pereira — Tue, 30 Jun 2026 22:04:06 +0000

Sonnet 5 landed today and everyone's busy benchmarking it. Fair, it looks like a strong, cheap, very agentic model, close to Opus 4.8 for a fraction of the cost. But the most important model news this month isn't Sonnet 5. It's that Fable 5, Anthropic's most powerful model, has been switched off worldwide since June 12, and it's still down.

Not rate-limited. Not deprecated. Export-controlled. Anthropic got an emergency US Commerce Department directive citing national security after a way to jailbreak Fable 5's safeguards surfaced, and since they couldn't verify the nationality of every request in real time, they pulled it for everyone, Americans included. Mythos 5 got a partial reprieve for a handful of cyber-defense orgs. Fable 5 is still dark.

Sit with that. The most capable model on the market right now isn't gated by price, it's gated by law. "Available to everyone who can pay" quietly became "available to whoever the government allows." That's a different universe to build a company in.

Now look at what shipped in the same window: Sonnet 5, explicitly the cheaper, more agentic, good-enough model. The industry's answer to "the frontier just got pulled" is "you probably didn't need the frontier anyway." Which, honestly, has been true for most of us for a while. Most production AI fails on reliability, eval, and retrieval, not on raw model IQ. A bigger brain was never the thing standing between your demo and prod.

So the take: chasing the newest frontier was always a weak moat, and now it's a fragile one, it can be switched off by a government letter on a Tuesday, it literally just was. The teams that win the next year are the ones who (a) made their systems reliable enough to run a cheaper or open model without flinching, and (b) didn't bet the company on a single vendor in a single jurisdiction. Reliability and independence beat raw capability the moment capability becomes a policy decision.

Tell me I'm wrong. Is Sonnet 5 a real step change for your use case, or is the actual headline that the frontier now ships with an export-control letter attached?

What changed since the last scrape? A small change-detection layer (stdlib only)

Vinicius Pereira — Tue, 30 Jun 2026 20:23:53 +0000

Most of my scrapers answer one question: what's on the site right now. But that's almost never the question I actually have. What I care about is what changed since the last run. A new listing showed up, a price dropped, a product disappeared, a status flipped from open to closed. The current snapshot on its own doesn't tell me any of that.

For a while I rebuilt the same thing on every project: load last run's JSON, compare it to this run, work out what's new, what's gone, and what changed. It's never hard, but it's fiddly, and I kept getting the same details wrong (more on that below). So I pulled it into one small reusable piece and stopped rewriting it. It's called scrape-sentinel.

This post is about the design more than the tool, because the interesting part is the handful of decisions that make change detection annoying to get right.

The core idea

You give it the records from this run and a key, and it tells you what was added, removed, and changed since last time. For changed records, it tells you which fields moved and from what to what.

The diff itself is a plain function with no I/O:

from scrape_sentinel import diff

cs = diff(previous_records, current_records, key="sku", ignore_fields=["scraped_at"])

for r in cs.added:
    print("new:", r["sku"])
for changed in cs.changed:
    for d in changed.deltas:
        print(changed.key, d.field, d.old, "->", d.new)

Which prints something like:

new: W-104
W-101 price 39.0 -> 35.0
W-101 in_stock True -> False

The details that kept biting me

A few decisions are the whole reason this is worth extracting instead of rewriting inline every time:

Match by key, not by position. This is the big one. If you diff two lists positionally, a re-sorted page or a reordered API response looks like every row changed. Matching on a stable key (one field or a few) means a reordered run shows zero changes, which is correct.
The first run is a baseline. With no previous snapshot, everything looks new. The first run just records state and stays quiet instead of firing an alert for all 4,000 items.
Ignore the noisy fields. A scraped_at timestamp or a session token changes every single run. You drop those from the comparison, or restrict it to an allow-list of fields you actually care about.
Write snapshots atomically. The state file is written to a temp file and renamed, so a run that dies halfway can't leave you with a corrupted snapshot that breaks the next comparison.

Using it for real

In practice you want the diff plus the I/O around it: load the last snapshot, run your scraper, compare, alert, save the new snapshot.

from scrape_sentinel import (
    CallableSource, PipelineConfig, SnapshotStore,
    ConsoleAlerter, WebhookAlerter, run_once,
)

def scrape() -> list[dict]:
    # your requests / Playwright / API code, returns a list of dicts
    return fetch_products()

config = PipelineConfig(
    key="sku",
    ignore_fields=["scraped_at"],
    alerters=[
        ConsoleAlerter(title="catalog", key_fields=("sku",)),
        WebhookAlerter(SLACK_URL, title="catalog", key_fields=("sku",)),
    ],
)

changes = run_once(CallableSource(scrape), SnapshotStore("./.state"), config)
print(changes.summary())   # {'added': 1, 'removed': 1, 'changed': 1, 'unchanged': 2}

Alerts go to the console, a webhook (Slack or Telegram), or a JSON change log. There's also a CLI with a --fail-on-change exit code, so you can put it on a cron job or a CI step and have the next step run only when something actually moved:

scrape-sentinel run --source json:catalog.json --key sku --state ./.state --webhook "$SLACK_WEBHOOK"

What it is not

It's not a scraper. It doesn't crawl anything for you. You bring your own requests, Playwright, or API client and hand it a list of dicts, and it owns the diff, the alert, and the snapshot. It's also standard library only, no dependencies, so dropping it into an existing project doesn't pull in a tree. The diff being a pure function is what made it easy to test heavily, which is where most of the suite lives.

Repo

MIT licensed: https://github.com/vinimabreu/scrape-sentinel

Honestly curious how other people handle this. Do you diff inside the database, keep snapshots on disk like this, hash each record, or something cleaner? It feels like the kind of thing everyone quietly rebuilds, so I'd like to know what I missed.