Get Notified When You Gain or Lose a Backlink: A Python Monitor Without an SEO Subscription

Losing a good backlink is silent. Nobody emails you when a partner redesigns their site and drops your link, or when a competitor gets a mention in a big publication. The large SEO suites will alert you, but they're priced as monthly subscriptions, and often you only need the alert for a handful of domains.

This post covers:

1. where backlink data comes from (and why scraping an SEO tool's pages is a bad idea),
2. how to turn "all backlinks" into "what changed since last time",
3. what it costs, with numbers from a real run.

Where backlink data comes from

You can't check backlinks by visiting your own site. A backlink lives on someone else's page, so you need a crawler that has seen a large part of the web and kept the link graph. Building that yourself isn't realistic, and scraping a vendor's web UI breaks often and goes against their terms. What's left is to license an index through an API.

We use DataForSEO's Backlinks API. For each link it returns the linking page, anchor text, dofollow/nofollow, first and last seen dates, and 0–100 ranks for the linking domain and page.

The diff is the product

A full list of a big site's backlinks is interesting once. What you want every week is:

• new: links whose first_seen is after your last check,
• lost: links the crawler found missing since your last check.

The core of a DIY version is small: keep a hash of every change you've already reported, and only report what's new.

import hashlib, json, pathlib

STATE = pathlib.Path("backlink_state.json")

def change_key(row):
    # one key per (change type, linking page, target page)
    raw = f'{row["change"]}|{row["sourceUrl"]}|{row["targetUrl"]}'
    return hashlib.sha1(raw.encode()).hexdigest()

def only_unseen(changes):
    seen = set(json.loads(STATE.read_text())) if STATE.exists() else set()
    fresh = [c for c in changes if change_key(c) not in seen]
    STATE.write_text(json.dumps(sorted(seen | {change_key(c) for c in fresh})))
    return fresh

Three gotchas we hit while building this:

1. Index lag. A link first seen yesterday may only show up in the index today. If you query strictly "since my last run", you miss it for good. Query with an overlap window instead (we use 48 hours) and drop changes you already reported, using the hashes above. That way each change shows up exactly once.
2. Big domains are noisy. Large sites gain and lose links all the time, many of them sitewide or scraper links. For big sites, monitor referring domains instead of individual links, and sort the changes by the linking domain's rank.
3. Filters don't behave like SQL. On the referring-domains endpoint, an OR filter that includes lost_date silently drops every row where lost_date is null, which means all new domains. We ended up running "new" and "lost" as two separate queries.

Where to keep the state: one small JSON record per (monitor, domain) in a key-value store, holding the last check time and recent change hashes. We prune hashes after 60 days so the record stays small.

What the data looks like

Here's a real run: the 100 strongest backlinks each for backlinko.com and plausible.io, sorted by the linking domain's rank.

Domain	Domain rank	Backlinks (index total)	Referring domains
backlinko.com	70	886,292	59,047
plausible.io	74	680,015	17,423

A few things in that output are worth knowing before you build on it:

• The top 100 links by rank came from only 13 linking domains for backlinko.com and 4 for plausible.io. Strong sites often link many times from many pages. If you want breadth, deduplicate to one link per domain.
• Only 34 of the 200 links were dofollow. High-rank sites like GitHub mark most outbound links nofollow, so filter on dofollow if that's what you care about.
• Field fill rates over those 200 rows: linking URL, anchor, dofollow, first/last seen, domain rank and country were 100%; page language 99%; link attributes 90%; target status code 87%; the surrounding link text 69%.

The tool we use

Disclosure: we (SteadyScrape) built the tool below and get paid when it's used.

We packaged all of the above as an Apify Actor: Backlink Checker API: New and Lost Link Monitor (steady-scrape/backlink-explorer). It has four modes: backlinks, referring_domains, summary (up to 10,000 domains in one run) and monitor.

For the weekly alert, schedule it with an input like this:

{ "domains": ["mysite.com", "competitor.com"], "mode": "monitor", "monitorId": "weekly", "firstRunLookbackDays": 7 }

Each run returns only the links gained or lost since the previous run, each with anchor, dofollow and the linking domain's rank. A domain with no changes skips the per-link lookup.

Calling it from Python:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("steady-scrape/backlink-explorer").call(run_input={
    "domains": ["competitor.com"], "mode": "backlinks",
    "dofollowOnly": True, "minSourceRank": 40, "onePerDomain": True, "maxRowsPerDomain": 200,
})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
    if row["rowType"] == "backlink":
        print(row["sourceDomainRank"], row["sourceUrl"], "->", row["anchor"])

What it costs

Pricing is pay per event, with no subscription. List prices (Apify Free plan):

Event	Price
Run start (only if at least one domain returns data)	$0.03
Domain looked up in backlinks or referring-domains mode (summary row included)	$0.05
Backlink row	$0.0008 ($0.80 per 1,000)
Referring-domain row	$0.001
Domain in summary mode	$0.004
Domain checked in monitor mode	$0.05
New or lost link reported	$0.002

Apify also charges its standard $0.00005 start event. Row prices drop on paid Apify plans: a backlink row is $0.00072 on Starter, $0.00064 on Scale and $0.0006 on Business. Invalid inputs and domains with no data are free.

The run above (2 domains × 100 backlinks) works out to $0.03 + 2 × $0.05 + 200 × $0.0008 = $0.29 at list price, and it took about 6 seconds. Other examples at list price:

• 1 domain, 100 strongest backlinks: about $0.16
• Summary metrics for 100 domains: about $0.43
• Weekly monitor check of 10 domains: about $0.53, plus $0.002 per new or lost link

Most of the cost is the per-domain fee, so it pays to pull many links per domain, or to monitor only the domains you actually care about.

Caveats

• Ranks are the index's own 0–100 link-graph scores. They aren't the authority score from any other SEO tool, though they tend to move in the same direction.
• Every backlink index sees a different slice of the web. Compare trends over time, not absolute counts between tools.

Whatever you use to fetch the data, the monitoring rules above still apply: query with an overlap window, deduplicate changes by hash, and switch to referring domains for big sites.