Backlinks as structured observations: designing a conservative link-audit result

#architecture #debugging #seo

A backlink audit is easier to debug when it returns observations rather than a single success flag. A page can be reachable while its link is missing; a link can exist while the checker has no evidence about indexing.

This article proposes a small data contract for that distinction. It is a design note prepared with AI assistance, not a production benchmark.

Keep evidence separate from conclusions

For each check, record:

source URL and intended destination;
check timestamp and observation method (raw HTML or rendered DOM);
fetch outcome, including timeout and access denial;
observed href, anchor text, and rel tokens;
redirect destination, if actually followed;
scope limitations.

An empty list of anchors is meaningful only if you successfully inspected the relevant document. If the request failed, an empty list says nothing about the page's links.

Use explicit states

A minimal result can look like this:

{
  "fetch_state": "ok",
  "document_scope": "raw_html",
  "link_state": "present",
  "observed_href": "https://example.org/guide",
  "rel": ["ugc", "nofollow"],
  "index_state": "not_checked",
  "referral_state": "not_measured"
}

In this contract, present means the expected anchor was observed. It does not mean that the link passes ranking value or has generated visits. Likewise, nofollow belongs in metadata, not in a missing-link bucket.

A reducer with a conservative failure path

def link_state(fetch_state, document_complete, matching_anchor_count):
    if fetch_state != "ok" or not document_complete:
        return "unknown"
    return "present" if matching_anchor_count > 0 else "absent_in_observed_document"

assert link_state("timeout", False, 0) == "unknown"
assert link_state("denied", False, 0) == "unknown"
assert link_state("ok", False, 0) == "unknown"
assert link_state("ok", True, 1) == "present"
assert link_state("ok", True, 0) == "absent_in_observed_document"

These are illustrative assertions for the reducer, not evidence about any external website. The caller must decide what complete means. Raw HTML can be complete as an HTTP response while still lacking links added by JavaScript.

What to test around the reducer

Use fixtures for plain URL text, an anchor with href, a relative link, a different subdomain, a redirect, and a lookalike hostname. Match hosts using a URL parser rather than substring checks. Decide explicitly whether www and other subdomains belong to the same target set. Resolve relative links against the observed page URL before matching.

In a scheduled job, preserve the last successful observation and the latest attempt separately. An outage should not silently overwrite a previously observed link with a deletion claim. Retry conservatively and honor the site's access rules.

Where human review still matters

A technically present anchor can point to the wrong guide or appear in an irrelevant paragraph. A reviewer should assess the surrounding explanation and whether the destination answers the reader's question. That is an editorial decision, not a parser feature.

Google's link documentation provides the crawlable-anchor guidance. The contract above is one implementation proposal; it does not describe Google's internal systems.

Disclosure: I am involved in Keyserp. This example is shared as a technical design note about backlinks and evidence handling. It was prepared with AI assistance and makes no ranking or conversion claims.

DEV Community