403 doesn't mean dead: four ways a link checker lies to you

nelson-digital — Fri, 21 Aug 2026 19:15:00 +0000

tags: node, webdev, showdev, testing

I had a list of 115 domains and one question: which of these are still alive?

The naive version is ten lines. Fetch each one, check the status code, done. I wrote that, ran it, and it got 12 of the 115 wrong — in both directions. Here is what it missed, because every one of these is a different failure wearing the same disguise.

1. A 403 is almost never a dead site

Cloudflare's challenge page returns HTTP 403 to anything that doesn't look like a browser. The body is about 23 words and the title is Just a moment....

My first pass called every one of those dead. That threw away eight perfectly good domains, including Sooper Articles, The Free Library, Diigo, HubPages, Typepad and the SitePoint and Joomla forums. All alive. All reachable by a human. All reported as gone.

if (res.status === 403 || /just a moment|attention required/i.test(title)) {
  return { verdict: "LIVE", note: "bot wall, host is up" };
}

The trade-off is real: a genuinely dead host that happens to 403 now reads as alive. For my use case that is the cheaper mistake by a wide margin. Deleting a working entry costs me something real. Keeping a dead one costs five minutes when someone tries it.

2. Parked domains return a cheerful 200

A domain that lapsed and got picked up by a parking service is up. It resolves, it serves HTML, it returns 200. It is also completely useless.

My first attempt matched on parking-service brand names — godaddy, sedo, bodis, namecheap. That is the obvious approach and it is wrong, because plenty of live sites load a script, a font or an analytics beacon from one of those hosts.

It confidently reported that Zotero was for sale.

The fix is to match only phrases that appear on a genuine sale page and nowhere else:

const FOR_SALE = /\b(this domain (name )?is for sale|buy this domain|make an offer on this domain)\b/i;

Narrow and boring beats clever here. The other parking pattern worth catching is the JS bounce, where the page is a stub that redirects to /lander on load.

3. Byte length is not a proxy for content

I tried "is the response under N bytes" as a shortcut for "this page is empty". It fails immediately on anything client-rendered — an SPA ships a big HTML file with a <div id="root"> and nothing else.

Strip the scripts and the markup first, then count what a human would actually read:

const wordCount = (html) =>
  html.replace(/<script[\s\S]*?<\/script>/gi, "")
      .replace(/<style[\s\S]*?<\/style>/gi, "")
      .replace(/<[^>]+>/g, " ")
      .replace(/\s+/g, " ").trim()
      .split(" ").filter(Boolean).length;

Under about 30 words and there is no page there, whatever the byte count says.

4. A connection error is several different problems

fetch throwing tells you almost nothing on its own. These all land in the same catch block and mean very different things:

Code	What it actually means
`ENOTFOUND`	The host does not resolve. Genuinely gone.
`ECONNREFUSED`	DNS resolves, nothing is listening.
`CERT_HAS_EXPIRED`	The site works fine in a browser after a click-through.
`ERR_TLS_CERT_ALTNAME_INVALID`	Certificate does not cover this hostname. Often a live site behind a misconfigured proxy.
`AbortError`	Slow, not dead. Worth a retry before you write it off.

Lumping the last three in with the first is how you delete working entries.

What I ended up with

Four verdicts instead of a boolean:

LIVE — real page, or a bot wall, which still means a human can reach it
DEAD — does not resolve, refused, expired cert, 4xx or 5xx
PARKED — for sale, or bouncing to a parking lander
EMPTY — 200 with essentially nothing rendered

$ node check.mjs sites.txt
LIVE   sooperarticles.com    bot wall, could not read the page but the host is up
LIVE   zotero.org
DEAD   some-old-directory.com   host does not resolve

3 checked  LIVE 2  DEAD 1

No dependencies, Node 18+, MIT. It's here if it's useful: github.com/damani-pixel/citation-check