I spent about fifteen minutes on Monday crawling a site I don't own: thestackanalyst.com, a small independent SEO-tools blog. Not a client, not a competitor — I picked it because it's the kind of site most of us actually run. WordPress, a few dozen posts, one person maintaining it.
I wanted to answer one question with real data instead of vibes: if you crawl a small site with free tooling, what do you actually catch?
The answer, in this case: two orphan pages, one dead outbound link, and zero broken internal links. That last part matters — this site is in good shape. The point of this post isn't "look at this broken site." It's the method, run against a real example, so you can point it at your own domain this week.
The method: three steps, no paid tools
Every step here is something you can do with a free crawler and a spreadsheet. I used the free crawler at rankforge.cc, but the technique is tool-agnostic — anything that will fetch your URLs and record internal links works.
Step 1 — Crawl from the homepage
Start at the root and let the crawler follow internal links until it runs out. This gives you the set of pages that are reachable by following links. That distinction is the whole trick.
Step 2 — Diff that set against your sitemap
Your sitemap is the set of pages you say exist. Your crawl is the set of pages a crawler can find by walking your site. Anything in the sitemap but not in the crawl is an orphan: live, indexable, and structurally invisible. The diff, on this site:
$ crawl https://www.thestackanalyst.com | diff - sitemap.xml
> https://www.thestackanalyst.com/seo-tools (0 internal links pointing at it)
> https://www.thestackanalyst.com/text-tools (0 internal links pointing at it)
Both lines exist in the sitemap. Neither appears as the target of a single internal link anywhere in the crawl.
Step 3 — Check link statuses, internal and outbound
Same crawl, one extra pass. Internal 404s are self-inflicted damage. Outbound dead links are a different opportunity entirely — more on that at the end.
What the crawl actually found
Hard numbers, no rounding up:
| Metric | Result |
|---|---|
| URLs fetched (crawl) | 200 |
| HTTP 200 responses | 197 |
| Non-200 | 2× wp-json oEmbed (400), 1× xmlrpc.php (403) — WP infrastructure, not user pages |
| Sitemap URLs declared | 62 |
| Broken internal links | 0 (beyond those two WP endpoints) |
| Homepage internal links | 53 |
| Homepage external links | 3 |
| Orphan pages | 2: /seo-tools and /text-tools |
| Outbound links checked (Top 100 listicle) | 134, exactly 1 dead (keywordshitter.com) |
Both orphans return HTTP 200. Both are in the sitemap. And across the entire 200-URL crawl — homepage, blog index, every post — zero internal links point to either one.
They're tool-hub pages. Someone built them, published them, submitted them in the sitemap, and never got around to wiring them into the nav.
They're two doors with no hallway leading to them.
I want to be completely fair here: this is exactly what almost every site looks like, mine included. Two orphans and no broken internal links on a site this size is a healthy result. The site owner is doing fine.
Why orphans are worth caring about
Three practical reasons, no statistics required.
No internal anchor text signals. Every internal link is a small statement about what a page is about. An orphan has none. Google has to guess from the page alone.
Discovery depends entirely on the sitemap. A sitemap is a hint, not a guarantee. When it's the only path to a page, you've removed every redundancy from that page's discoverability.
Link equity can't flow to it. That homepage with 53 internal links is distributing authority across the site. Orphans get zero of it, forever, no matter how good they are.
For /seo-tools and /text-tools, the sting is that these are hub pages — the page type that should be collecting internal links from every relevant post and passing authority down to individual tools.
What to do with each orphan
Don't reflexively delete. Run the page through three questions:
Is it good and still relevant? → Link it in. Not a footer dump — footer links are weak and readers ignore them. Put it in the main nav if it's a hub, and add contextual in-body links from the two or three posts most related to it. For /seo-tools, that means linking it from every post that mentions an SEO tool. Anchor text should describe the destination, not say "click here."
Is it redundant with a stronger page? → 301 redirect it to that stronger page and fold any unique content in. Neither thin page beats one solid page — merge them.
Is it necessary but not search-worthy? (thank-you pages, filtered views, internal utilities) → noindex, and remove it from the sitemap. An orphan you intend to be an orphan isn't a bug — but it shouldn't be sitting in your sitemap sending mixed signals.
The failure mode to avoid: leaving a page indexable, sitemapped, and unlinked.
The bonus you get for free: outbound link rot
Same crawl, extra pass. I checked the outbound links on their big "Top 100 Free SEO Tools" listicle — 134 tool links checked.
Exactly one was dead: keywordshitter.com (yes, real tool, real name — Dan Shure's free keyword tool). It timed out across http, https, and www variants. The other 133 resolved fine — for a listicle that size, a genuinely good ratio. But a dead link in a "here are the tools" post is a bad reader experience, and it's the kind of thing an author never notices because nobody re-clicks their own links.
Here's why this is worth your time beyond tidiness: it's the most honest outreach angle there is. You email the editor, name the dead link sitting in their list, and suggest a live alternative that does the same job. No pitch, no "I noticed you link to X, would you consider linking to me" dance. You're doing unpaid editorial QA on their most important page. Sometimes you get a link out of it. Often you just get a reply from a real person, which is worth more than most cold outreach ever produces.
Run this against any listicle in your niche and you'll find candidates — roundup posts rot constantly, and tools die quietly.
Fifteen minutes, this week
That's the whole exercise. One crawl, one sitemap diff, one link-status pass. Two orphan pages surfaced, one dead outbound link found, and a clean bill of health confirmed on everything else — on a site I'd never touched before.
You almost certainly have orphans right now. Not because you're careless, but because sites accumulate pages faster than they accumulate links. This one had two. Yours almost certainly has at least one.
If you want the fast version of step 2, the orphan page checker does the crawl-versus-sitemap diff for you and hands you the list. Then work the three questions above on each result: link it, redirect it, or noindex it.
Then go check that one listicle you keep linking to. I'd bet something on it is dead.
Top comments (0)