DEV Community

asrocky010
asrocky010

Posted on

Your internal link graph is a to-do list: 33 pages, 6 neglected, 2 hours

#seo #webdev #beginners #analytics

Every SEO audit start with "you need more content". That is usually the expensive answer, and on a small site it is often the wrong one. The cheap answer is buried in the internal link graph, and it takes about twenty minutes to compute.

Parse your own site

You do not need a crawler. Glob the HTML files, pull href values with a regex, and count.

pages = {f[:-5]: f for f in glob.glob('*.html')}
inbound = collections.Counter()
for slug, f in pages.items():
    h = open(f).read()
    for t in set(re.findall(r'href="/([a-z0-9-]*)"', h)):
        if t in pages and t != slug:
            inbound[t] += 1
Enter fullscreen mode Exit fullscreen mode

That is the whole crawler. On a 33-page site it runs in under a second, and it produces the only number that matters here: how many other pages link to each page.

What the numbers looked like

The distribution was not flat. The homepage had 32 inbound links, the top three guides had 19 to 22, and then a long tail:

  • four pages with 2 inbound links
  • one page with 1

A page with one inbound link is not being linked, it is being mentioned. If it is also not in a sitemap, or the sitemap is stale, the only path a crawler has to it is whatever it picks up externally.

Why low in-degree hurts more than it looks

Internal links do three things, and only the first is about PageRank:

  1. Discovery. A page nobody links to has to be found some other way.
  2. Context. The anchor text tells the engine what the page is about. A page linked as "read more" is linked less usefully than one linked as "how long is the ServSafe test".
  3. Priority. Crawl budget on a new domain is small. Pages that are linked from many pages get revisited; pages linked from none get visited once.

The second point is underrated. Anchor text is one of the few on-page signals you fully control, and it is free.

The fix is smaller than it sounds

Not a link farm, not a footer stuffed with every URL. I added three contextual links to each neglected page, from pages that were topically adjacent, by appending a list item to an existing "related" list at the bottom of the host page.

The two rules that keep it honest:

  • the anchor text is the target page's heading, so the user knows where they are going
  • the link lives in a section that already existed for that purpose

The result: four pages went from 2 inbound links to 5, one from 1 to 4, and the only new content on the host pages was three list items.

Running it without breaking anything

The script that does this is idempotent in the same way as the injector: it skips a target if the host already contains the href. That means you can re-run it after adding pages, and it will only fill in what is still missing.

Then verify, because a regex-driven HTML edit is exactly the kind of change that silently breaks a page:

  • each file still has exactly one closing html tag
  • the related section count is unchanged
  • the new in-degree numbers come out where you expected

What I would do before writing another article

Run the link graph first. If you have pages with an in-degree below 3, you have material that is already written and already indexed that nobody can reach. Fixing that is an hour of work with a script, versus a day of work to produce one more page that will also need links.

The numbers above come from a 33-page study site - the same three-line crawler works on any static site: https://foodsafeprep.com/

Top comments (0)