DEV Community

Cover image for Your README link is nofollow. I checked. Here's the script.
Livraith
Livraith

Posted on

Your README link is nofollow. I checked. Here's the script.

#seo #github #webdev #programming

My new domain had exactly one backlink.

I had just launched ainclave.com, ran a backlink report, and got one row back: a .ru redirect page with 44 outbound links and a spam score of 40. It pointed to my homepage with the anchor text "Ainclave.com".

Classic automated domain-scraper stuff.

That result didn't make sense at first. I had added the site to our GitHub organisation profile, several repositories, LinkedIn, and Mastodon. Four places linking to the domain, yet the report showed none of them.

I could have waited for another crawl and assumed the tool was behind. Instead, I checked the links themselves.

This is what I found, plus a small script you can use to check any page in about thirty seconds.

The claim I had never bothered to verify

At some point, I picked up the idea that a link in a GitHub README counts as an SEO backlink. You see this advice everywhere in developer SEO guides: add your site to GitHub and get a free link from a high-authority domain.

So I tested it.

curl -sL -A "Mozilla/5.0" "https://github.com/n8n-io/n8n" \
  | grep -o '<a[^>]*href="https\?://[^"]*n8n\.io[^"]*"[^>]*>' \
  | sort -u | head
Enter fullscreen mode Exit fullscreen mode

Output:

<a href="https://n8n.io/workflows" rel="nofollow">
<a href="https://docs.n8n.io" rel="nofollow">
<a href="https://community.n8n.io" rel="nofollow">
<a href="https://n8n.io/integrations" rel="nofollow">
Enter fullscreen mode Exit fullscreen mode

Every link had rel="nofollow".

Then I counted links to n8n.io that did not:

curl -sL -A "Mozilla/5.0" "https://github.com/n8n-io/n8n" \
  | grep -o '<a[^>]*href="https\?://[^"]*n8n\.io[^"]*"[^>]*>' \
  | grep -v nofollow | wc -l
# 0
Enter fullscreen mode Exit fullscreen mode

Zero.

That included the repository's Website field in the About sidebar. I had assumed GitHub might treat that one differently. It doesn't.

I repeated the check on my own organisation page. GitHub rendered eight links to my site, including the profile URL with itemprop="url". All eight had rel="nofollow".

GitHub applies this through its HTML sanitiser. It affects user-generated content across the platform: READMEs, About fields, issues, pull requests, discussions, wikis, and gists. There is no repository setting that turns it off.

A script for checking any page

Give it a page URL and your domain:

#!/usr/bin/env bash
# usage: ./check-link.sh <page-url> <your-domain>
set -euo pipefail

page="$1"
domain="$2"

html=$(curl -sL -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" --max-time 25 "$page")

anchors=$(printf '%s' "$html" | DOMAIN="$domain" perl -0777 -ne '
  my $d = quotemeta $ENV{DOMAIN};
  while (/<a\b[^>]*>/gs) {
    my $t = $&;
    $t =~ s/\s+/ /g;
    print "$t\n" if $t =~ /href\s*=\s*["\x27]https?:\/\/[^"\x27]*$d/i;
  }')

if [ -z "$anchors" ]; then
  echo "No external anchor to $domain found on $page"
  exit 1
fi

total=$(printf '%s\n' "$anchors" | grep -c '<a')
nf=$(printf '%s\n' "$anchors" | grep -ci 'nofollow' || true)

echo "anchors: $total | nofollow: $nf | without nofollow: $((total - nf))"
printf '%s\n' "$anchors" | grep -vi nofollow | head -5
Enter fullscreen mode Exit fullscreen mode

Checked against three pages:

github.com/n8n-io/n8n     n8n.io        13 anchors, 13 nofollow
github.com/ainclave       ainclave.com   8 anchors,  8 nofollow
ainclave.github.io        ainclave.com   7 anchors,  0 nofollow
Enter fullscreen mode Exit fullscreen mode

Two details in that script are load-bearing, and I got both wrong on my first attempt. More on that below.

What I found on sites developers commonly use

I used the same method on each site. These were the results when I checked; platforms can change their link policies, so I would rerun the test before relying on this table.

Where Result
GitHub README, About field, issues, gists nofollow throughout
LinkedIn company page website field nofollow; much of the page is also closed to crawlers
Mastodon profile metadata rel="me nofollow noopener"; many instances also use noindex
dev.to article body no rel attribute
Hacker News front-page story links no rel attribute
awesome-selfhosted.net 72 external anchors, none with nofollow

The last result caught my attention.

The awesome-selfhosted list is maintained on GitHub. Links on the GitHub version are sanitised like other user-generated links. But the rendered version on awesome-selfhosted.net is generated as the maintainers' own HTML, so those links do not inherit GitHub's attributes.

The distinction is simple: GitHub sanitises content displayed on github.com. It does not control HTML that someone publishes on a separate site.

What about GitHub Pages?

A site on username.github.io contains your HTML. GitHub Pages does not run your outbound links through the same content sanitiser used on github.com.

I published a small technical page there and checked the result:

<a class="deep-link" href="https://www.ainclave.com/security/microvm-isolation">
Enter fullscreen mode Exit fullscreen mode

No rel attribute. The page had seven outbound links, and none were marked nofollow.

That does not mean you should throw up a page containing nothing but links to your main domain. A thin page built only to pass link equity is unlikely to help and may look like a doorway page.

Give it a reason to exist. Mine contains a technical comparison that can stand on its own. The links point to supporting material rather than making up the whole page.

I also set the canonical URL to the GitHub Pages page itself. Pointing rel="canonical" at the main site would tell search engines that the page is a duplicate and that another URL should be indexed instead. That would work against the reason for publishing it separately.

nofollow does not make a link useless

It is easy to overcorrect here.

Google has treated nofollow as a hint rather than a strict directive since 2019. A nofollow link can still help a crawler discover a new URL. It can also send actual people to your site, which tends to get lost in backlink discussions.

For a new domain, discovery matters. A public GitHub profile that mentions your URL still has value. It just does not provide the clean, authority-passing backlink that many SEO guides imply.

I now think about these links in two separate categories:

  • Links that help crawlers and people find the site
  • Links that may also pass ranking signals

The rel attribute helps you understand which kind you may be looking at. It does not tell you the full SEO value of a link.

Two mistakes I made while checking

I nearly published false claims in this article. Twice, in different ways, and both times the command told me something I misread as a fact about the world.

grep reads one line at a time

While checking my GitHub Pages site, I looked for its meta description with:

curl -sL "$url" | grep -io '<meta[^>]*name="description"[^>]*>'
Enter fullscreen mode Exit fullscreen mode

The command returned nothing. I concluded that the tag was missing and repeated that conclusion twice.

The tag was there. It just looked like this:

<meta
  name="description"
  content="..."
>
Enter fullscreen mode Exit fullscreen mode

grep processes input one line at a time. My pattern expected the entire tag to appear on one line, so the command could never have matched the actual HTML.

I fixed the check by reading the document as a single string:

curl -sL "$url" \
  | perl -0777 -ne 'while (/<meta\b[^>]*>/gs) { my $t=$&; $t =~ s/\s+/ /g; print "$t\n" }'
Enter fullscreen mode Exit fullscreen mode

In Perl, -0777 reads the whole document at once, and /s allows . to match newlines.

A dot in a domain is a regex wildcard

The second one was subtler, and it produced a confident wrong answer rather than an empty one.

My first version of the link checker filtered anchors with grep -i "$2", passing the domain straight through as a pattern. Run it with n8n.io and the dot matches any character, so it also matches n8n-io — which appears in every internal repository path on the page. It reported this:

anchors: 154 | nofollow: 17 | without nofollow: 137
Enter fullscreen mode Exit fullscreen mode

A tidy, plausible, completely wrong result. The correct answer is 13 anchors, all of them nofollow.

Two changes fixed it. quotemeta escapes the domain so the dot is a dot. And the pattern now requires the domain to appear inside an absolute href, which drops internal links entirely.

The empty output taught me one thing; this one taught me a worse lesson. An empty result proves the check found nothing. A populated result does not prove the check was measuring what I thought it was. The second failure mode is more dangerous, because it looks like data.

If you know of a platform whose actual rel policy differs from the usual SEO advice, send me the page. I would rather test it than add another unchecked row to the table.

I write about sandboxing and EU-hosted infrastructure for coding agents at ainclave.com.

Top comments (4)

Collapse
 
alexshev profile image
Alex Shev •

I like the script-first framing. For local SEO, I would use the same habit on citations and partner pages: crawl the actual rendered links, capture rel attributes, canonical state, and indexability, then decide whether a mention is evidence or just decoration. Manual spot checks miss too much.

Collapse
 
fhiltscher profile image
Livraith •

Correct, and the script has that gap. It only flags nofollow. ugc and sponsored belong in the same check. Canonical and indexability is the part I underrated. A dofollow link on a noindex page is decoration. Minimum honest check: rendered HTML, full rel value, robots meta, canonical target.
Citation and partner pages are the harder case. How many mentions survive that check on your side?

Collapse
 
alexshev profile image
Alex Shev •

That minimum honest check is the right bar. I would add one more boring field: when the page was last verified. A partner page that passed six months ago can quietly become noindex, canonicalized elsewhere, or wrapped in a JS shell. The link is only as useful as the last crawl that proved it.

Collapse
 
fhiltscher profile image
Livraith •

Left out because I couldn't verify it: does Google actually follow nofollow links for discovery, or just reserve the right to?
Anyone seen a URL indexed whose only inbound link was nofollow?