DEV Community

Cover image for Your README link is nofollow. I checked. Here's the script.
Livraith
Livraith

Posted on

Your README link is nofollow. I checked. Here's the script.

#seo #github #webdev #programming

My new domain had exactly one backlink.

I had just launched ainclave.com, ran a backlink report, and got one row back: a .ru redirect page with 44 outbound links and a spam score of 40. It pointed to my homepage with the anchor text "Ainclave.com".

Classic automated domain-scraper stuff.

That result didn't make sense at first. I had added the site to our GitHub organisation profile, several repositories, LinkedIn, and Mastodon. Four places linking to the domain, yet the report showed none of them.

I could have waited for another crawl and assumed the tool was behind. Instead, I checked the links themselves.

This is what I found, plus a small script you can use to check any page in about thirty seconds.

The claim I had never bothered to verify

At some point, I picked up the idea that a link in a GitHub README counts as an SEO backlink. You see this advice everywhere in developer SEO guides: add your site to GitHub and get a free link from a high-authority domain.

So I tested it.

curl -sL -A "Mozilla/5.0" "https://github.com/n8n-io/n8n" \
  | grep -o '<a[^>]*href="https\?://[^"]*n8n\.io[^"]*"[^>]*>' \
  | sort -u | head
Enter fullscreen mode Exit fullscreen mode

Output:

<a href="https://n8n.io/workflows" rel="nofollow">
<a href="https://docs.n8n.io" rel="nofollow">
<a href="https://community.n8n.io" rel="nofollow">
<a href="https://n8n.io/integrations" rel="nofollow">
Enter fullscreen mode Exit fullscreen mode

Every link had rel="nofollow".

Then I counted links to n8n.io that did not:

curl -sL -A "Mozilla/5.0" "https://github.com/n8n-io/n8n" \
  | grep -o '<a[^>]*href="https\?://[^"]*n8n\.io[^"]*"[^>]*>' \
  | grep -v nofollow | wc -l
# 0
Enter fullscreen mode Exit fullscreen mode

Zero.

That included the repository's Website field in the About sidebar. I had assumed GitHub might treat that one differently. It doesn't.

I repeated the check on my own organisation page. GitHub rendered eight links to my site, including the profile URL with itemprop="url". All eight had rel="nofollow".

GitHub applies this through its HTML sanitiser. It affects user-generated content across the platform: READMEs, About fields, issues, pull requests, discussions, wikis, and gists. There is no repository setting that turns it off.

A script for checking any page

Give it a page URL and your domain:

#!/usr/bin/env bash
# usage: ./check-link.sh <page-url> <your-domain>
set -euo pipefail

page="$1"
domain="$2"

html=$(curl -sL -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64)" --max-time 25 "$page")

anchors=$(printf '%s' "$html" | DOMAIN="$domain" perl -0777 -ne '
  my $d = quotemeta $ENV{DOMAIN};
  while (/<a\b[^>]*>/gs) {
    my $t = $&;
    $t =~ s/\s+/ /g;
    print "$t\n" if $t =~ /href\s*=\s*["\x27]https?:\/\/[^"\x27]*$d/i;
  }')

if [ -z "$anchors" ]; then
  echo "No external anchor to $domain found on $page"
  exit 1
fi

total=$(printf '%s\n' "$anchors" | grep -c '<a')
nf=$(printf '%s\n' "$anchors" | grep -ci 'nofollow' || true)

echo "anchors: $total | nofollow: $nf | without nofollow: $((total - nf))"
printf '%s\n' "$anchors" | grep -vi nofollow | head -5
Enter fullscreen mode Exit fullscreen mode

Checked against three pages:

github.com/n8n-io/n8n     n8n.io        13 anchors, 13 nofollow
github.com/ainclave       ainclave.com   8 anchors,  8 nofollow
ainclave.github.io        ainclave.com   7 anchors,  0 nofollow
Enter fullscreen mode Exit fullscreen mode

Two details in that script are load-bearing, and I got both wrong on my first attempt. More on that below.

What I found on sites developers commonly use

I used the same method on each site. These were the results when I checked; platforms can change their link policies, so I would rerun the test before relying on this table.

Where Result
GitHub README, About field, issues, gists nofollow throughout
LinkedIn company page website field nofollow; much of the page is also closed to crawlers
Mastodon profile metadata rel="me nofollow noopener"; many instances also use noindex
dev.to article body no rel attribute
Hacker News front-page story links no rel attribute
awesome-selfhosted.net 72 external anchors, none with nofollow

The last result caught my attention.

The awesome-selfhosted list is maintained on GitHub. Links on the GitHub version are sanitised like other user-generated links. But the rendered version on awesome-selfhosted.net is generated as the maintainers' own HTML, so those links do not inherit GitHub's attributes.

The distinction is simple: GitHub sanitises content displayed on github.com. It does not control HTML that someone publishes on a separate site.

What about GitHub Pages?

A site on username.github.io contains your HTML. GitHub Pages does not run your outbound links through the same content sanitiser used on github.com.

I published a small technical page there and checked the result:

<a class="deep-link" href="https://www.ainclave.com/security/microvm-isolation">
Enter fullscreen mode Exit fullscreen mode

No rel attribute. The page had seven outbound links, and none were marked nofollow.

That does not mean you should throw up a page containing nothing but links to your main domain. A thin page built only to pass link equity is unlikely to help and may look like a doorway page.

Give it a reason to exist. Mine contains a technical comparison that can stand on its own. The links point to supporting material rather than making up the whole page.

I also set the canonical URL to the GitHub Pages page itself. Pointing rel="canonical" at the main site would tell search engines that the page is a duplicate and that another URL should be indexed instead. That would work against the reason for publishing it separately.

nofollow does not make a link useless

It is easy to overcorrect here.

Google has treated nofollow as a hint rather than a strict directive since 2019. A nofollow link can still help a crawler discover a new URL. It can also send actual people to your site, which tends to get lost in backlink discussions.

For a new domain, discovery matters. A public GitHub profile that mentions your URL still has value. It just does not provide the clean, authority-passing backlink that many SEO guides imply.

I now think about these links in two separate categories:

  • Links that help crawlers and people find the site
  • Links that may also pass ranking signals

The rel attribute helps you understand which kind you may be looking at. It does not tell you the full SEO value of a link.

Two mistakes I made while checking

I nearly published false claims in this article. Twice, in different ways, and both times the command told me something I misread as a fact about the world.

grep reads one line at a time

While checking my GitHub Pages site, I looked for its meta description with:

curl -sL "$url" | grep -io '<meta[^>]*name="description"[^>]*>'
Enter fullscreen mode Exit fullscreen mode

The command returned nothing. I concluded that the tag was missing and repeated that conclusion twice.

The tag was there. It just looked like this:

<meta
  name="description"
  content="..."
>
Enter fullscreen mode Exit fullscreen mode

grep processes input one line at a time. My pattern expected the entire tag to appear on one line, so the command could never have matched the actual HTML.

I fixed the check by reading the document as a single string:

curl -sL "$url" \
  | perl -0777 -ne 'while (/<meta\b[^>]*>/gs) { my $t=$&; $t =~ s/\s+/ /g; print "$t\n" }'
Enter fullscreen mode Exit fullscreen mode

In Perl, -0777 reads the whole document at once, and /s allows . to match newlines.

A dot in a domain is a regex wildcard

The second one was subtler, and it produced a confident wrong answer rather than an empty one.

My first version of the link checker filtered anchors with grep -i "$2", passing the domain straight through as a pattern. Run it with n8n.io and the dot matches any character, so it also matches n8n-io — which appears in every internal repository path on the page. It reported this:

anchors: 154 | nofollow: 17 | without nofollow: 137
Enter fullscreen mode Exit fullscreen mode

A tidy, plausible, completely wrong result. The correct answer is 13 anchors, all of them nofollow.

Two changes fixed it. quotemeta escapes the domain so the dot is a dot. And the pattern now requires the domain to appear inside an absolute href, which drops internal links entirely.

The empty output taught me one thing; this one taught me a worse lesson. An empty result proves the check found nothing. A populated result does not prove the check was measuring what I thought it was. The second failure mode is more dangerous, because it looks like data.

If you know of a platform whose actual rel policy differs from the usual SEO advice, send me the page. I would rather test it than add another unchecked row to the table.

I write about sandboxing and EU-hosted infrastructure for coding agents at ainclave.com.

Top comments (1)

Collapse
 
fhiltscher profile image
Livraith •

Left out because I couldn't verify it: does Google actually follow nofollow links for discovery, or just reserve the right to?
Anyone seen a URL indexed whose only inbound link was nofollow?