Waleed Arshad

Posted on Aug 8

Audit Backlink Domain Concentration with Node.js (Without Fooling Yourself)

#seo #javascript #node #ai

A spreadsheet can contain hundreds of valid backlink URLs and still describe a fragile off-page footprint.

I ran into this while auditing 1,023 verified public placements. About 84.8% of them lived on four hosts. The URLs were real, but the graph was concentrated.

This tutorial builds a small Node.js audit that answers four practical questions:

How many unique placement URLs do we have?
How many unique referring hosts?
What share belongs to the largest one and four hosts?
Which hosts should be paused before the next publishing sprint?

The code uses only built-in Node APIs, so it is easy to run in CI or adapt to a Google Sheets export.

Input format

Export the tracker as a CSV with two columns:

name,url
Medium article,https://medium.com/@example/article
Company profile,https://directory.example.com/company
Technical guide,https://dev.to/example/guide

For a production tracker, use a proper CSV parser because titles can contain commas. The parsing below is intentionally small so the concentration logic is easy to see.

Normalize URLs before counting

Tracking parameters, fragments, casing, and www variants can make identical destinations look different.

function normalizeUrl(raw) {
  const url = new URL(raw.trim());
  url.hash = '';

  for (const key of [...url.searchParams.keys()]) {
    if (key.startsWith('utm_') || key === 'ref' || key === 'source') {
      url.searchParams.delete(key);
    }
  }

  url.hostname = url.hostname.toLowerCase().replace(/^www\./, '');
  url.pathname = url.pathname.replace(/\/+$/, '') || '/';

  return url.toString();
}

function referringHost(raw) {
  return new URL(normalizeUrl(raw)).hostname;
}

Do not remove every query parameter blindly. Some platforms use query parameters as part of the canonical identity of a public page. Strip only parameters you have classified as tracking noise.

Calculate concentration

function concentrationReport(rows) {
  const uniqueUrls = new Map();

  for (const row of rows) {
    try {
      const normalized = normalizeUrl(row.url);
      uniqueUrls.set(normalized, row);
    } catch {
      // Keep invalid or non-public URLs out of verified totals.
    }
  }

  const byHost = new Map();

  for (const normalized of uniqueUrls.keys()) {
    const host = referringHost(normalized);
    byHost.set(host, (byHost.get(host) || 0) + 1);
  }

  const hosts = [...byHost.entries()]
    .map(([host, placements]) => ({ host, placements }))
    .sort((a, b) => b.placements - a.placements);

  const total = uniqueUrls.size;
  const topOne = hosts[0]?.placements || 0;
  const topFour = hosts.slice(0, 4)
    .reduce((sum, row) => sum + row.placements, 0);

  return {
    totalPlacements: total,
    uniqueHosts: hosts.length,
    topOneShare: total ? topOne / total : 0,
    topFourShare: total ? topFour / total : 0,
    hosts
  };
}

Turn the result into a publishing control

A report is useful only if it changes the queue.

function allocationDecision(report, host) {
  const current = report.hosts.find(row => row.host === host);
  const share = current ? current.placements / report.totalPlacements : 0;

  if (share >= 0.20) return 'PAUSE';
  if (share >= 0.08) return 'REVIEW';
  return 'ELIGIBLE';
}

The thresholds are policy choices, not search-engine rules. Their purpose is to keep an automated publishing process from selecting the lowest-friction host forever.

One useful sprint rule is stricter: give the current top four hosts a temporary allocation of zero. That forces research into new editorial, technical, professional, and reviewed-directory surfaces.

Add source types

Hostname diversity alone is not enough. Ten unrelated directories still represent one kind of evidence.

Add a sourceType field such as:

editorial
technical
company_profile
reviewed_directory
social_post
community_answer
earned_media

Then calculate the same distribution by source type. Your goal is a portfolio of independent sources doing different jobs.

Verify links before counting

A row should enter the verified corpus only when:

The URL returns a public page in a signed-out session.
The page contains a clickable link to the intended site.
The placement is stable, relevant, and not duplicated.
Pending review and drafts remain in separate states.

For dynamic sites, a raw HTTP fetch may miss client-rendered content. Use a real browser check as a fallback, but do not treat an authenticated preview as proof that the page is public.

Why this matters for SEO and AEO

A raw backlink total is a production metric. Referring-domain and source-type diversity are closer to evidence metrics.

For answer-engine optimization, the external page also needs to contribute something worth citing: a definition, method, dataset, checklist, or defensible point of view. The linked destination should match the off-site context.

For example, a technical post about entity consistency should link to the relevant guide rather than defaulting to a homepage.

I am applying this measurement approach to the public SEO/AEO work documented by Corank.

The dashboard I would ship

At minimum, expose:

verified placement URLs
unique referring hosts
net-new hosts this sprint
top-one and top-four shares
source-type distribution
deep-link distribution
indexed placements
referral visits
non-branded impressions
AI answer mentions and cited-source appearances

That makes a failure mode visible before another few hundred URLs accumulate on the same host.

The point is not to replace judgment with a formula. It is to give the agent enough memory and constraints that judgment survives scale.

DEV Community