DEV Community

Daniel Pertu
Daniel Pertu

Posted on

Fifteen pages, 45 sibling links, and the one page no sibling links to

Notifio is a desktop app that watches rental search pages and tells you when a new listing appears. Its marketing site is 38 URLs: ten routes typed out by hand, fifteen per-site pages, five comparison pages, and eight long-form articles.

I have written about where those pages come from before. Fifteen required fields and no optional ones was about the type that refuses to let a site have a page unless there is something true to say about that site alone. Our footer links five rental sites out of fifteen was about why the footer hand-picks five of them instead of importing the catalogue.

Neither post asked the obvious follow-up question: those pages link to each other, that linking is content I typed, and nothing in the repo checks it. So I counted the graph.

Three slugs per page, resolved at render time

Each site in the catalogue names three siblings:

export type AlertSite = {
  slug: string;
  name: string;
  // ...
  /** Slugs of three sibling pages, for internal linking. */
  related: string[];
};
Enter fullscreen mode Exit fullscreen mode

string[], not a union of the slugs that exist. The resolution happens at render:

/** Resolves `related` slugs, dropping any that do not exist yet. */
export function relatedSites(site: AlertSite): AlertSite[] {
  return site.related
    .map(getAlertSite)
    .filter((related): related is AlertSite => related !== undefined && related.slug !== site.slug);
}
Enter fullscreen mode Exit fullscreen mode

That is a deliberate choice and a trap at the same time. The upside is that I can write related for a page before its sibling exists, and the link simply appears when the other page lands. The downside is that "rightmve" compiles, renders nothing, and looks exactly like a page with two siblings instead of three.

The articles have the same arrangement twice over:

export type Article = {
  // ...
  /** Slugs of /alerts/<slug> pages worth linking from here. */
  relatedSites: string[];
  /** Slugs of other articles, any kind. */
  relatedArticles: string[];
};
Enter fullscreen mode Exit fullscreen mode

relatedArticles goes through a resolver that drops unknowns. relatedSites is resolved inline in the layout component instead, with its own copy of the same filter:

const sites = article.relatedSites
  .map(getAlertSite)
  .filter((site): site is NonNullable<ReturnType<typeof getAlertSite>> => site !== undefined);
Enter fullscreen mode Exit fullscreen mode

Two implementations of "drop what does not resolve", in two files, for one rule.

Counting it

The catalogue is a TypeScript literal in three files, so a regex over the source is good enough to count edges. It is not a parser and it does not have to be: I control the shape of the file it reads.

const SITES = ["netherlands", "uk", "europe"]
  .map((f) => fs.readFileSync(`src/lib/alerts/sites/${f}.ts`, "utf8"))
  .join("\n")
  .matchAll(/slug: "([^"]+)",.*?related: \[([^\]]*)\]/gs);

const inbound = new Map();
for (const [, slug, rel] of SITES) {
  for (const [, target] of rel.matchAll(/"([^"]+)"/g)) {
    inbound.set(target, (inbound.get(target) ?? 0) + 1);
  }
}
Enter fullscreen mode Exit fullscreen mode

Three numbers come out of that, plus the same treatment for the articles:

  • 45 sibling links between the 15 site pages, three per page, every one of them resolving
  • 30 links from the 8 articles to site pages, all resolving
  • 24 links between articles, all resolving

So 99 internal links are typed as slugs in content files, and 99 of them point at something real. That part is fine. The distribution is not.

Inbound links, by page

Page From siblings From articles From footer
kamernet 8 5 yes
pararius 4 6 yes
funda 2 4 yes
rightmove 3 3 yes
spareroom 4 2 yes
openrent 4 1 no
housinganywhere 4 1 no
idealista 3 2 no
huurwoningen 4 0 no
zoopla 3 0 no
spotahome 2 1 no
wg-gesucht 1 2 no
immobilienscout24 2 0 no
roommatch 1 1 no
facebook-marketplace 0 2 no

Kamernet is the centre of gravity: eight of the 45 sibling links point at it, including one from every page in the Europe file. That was not a decision I remember making. It is what happens when you write related for a new page by asking yourself which existing page a reader would most likely want next, fifteen times in a row.

The four UK pages are a complete graph. Rightmove, Zoopla, SpareRoom and OpenRent each name the other three, so every one of them has three siblings pointing back. The Dutch five are nearly that, with one asymmetry: Kamernet does not name Funda, and Funda names Kamernet.

And then there is Facebook Marketplace, which names three siblings and is named by none.

What zero sibling links actually costs

It is not zero internal links. That page is in the hub's ItemList, in the sitemap, and in the related-sites block of two guides: rental scams in the Netherlands and the Amsterdam page. Both of those mention it for the same reason, which is that a marketplace with no landlord verification is where scam listings live.

What it does not have is a route in from a page about a similar product. A reader on the OpenRent page who wants the next thing to read is offered Rightmove, SpareRoom and Zoopla. A crawler doing the same walk never arrives. On a 38 page site that is a small effect, and I am not going to pretend I can measure it. The reason it bothers me is different: the asymmetry is invisible in the source. Each of the three files looks complete on its own, because each site page names three siblings and nothing in the file shows which pages name it.

The assertion I should have written first

The website's tests run under node --test with no framework, in five files, and the ones I like best do not test code at all: they assert that content is true. There is already a test that counts rendered title lengths against the 60 character budget, because the layout's title template quietly spends ten of them.

The link graph deserves the same treatment, and it is four assertions:

describe("alert site link graph", () => {
  const slugs = new Set(ALERT_SITES.map((s) => s.slug));

  it("names only sites that exist", () => {
    for (const site of ALERT_SITES) {
      for (const slug of site.related) {
        assert.ok(slugs.has(slug), `${site.slug} links to missing "${slug}"`);
      }
    }
  });

  it("never names itself", () => {
    for (const site of ALERT_SITES) {
      assert.ok(!site.related.includes(site.slug), `${site.slug} links to itself`);
    }
  });

  it("gives every page at least one sibling pointing at it", () => {
    const inbound = new Map(ALERT_SITES.map((s) => [s.slug, 0]));
    for (const site of ALERT_SITES) {
      for (const slug of site.related) inbound.set(slug, (inbound.get(slug) ?? 0) + 1);
    }
    const orphans = [...inbound].filter(([, n]) => n === 0).map(([slug]) => slug);
    assert.deepEqual(orphans, [], `no sibling links to: ${orphans.join(", ")}`);
  });
});
Enter fullscreen mode Exit fullscreen mode

The first two pass today. The third fails on one page, which is the right outcome for a test: it is not describing a bug in code, it is describing a gap in content, and the fix is a word in a file rather than a patch.

The same test for articles is the same shape, because the data is the same shape. That is the part worth taking away from this if you have a content-driven site: the moment a slug appears inside a content file, that slug is a foreign key with no database to enforce it, and the only thing standing between you and a quietly broken link graph is an assertion you wrote on purpose.

You can see both ends of this graph live. The hub lists all fifteen sites grouped by market, each per-site page ends with its three siblings, and each guide ends with the sites it touches. The product those pages are selling is on the pricing page, which is one link away from every one of them, because that link is in the template rather than in the content.

Top comments (0)