DEV Community

Daniel Pertu
Daniel Pertu

Posted on Edited on

I checked all 40 outbound links on our site, and the dead one is inside our privacy policy

#seo #webdev #nextjs #testing

Notifio is a desktop app that watches rental search pages for new listings. Its website is 38 pages, and an unusual share of those pages exist to send you somewhere else: each supported site gets a page that explains that portal's own alert feature and links to it, each comparison page quotes a competitor and links to their pricing, and the guides cite sources. Linking out is the product's argument, not a courtesy.

I had never checked whether any of those links still work.

curl -s https://notifio.app/sitemap.xml | grep -o '<loc>[^<]*' | sed 's/<loc>//' > urls.txt
# fetch all 38, extract every href, keep the absolute ones
Enter fullscreen mode Exit fullscreen mode 
45  distinct internal paths
81  distinct absolute URLs
41  of those on notifio.app itself (canonicals and JSON-LD)
40  distinct third-party URLs
51  link instances across the 38 pages
 9  URLs linked from more than one page
Enter fullscreen mode Exit fullscreen mode

Where they sit:

25  the alerts hub and its 15 per-site pages
 8  the five guides
 6  the three audience pages
 6  the five comparison pages
 4  the about page (our other apps)
 2  the privacy policy
Enter fullscreen mode Exit fullscreen mode

Then the check, following redirects, with an ordinary desktop user agent:

while read -r u; do
  curl -s -o /dev/null -L --max-time 25 -A "$UA" \
    -w "%{http_code}\t%{num_redirects}\t%{url_effective}\t$u\n" "$u"
done < third-party.txt
Enter fullscreen mode Exit fullscreen mode 
28  200
 9  403
 1  404
 1  401
 1  400
Enter fullscreen mode Exit fullscreen mode

The 404 

404  https://resend.com/privacy
Enter fullscreen mode Exit fullscreen mode

Linked from exactly one page: our privacy policy. It is the citation at the end of the sentence that tells you Resend delivers your activation token and your listing alerts, and that your email address is passed to them for that purpose. The live URL is https://resend.com/legal/privacy-policy. Ours has been pointing at a path that no longer exists.

That page names five processors: Vercel, Stripe, Resend, Upstash and Sentry. It links the policies of two of them. One of those two links is the 404. So the one document on the site with a real obligation to be accurate about third parties has a 50 per cent hit rate on the only part of itself a reader can click to verify.

The other link on that page is interesting in a smaller way:

200  https://stripe.com/privacy  ->  https://stripe.com/nl/privacy   (1 redirect)
Enter fullscreen mode Exit fullscreen mode

Stripe geolocates. A reader in Amsterdam and a reader in Leeds follow the same link from our policy and land on different localised copies of Stripe's document. Nothing wrong with that, and worth knowing before you ever quote a clause from it by paragraph number.

The 401 and the 400 

401  https://www.spotahome.com/my-alerts
400  https://www.facebook.com/facebook/posts/1233643168807701/
Enter fullscreen mode Exit fullscreen mode

The 401 is on our Spotahome page and it is the correct destination: it is where Spotahome's own saved-alert settings live, which is the thing the sentence is about, and it requires you to be signed in to them. Correct link, dead end for an anonymous reader, and not fixable by us beyond saying so in the link text.

The 400 is Facebook refusing a request that is not a browser. It opens fine for a person. It is on our Facebook Marketplace page and it is the oldest citation on the site.

The nine 403s, and why I am not writing a CI job 

403  pararius.com/info/pararius-plus
403  pararius.com/info/about-parariusplus
403  pararius.nl/nieuws
403  idealista.com/ayuda/categorias/alertas-inmueble/
403  idealista.com/ayuda/articulos/como-recibir-alertas-en-tu-email/
403  huurwoningen.nl/lidmaatschap/
403  help.openrent.co.uk/hc/en-gb/articles/360018359252-...
403  help.zoopla.co.uk/hc/en-gb/sections/360001666418-...
403  politie.nl/aangifte-of-melding-doen
Enter fullscreen mode Exit fullscreen mode

Every one of those opens in a browser. The last one is the Dutch police's "report a crime" page, cited from our guide on rental scams, and it is about as alive as a web page gets.

A 403 to a command-line client tells you about the client, not the link. Which is the practical finding of this whole exercise: 11 of 40 URLs answer something other than 200 to curl while being perfectly healthy, so a link checker wired into CI on a non-200 rule would be red forever and would get ignored within a week. The only automatable rule worth having is the narrow one: a 404 or a 410 is a real failure, everything else goes on a list a human reads. One of the 40 met that bar, and it was ours to fix.

The redirect that is a content bug

Two of the 28 successes went somewhere other than where we pointed them:

200  rentslam.com/en/pricing/  ->  rentslam.com/en/what-it-costs/
200  kamernet.nl/en/premium-account-payment
     ->  id.kamernet.nl/nl/login?authRequestID=...&prevUrl=...&target=tenant
Enter fullscreen mode Exit fullscreen mode

Our Rentslam comparison quotes their subscription price and links to the page it came from. They have since renamed that page. The redirect means the link still lands correctly today, and it also means the page we read the number off is not at the URL in our markup any more, which is the first sign that a quoted competitor price needs re-reading rather than re-linking. The comparison pages already carry a higher change frequency in the sitemap than their edit rate justifies, for exactly this reason: a stale sentence is a shrug, a stale price about somebody else is a problem.

The Kamernet one is a link to a paid feature that now bounces an anonymous visitor through to a login form with a request id in the query string. Still the right destination for a reader who has an account, still the wrong experience for the reader we wrote it for.

What changes

Three things, in order of how much they matter:

  1. resend.com/privacy becomes resend.com/legal/privacy-policy, and while I am in there, the three processors we name without a policy link get one. A legal page is the one place a reader deserves a working citation.
  2. The Rentslam link follows their rename, and the price on that page gets re-read rather than assumed.
  3. A monthly manual pass over 40 URLs, because 40 is a number a person can check over a coffee, and because the automatable version of this check would have caught the one real failure and shouted about ten healthy links at the same time.

The measurement that surprised me was not the 404. It was that the single dead link on a 38 page marketing site had found its way into the one document on that site whose entire job is to be true.

Top comments (0)