DEV Community

Cover image for My Site Serves 4,008 Recommendations, Never Names 537 Posts
Mustafa ERBAY
Mustafa ERBAY

Posted on Originally published at mustafaerbay.com.tr

My Site Serves 4,008 Recommendations, Never Names 537 Posts

I pulled my own archive page with curl today: 2,079,641 bytes. Two megabytes of HTML holding the titles of 1,336 posts, sorted by year, delivered in a single request. The page weight didn't bother me — the question that surfaced while I scrolled it did. What do these 1,336 posts have to do with each other?

I thought I knew the answer. Every post page ends with a "Related Posts" section: three cards, picked by category and tag similarity. So the archive should be walkable from the inside. That's what I assumed.

Then I counted. (Every number below describes the Turkish half of the site — 1,336 posts. The English half runs its own engine, and I come back to it later.) Links I wrote between my own posts: 301. Recommendation cards the engine produces: 4,008. Posts those 4,008 cards never touch: 537.

An archive grows by addition, a library by connection. I automated the first and left the second to a rule I wrote in August.

The links I wrote by hand: 301

The measurement is simple: I searched the bodies of all 1,336 Turkish .mdx files under src/content/blog/ — the archive as it stood before this post — for the pattern ](/blog/<category>/<slug>/) and deduplicated the source-target pairs.

The most dangerous part of any measurement is assuming the data outside your pattern doesn't exist. So I went after my own script first: what if I'd written links with different syntax? <a href="/blog/…"> — zero. JSX href={...} — zero. Relative paths (](../) — zero. Slash-less ](blog/ — zero. The absolute form mustafaerbay.com.tr/blog/ does appear twice; I opened both, and they sit inside curl examples in a code block in a single post — and both point at /blog/technology/some-post/, a placeholder that isn't even a real post.

The script itself didn't come out clean either. It read dates with the pattern publishDate: 2026-05-02, and one post in the archive writes its date in quotes (publishDate: "2026-05-02"), which my pattern walked straight past. A single record — but it is exactly the failure I was congratulating myself for avoiding two paragraphs ago: treating what falls outside your pattern as nonexistent. Every number below comes from the corrected script.

What's left is clean: 309 raw occurrences, 301 unique edges after deduplication. The gap comes from posts naming the same target twice. No post links to itself.

Here's the distribution:

Metric Value
Turkish posts 1,336
Post-to-post links (unique) 301
Posts with zero inbound links 1,136 (85.0%)
Posts with zero outbound links 1,181 (88.4%)
Highest inbound count on any post 6

My most-linked post has six inbound links. Six. In a library, being cited six times is a compliment; being the top of a 1,336-book shelf with it is not.

By category, the worst score belongs to the one this post sits in: 92.4% of life posts (194 of 210) have nothing of mine pointing at them. career is 89.9%, tutorials 82.6%, technology 81.0%. The personal pieces stand alone; the technical ones call each other more often, because a post explaining a mechanism has to lean on another mechanism.

The habit has a birth date

Broken down by month, the table talks. Share of posts that link out:

Month Posts Linking out
2024-05…07 27 0 (0%)
2026-03 5 0 (0%)
2026-04 368 0 (0%)
2026-05 274 4 (1.5%)
2026-06 232 2 (0.9%)
2026-07 200 0 (0%)
2026-08 97 20 (20.6%)
2026-09 96 92 (95.8%)
2026-10 37 37 (100%)
Total 1,336 155

(The first two rows are the blog's original 2024 run and March 2026 — a handful of posts, none of them linking out.)

In August I added a line to the production recipe: at least one internal link. The line does its job — 95.8% in September, 100% in October. The rule works. The rule is young. The gap between those two sentences is named 1,106 posts: everything published before August, of which 91.2% is still orphaned today.

Something else caught my eye: these new links don't only look next door. The median age gap of an edge is 14.5 days, the mean 55.3 — the distribution has two humps. 37.7% point at something from the past week, but 31.7% reach 91 to 180 days back. Sixty-six edges land in the April archive. So the habit does look backwards; the problem isn't intent, it's arithmetic.

What the engine picks

The code behind "Related Posts" is short. For each post it scores every other post: same category +3, each shared tag +1. It drops the zeros, sorts by score, prints the top three.

The real detail is in the sort. The post list arrives from the database with ORDER BY publish_date DESC, and JavaScript's sort is stable: two candidates with equal scores keep the order they came in. So when scores tie, the newer post wins. A three-card shelf always shows the three most recent eligible posts.

I simulated that mechanism for all 1,336 posts — doing exactly what the code does, minus the HTTP request:

  • Recommendation slots produced: 4,008 (1,336 × 3; the theoretical ceiling is fully used, so no post is left without recommendations of its own)
  • Distinct posts named across those 4,008 cards: 799 (59.8%)
  • Posts never named at all: 537 (40.2%)

The single most-recommended post appears on 102 different pages. The window display is full; the shelves are deserted.

A measurement should audit its own assumptions: in my simulation, ties between posts published on the same day resolve by file order, while in production they resolve by SQLite's row order. With an April average of twelve posts a day, that's not a footnote. So I reran it with five different random tie orders: 797, 798, 800, 801, 802 distinct posts named. The identity of the winners shifts; the result doesn't. Roughly 800 — the rest stay dark.

Combining both paths leaves this number: 476 posts (35.6%) receive neither a sentence of mine nor a card from the engine. No post page on the site points at them.

Invisibility has two causes: thin tags, and age at the tiebreak

My first guess was that the 537 unnamed posts were simply the oldest ones. If newer wins the ties, the bottom of the pile should be the posts from the blog's first run.

That guess was half right. The 2024 archive really is almost entirely dark: 70.6% of May 2024, all of June, 83.3% of July are never named. But the monthly table muddies it — 25.5% in April, 26.3% in May, 56.5% in June, 75.5% in July. April's posts, three months older, are twice as visible as July's. Age alone doesn't explain it.

The missing piece sits in the average tag count for those same months: 5.47 tags in April, 5.04 in May, 2.85 in June, 2.23 in July. Remember the scoring — same category +3, each shared tag +1. A two-tag post competes with most rivals on at most two shared-tag points, and never cracks a three-slot window.

To separate the two variables I held each one still in turn. Age first: even among posts born in the same month, tag count draws the line.

Month Never named Named
2026-05 72 posts, avg 4.24 tags 202 posts, avg 5.32 tags
2026-06 131 posts, avg 2.27 tags 101 posts, avg 3.59 tags
2026-07 151 posts, avg 2.05 tags 49 posts, avg 2.82 tags

Then tags: among posts carrying the same number of tags, age does its own work. Here is the monthly split of the 343 two-tag posts beside that of the 373 five-tag posts:

Month 2-tag cohort 5-tag cohort
2026-04 — 44/174 (25.3%)
2026-05 — 22/105 (21.0%)
2026-06 108/128 (84.4%) 1/26 (3.8%)
2026-07 144/157 (91.7%) —
2026-08 27/42 (64.3%) —
2026-09 — 10/38 (26.3%)
2026-10 — 4/28 (14.3%)

Carrying two tags makes you invisible 84% of the time in June and 92% in July; carrying five drops June to 3.8%. Yet inside the five-tag cohort the rate still swings between 3.8% and 26.3% by month. Both forces are real: tag count gets you into the race, age settles the ties. The code said so all along — tags set the score, ORDER BY publish_date DESC breaks the draws.

Across the whole archive the tag threshold is startlingly sharp:

Tags Never named
1 19 / 19 (100%)
2 293 / 343 (85.4%)
3 41 / 150 (27.3%)
4 48 / 149 (32.2%)
5 83 / 373 (22.3%)
6 or more 53 / 302 (17.5%)
Total 537 / 1,336

All nineteen single-tag posts are invisible, without exception. So are 85.4% of the two-tag ones. At three tags the rate drops to 27%. The cliff sits between two and three.

I ran the prescription before writing it down

The conclusion from the tables above looked obvious: there are 362 posts carrying two tags or fewer, and 312 of them are named nowhere. Give each of them one more tag and 312 posts come back. An afternoon's work.

I simulated it before writing it. Just as well:

Intervention Never named Rescued Newly invisible
(none) 537 — —
+1 popular tag on all 362 520 24 7
+1 category-appropriate tag 526 21 10
+2 popular tags 515 38 16
+3 popular tags 520 38 21

Not 312 — 22 posts net, at best. And piling on more tags makes the result worse again.

The reason fits in one line: 4,008 is a fixed ceiling. Every page prints three cards, no more and no fewer. Adding tags creates no new slots; it only changes who fills them, pushing one post out for every post it rescues. I taught the archive to play a zero-sum game better. I didn't make the game bigger.

And the English half? There the engine produces its own separate 4,008 slots, and 512 of the 1,336 English posts (38.3%) are named on no page at all — very nearly the same picture. No surprise: translations inherit their tags verbatim, so a thinly tagged post goes invisible in both languages at once. One more detail: because the engine filters posts by language, a Turkish post never recommends its own English sibling. Two versions of the same piece, each unaware the other exists.

That corrects the diagnosis too. Thin tagging isn't the cause of invisibility; it's the visible face of the cause. The real cause is that 4,008 slots, handed out to 1,336 posts by a score, naturally pile up. Whoever loses, somebody has to lose.

I am not saying "unreachable"

Here's where I have to keep myself honest: having no inbound links is not the same as being unreachable.

The archive page lists every one of them — a curl against it today returns HTTP 200 and 1,336 unique post links, which is exactly why it weighs two megabytes. The index pages hold 24 each, so 56 pages of browsing. There are category pages, search, feed.json and RSS. My sitemap is complete too: sitemap-0.xml carries 2,703 addresses, 2,672 of which are every post in both languages. Mechanical access is solid, and that part was deliberate.

Diagram

But those three paths are not the same thing. The archive is a filing cabinet; nobody browses to page 56 out of curiosity. A link inside a sentence, on the other hand, carries a judgment: look there, because the claim here leans on it. A cabinet doesn't tell you what to read.

I can't count tag pages in that list, because my own robots.txt closes them: Disallow: /tags/ and Disallow: /en/tags/. The reasoning is written right there — low-value pages, don't spend crawl budget on them. The decision is defensible on its own terms, but its consequence is this: tags work inside the engine's scoring and are not a discovery path at all on the search-engine side.

The search-engine side rhymes with that. Google's discovery documentation describes part of how pages are found as extracting "a link from a known page to a new page"; a sitemap doesn't settle the matter — the same docs state plainly that they don't guarantee every item in a sitemap gets crawled and indexed, and that crawling and indexing aren't guaranteed in general. What they say about links is unambiguous: if a site's pages are "properly linked," Google can usually discover most of it.

"Properly linked." My sitemap does declare every post, which is true and worth saying — but the same documentation is equally clear that a sitemap promises nothing about what gets crawled and indexed. The gap between declaring and linking is the one I can't close for 35.6% of my archive.

The traffic numbers were telling me this story already and I wasn't listening: in a thirty-day window, only six of 13,975 page views arrived as a referral from anywhere. In that post I was surprised the outside world wasn't linking to me. Measuring that my own posts don't link to each other took another eighteen days.

908 tags lead to a single post

In theory tags fill exactly this gap; in the engine's scoring, every shared tag is a point. So I counted them too: 1,356 distinct tags for 1,336 posts. More tags than posts.

Worse: 908 of those tags appear in exactly one post. Two thirds of my tag pages are shelves holding a single item. The system meant to cluster things produced 908 dead ends. They're useless to the scoring as well: a tag used once can never give any pair a shared point.

That's why I didn't invent fresh tags for this post. olcum, blog, teknik-borc, uretkenlik, life — all of them already in use across the archive. Minting a new one would have added another line to the very thing I'm complaining about.

The arithmetic of going back

Saying "from now on I'll link to older posts" is easy. Doing the arithmetic is sobering.

The 37 posts published in October produced 80 edges. Of those, 11 reach pre-August territory, the genuinely orphaned zone. Eleven old targets in thirteen days: 0.85 a day.

There are 1,009 pre-August orphans. At that rate: 1,009 ÷ (11/13) ≈ 1,192 days, roughly 3.3 years.

That estimate rests on three assumptions, all generous to me: that the pace holds, that every new link lands on a post never touched before, and that the pool stops growing. The second doesn't happen in practice — links cluster, returning to the same strong post again and again. Neither does the third: three posts a day keep arriving, and 17.7% of September's are already named nowhere. So 3.3 years is a floor, not a forecast.

Which means the archive can't be repaired by the habit that produced it. A pipeline writing three posts a day rescues one old post a day. The debt grows faster than the interest payments.

Two odd findings

Two things snagged my attention in the data; both are byproducts of the measurement.

Three links travel forward in time. A post published in May links to a post published in September. My first reaction was "measurement error." It wasn't: git blame shows that line was added on 4 September 2026 in commit e1817fd9 — while fixing a stale overlay2 claim in the old post, I dropped a link to the new one inside it. So the archive isn't frozen; going back and linking is possible, and I've done it three times. Three. A modest number next to 1,136 orphans, but not zero — and it points in precisely the direction that rescues an archive: linking the old to the new, not the new to the old.

One link is broken. One of the 301 edges points at a target that doesn't exist: a career post links to a "transactional outbox" post under tutorials that was never written. It isn't in the repository, and curl returns 404 on the live site. Most likely a topic that was scheduled, then rejected; the post shipped, its target didn't.

The irritating part isn't the ratio (1 in 301 is survivable), it's that nothing caught it. My pre-publication gates check external sources by HTTP status and don't look at internal links at all. While triple-locking the front door, I skipped the doors inside the house.

What I'm changing

The measurement wrote two of these decisions for me, and talked me out of the third:

  1. An internal-link check in the publication gate. Does every ](/blog/...) target exist in the repository? A trivial check. I'd like the broken one to stay today's only example.
  2. Changing the number I watch. Until now I asked "did I link out?" — the answer in October is 100%. The real question is "is anything linking in?" and there the orphan rate is 85%. Inbound counts, not outbound.
  3. No bulk retagging. That was the decision I sat down to announce; the simulation brought 537 down to 515 and no further. The cheap lever turned out to have no handle.

The fourth is the thing I opened this post swearing I wouldn't do. I wasn't going to break the "Related Posts" engine for the sake of diversity, because adding randomness means showing readers irrelevant cards. But randomness was never the point. I left two of the three cards exactly as they are and redefined the third: among candidates scoring above zero, the one shown least often across the site. Not random — completely deterministic. The simulation says:

Never named Most-shown post
Today's engine 537 (40.2%) on 102 pages
Third card reserved 0 on 96 pages

Five hundred and thirty-seven becomes zero. Every one of the 1,336 posts gets named on at least one page, and the most popular post appears on 96 pages instead of 102 — since the first two slots never change, the entire cost in recommendation quality is six pages for the post that needed them least.

Three years of linking, an afternoon of tagging: neither was necessary. What was necessary was reserving one card for whoever is heard least.

What's left

Production speed and connection speed are two different machines. I wired the first one to a cron job, a queue, a scheduled agent; it runs every day. The second depends on a rule, and the rule was written in August. In between sits an era of 1,106 posts, 91.2% of which stand somewhere nobody points to.

I'll admit I was watching the wrong number for a long time. I knew how many posts I'd written — 1,336, from memory. How many of them have something pointing at them I learned today: 200 by my own hand, 799 by the engine, 476 by nothing at all.

The real lesson is about diagnosis. First I thought "old posts get forgotten," and a three-year linking campaign stared back at me. Then I thought "their tags are thin," and it looked like an afternoon's work — until running it moved 537 to 515. Both were wrong, because both tried to fix the input, when the narrow thing was never the input. It was the three-card shelf itself. You can't solve a distribution problem by improving what you're distributing.

Writing is addition, and it scales. Connecting is editing, and it doesn't scale with a cron job — though sometimes it scales with a single rule. What turns my two-megabyte filing cabinet into a library isn't touching 1,336 posts one at a time. It's giving one of three cards to whoever is heard least.

Official Sources

Top comments (0)