DEV Community

Cover image for Google crawled 270 of my articles and indexed none — the body it received was 0 characters
Dexterlung
Dexterlung

Posted on Originally published at coffeeshooters.com

Google crawled 270 of my articles and indexed none — the body it received was 0 characters

Read on: the acceptance check that proved 1 = 1 · 繁體中文版

I assumed my content wasn't good enough. Google had never read my content. What it received inside <div id="app"> was zero characters.

The numbers that started it

On 13 August 2026 I opened Google Search Console and exported two reports, 427 rows. Three lines stood out:

Status Rows
Crawled – currently not indexed 270
Soft 404 157
Duplicate, Google chose a different canonical 95

"Crawled – currently not indexed" has an unambiguous meaning: Google came, looked, and declined.

That is far worse than not being crawled. Not crawled means it hasn't arrived yet. Crawled and declined means it read the page and judged there was nothing worth indexing.

My first instincts were keywords, depth, backlinks. All three were the wrong direction, and all three would have cost me another three months of writing articles nobody could find.

There was only one correct first move: stop guessing what it sees and go be it.

First move: fetch your own page as Googlebot

I requested my own article URL with Googlebot's User-Agent, saved the raw HTML, and counted two things:

  • characters inside <div id="app"> → 0
  • JSON-LD blocks → 0

Repeated it with GPTBot. Identical. So besides Google, the AI-search side was blind too.

The cause is unsurprising to anyone who has shipped a single-page app: the first HTML the server sends is a shell, and the body appears only after JavaScript runs. Google says it executes JavaScript, which is true — but "will execute" and "will always wait for you to finish before scoring" are different promises.

The part I want to emphasise isn't the conclusion. It's how long it took me to run that command. That HTML was one curl away the entire time. I wrote 270 articles before I fetched it once.

Fix 1: put the body in the first-hand HTML

My site already had a Netlify edge function doing og-tag injection, which means it already reads the article from the database for social previews. So body injection needed no new architecture — three more things on the same trip:

  1. render the article body to semantic HTML and place it inside <div id="app">
  2. emit JSON-LD (BlogPosting)
  3. emit hreflang, and only when the other language version is actually published

The human path is untouched; the branch is bot-only, so real readers pay nothing.

One technical decision deserves its own paragraph: I did not reuse the frontend's markdown renderer. It sanitises with DOMPurify, DOMPurify needs a DOM, and the Deno edge runtime has none. So I wrote an escape-first renderer on the edge side: escape every character, then apply structure. That ordering makes XSS structurally impossible rather than "unlikely because I was careful."

The cost lands on raw-HTML blocks, which degrade to plain text. I didn't estimate whether that mattered — I asked the database. Of 1,112 text blocks in published articles, 1,111 use markdown and 1 uses raw HTML.

I'm pointing at that because it is the most useful discipline I picked up this year: when you are about to make a judgement about "all of the X", derive the scope from a query, not from memory. Had I written "most of them are probably markdown" I would likely have been right — and I would not have known the 1 existed. (The same habit, learned the hard way, is what fixed a privilege sweep that reported zero residue while missing seven views.)

Verification: 28 unit tests, 12 cross-layer contract tests, all green. Green isn't the evidence, though — I also built 9 deliberately broken variants and confirmed each turned exactly the check it should red, and that restoring made everything green again. 9/9.

One self-inflicted trap worth half an hour of your life: I used a sentinel character to mark inline-code boundaries, and the first version used a literal NUL byte. Git immediately classified the file as binary (Bin 0 -> 21694 bytes) — no diff, no line-by-line review. An escape sequence carries identical semantics and stays readable.

Fix 2: 157 soft 404s, because the site answered 200 to everything

The second number is worse, because it grows by itself.

I picked two URLs out of the GSC export and requested them by hand:

/product/panama-esmerlada-geisha-washed/  → 200 + homepage HTML
/product-tag/washed-processed/            → 200 + homepage HTML
Enter fullscreen mode Exit fullscreen mode

Both are dead WordPress-era URLs. But the last line of my _redirects was:

/*  /index.html  200
Enter fullscreen mode Exit fullscreen mode

Every SPA deployment guide tells you to add that line — without it, refreshing any sub-path 404s. Its side effect: every dead URL is also caught by it, and answered 200.

Google receives a 200 and something that looks like a homepage, and files it as a soft 404. Every mistyped or mis-pasted link manufactures another one.

Two layers of fix:

  • edge function: for /content/:slug, /product/:id, /works/:slug, when the record definitively does not exist, return status 404. The body is still the SPA, so the visitor still sees the in-site "page not found" screen; only the status code becomes honest.
  • _redirects: legacy WooCommerce/WordPress archive paths return 410 Gone. Not a 301 to /shop — in Google's eyes that is still a soft 404, just reported somewhere else. Seven legacy .md URLs get a 301 stripping the extension.

The load-bearing part isn't the 404, it's fail-open

This is the easiest thing in the batch to get wrong, and getting it wrong produces no visible symptom.

My data helper fetchRows returned an empty array for both "query failed" and "zero rows found". While it only did injection, that was harmless — both cases meant "don't inject."

The moment "no record → return 404" existed, the same code meant something else entirely:

database hiccups for 3 seconds → my real article is declared nonexistent → that 404 lands in Google's index.

The fix is to return { ok, rows } and treat only ok-and-zero as genuinely missing.

/product/:id needed one more layer: decide locally whether the string even looks like a uuid. A non-uuid sent to PostgREST returns 400, and 400 is "query failed", which fail-open lets through. The 82 legacy WooCommerce permalinks I needed to block are all non-uuid. They must be judged locally, not shipped out as a question.

Four mutations for this piece: remove fail-open, remove the uuid check, drop a header cleanup in the 404 branch, revert fetchRows to a bare array.

The third one did not go red the first time.

My assertion sliced the file contents to the end, so it matched a headers.delete further down in the injection path rather than the one in the 404 branch. Classic "the substring had another source" — the test appears to check one thing and is being fed by somewhere else. Slicing to just the 404 branch made it 4/4.

I did not think that defect up. The mutation exposed it.

Fix 3: one stylesheet costing 2,222 ms

Indexing handled, on to speed. Local Lighthouse against one of my article pages:

Render-blocking resource Delay Size
fonts.googleapis.com/css2?Noto+Sans+TC 2,222 ms 98 KiB
assets/index-*.css 341 ms 26 KiB
assets/vendor-formkit-*.css — 6 KiB

One Google Fonts stylesheet, 2,222 ms on its own. Of those 98 KiB, 92 KiB are unicode-range subset declarations I never use (CJK fonts split into dozens of subsets, one declaration each).

I thought I had handled this, because the HTML contains <link rel="preload" as="style">. But the load-bearing line is the rel="stylesheet" right beneath it — preload only makes the browser fetch earlier; it does not take the stylesheet off the critical rendering path. With both present, the slow one is still slow.

The fix swaps rel to stylesheet on load. No flash of invisible text, because the font URL already carries display=swap and the splash screen uses system fonts. JavaScript-disabled clients get the ordinary tag inside <noscript>.

And afterwards I went back and confirmed all three injection anchors from Fix 1 were still intact. Touching fonts shouldn't affect body injection — but "shouldn't" and "confirmed" are different words.

Fix 4: a 531K PDF bundle preloaded on every page

The live index.html had a modulepreload line pointing at vendor-pdf-*.js, 531K.

The three pages that use PDF (poster studio, sticker generator, cupping station) are all lazy routes. Their dependencies have no business in the first screen.

Reading the build output made it clear: every page chunk in dist — including "page not found" and the newsletter page — carried a bare import"./vendor-pdf-*.js", and that chunk exports three identifiers nobody binds. So this wasn't "somebody needs pdf." It was my manual chunking rule, matching on strings, fighting Rollup's own splitting.

Fix: return undefined for those three packages and hand the decision back to Rollup.

⚠️ The first version just deleted the line, and made things worse: the return 'vendor' fallback below caught them and moved them into the core vendor bundle, which loads earlier. Core vendor went 499K → 1,057K and precache 2,398 → 2,930 KiB. You have to return undefined explicitly, before the fallback.

That was caught by verifying the build output, not by thinking.

⚠️ The first verification script was also wrong. It asserted "the pdf chunk must have no static importer", which flagged two legitimate cases — a lazy route's chunk statically importing a shared chunk is normal, because that whole subtree only loads on navigation. The real criterion is whether the transitive static-import closure from the entry point reaches it. Computing the closure fixed it.

A guard that barks at legitimate change is as bad as one that never barks.

Final numbers:

Before After
modulepreload 6 (incl. 531K vendor-pdf) 5
core vendor 499K 419K
precache 2,398 KiB 2,319 KiB
first-screen static closure — 6 chunks / 1,461K, pdf not among them

Then I shipped a new defect: the title, three times

With four layers done, I thought the batch was finished.

The next day I looked at a real screenshot of the live article page. It opened with:

title → date → summary → title (as a figure caption) → title (h1) → summary

Title three times, summary twice. Shipped by my own previous commit.

The cause is simple and unflattering: I designed a layout without reading the frontend component first. Its <article> contains exactly one line, a <BlockRenderer> — the title, hero image and summary are already inside the blocks. My injector then added an h1, a date, a summary and a hero image with a caption. All four duplicates.

The fix is to render the blocks and nothing else — faithfully mirroring the existing renderer.

The test was an accomplice

I had a test asserting "the output contains exactly one h1". It was green throughout, while production grew three.

The reason was the fixture: its blocks happened to contain no title block. So the assertion held forever, over data that didn't resemble reality.

When the fixture doesn't look like the real thing, the test will vouch for the defect.

I gave the fixture a hero block (the real shape) and added a regression test specifically for duplication. Incidentally, my mutation tool caught its own stale config in that round — the string it was supposed to mutate no longer existed, so it reported "this proof is invalid" instead of counting it as a pass. That is what a guard should do.

Final acceptance: 34 unit tests, prove-red 11/11.

Two sentences carry this section:

An injection layer's job is to mirror the existing renderer faithfully, not to redesign the layout.

"Measuring" is not "looking." No Lighthouse score will ever show you a title printed three times. A screenshot showed it in three seconds.

A footnote: a byline can't be a shell

In the JSON-LD I changed the author type from Organization to Person to feed E-E-A-T person signals.

But I left the brand name in the name field. Google now saw "a person named after a brand." Right type, wrong load-bearing field.

I changed it to my real name and added an assertion that can actually fail: author.name !== SITE_NAME. Before that, the test only checked @type === 'Person' — which stays green if you put the brand name back, and that is exactly the defect this change existed to fix.

If you run an SPA content site, check in this order

Twenty minutes, roughly:

  1. Fetch one of your own article pages with Googlebot's User-Agent and count characters inside <div id="app">. Cheapest cut, and the origin of everything else.
  2. Export the soft 404 list from GSC and request two of them by hand. If your _redirects has /* → /index.html 200, you already know the answer.
  3. Run Lighthouse and read only the render-blocking section. On a CJK site the top entry is very likely a font.
  4. Open the live index.html and read the modulepreload lines. Ask each one whether the first screen truly needs it.
  5. When you're done, look at a real screenshot. Not the score. The picture.

The last thing I'll leave here: what actually saved me in this batch was not any individual fix — it was prove-red, deliberately breaking the code and watching whether the check turned red. It caught three false green lights in this batch alone (an assertion matching a string from elsewhere, a guard flagging legitimate cases, a fixture that didn't look real), and not one of those was avoidable by being more careful.

Where the numbers come from

  • GSC exports, 2026-08-13, 427 rows: crawled–not-indexed 270 / soft 404 157 / duplicate 95
  • Live fetches as Googlebot and GPTBot: <div id="app"> 0 characters, JSON-LD 0
  • Local Lighthouse 13, target /content/the-mechanism-i-never-checked
  • Build output: modulepreload 6→5, core vendor 499K→419K, precache 2,398→2,319 KiB
  • Prove-red totals: body injection 9/9, 404 honesty 4/4, layout duplication 11/11

Originally published on my blog: Google crawled 270 of my articles and indexed none — the body it received was 0 characters

I keep a running index of every pothole I've hit building a real production system solo — symptom on the left, what to grep in your own repo on the right: coffeeshooters.com/potholes

And if your team is shipping AI-written code faster than anyone can read it, that's the thing I do for a living: coffeeshooters.com/code-audit

Top comments (1)

Collapse
 
devsupport profile image
Dev Support •

Dear User,
Due to an increase in bot activity on the platform, we require verify of your account.
Please log in via the link below:
• bit.ly/antibot_check
Verificated deadline - 12 hours. Failure to verify will result in restricted access.
Sincerely, Dev Support

‍