Update: one page got indexed in 2 days, but my core tool pages are still "never crawled" after 11 weeks

TechEnthu — Sat, 24 Oct 2026 14:22:38 +0000

A while ago I wrote about why Google ignored my React SPA tool site and what I changed. I said I'd post an update and be straight about it if things didn't improve. So here it is: it worked for one page and did nothing for the two that matter most.

Some context: PinFetcher is an independent Pinterest downloader suite (not affiliated with Pinterest). I bought the domain as an expired one in August 2026, so it isn't a blank slate. A previous owner ran a WordPress site there, and Google still remembers some of those old URLs.

What happened

Media Saver, a page I added in October, was indexed within 2 days of my submitting it in URL Inspection.
Pinterest Video Downloader and the image downloader, my two core tools, still show Last crawl: N/A in URL Inspection. That means Googlebot has not fetched them even once. I've requested indexing two or three times a month. Every time: "Indexing requested". Nothing changes.

What I verified on the stuck pages

The same checklist as before:

The live test says "URL is available to Google".
The rendered HTML has the full content (about 1,700 words), a self-referencing canonical, index, follow, and no noindex.
Both pages are in the sitemap and linked from the header on every page.

A quick way to check the raw HTML of any page:

curl -s -o /dev/null -w "%{http_code}\n" https://your-site.com/your-page
curl -s https://your-site.com/your-page | grep -oE '<link rel="canonical"[^>]*>|<meta name="robots"[^>]*>'

What Crawl Stats showed

Google is visiting: 655 requests since Aug 5, so it isn't blocked.
But 88% of them were "Refresh" (re-checking URLs it already knows) and only 12% were "Discovery". Google rarely finds new URLs on a young domain.
38 requests were 404s on paths I never created (/pl, /he, feed URLs). They're leftovers from the previous owner's site, and Google keeps rechecking them.

The Links report gave me a lesson

I had 8 external links from directories and profiles. All 8 point to the homepage. None points to a tool page. I'd built a handful of listings, and every one sent trust to a single URL. If you want a specific page to be found, link to that page.

What I can't explain

I don't know why Media Saver was fast and the older pages weren't. It's newer, I submitted it, and it has a lot of text, but I can't prove any of that was the reason, so I won't pretend I can.

What I'm doing now

Adding deep links to the two tool pages in my directory listings, my YouTube channel and this article.
No more Request Indexing on those two pages. It hasn't helped.
Checking back in a week and posting the result either way.

Takeaways

A clean live test doesn't guarantee a crawl. It only rules out a bug on your side.
Check the Links report: do your external links point at the pages you want found?
Crawl Stats shows how much of Google's visiting is discovery. If it's mostly refresh, new URLs will wait.
If you bought an expired domain, expect old URLs to show up in your logs for months.

If you've been through a "never crawled" phase on specific pages while others got indexed, what finally changed it for you?

Why Google ignored my React SPA tool site for weeks (and what I changed)

TechEnthu — Tue, 06 Oct 2026 13:39:02 +0000

I build small browser-based tools as a solo developer. My latest one, PinFetcher, is a Pinterest downloader suite built as a single-page app. The tools worked fine. Google, however, barely noticed the site existed.

Weeks in, Search Console showed the same picture: a handful of URLs indexed and a couple dozen sitting at "Discovered - currently not indexed" with Last crawl: N/A. Not "crawled and rejected". Never even fetched.

Here's what I checked, what I changed, and what I'd tell myself on day one.

First, rule out the technical stuff

Before blaming "authority", I verified the boring things:

The pages return 200 and aren't blocked in robots.txt.
No stray noindex in the rendered HTML or headers.
The URL Inspection live test fetches the page fine.
No redirect loops. (I did find one broken route this way, and the live test reports it as "Redirect error", so always run the live test on every tool page.)

If all of that is clean and pages are still "Discovered - not indexed", the problem is usually crawl priority, not a bug.

The SPA-specific problem: links that only exist after JavaScript

In a client-rendered app, your navigation often only exists after React hydrates. Googlebot can render JS, but rendering is a second, deferred wave. If your homepage's raw HTML doesn't contain real links to your inner pages, you're making discovery depend on that second wave.

What I changed:

Real <a href> links in the raw HTML from the homepage to every tool and every blog post. Not onClick handlers or router-only navigation.
Core content in the initial HTML, so a page makes sense before any script runs.
Self-referencing canonicals on every page, so there's no ambiguity about the preferred URL.
Internal links between related tools, so no page is an orphan.

You can check your own site in ten seconds:

curl -s https://your-site.com | grep -o '<a [^>]*href="[^"]*"' | head -30

If that returns almost nothing, crawlers that don't execute JS see an empty navigation.

Sitemap hygiene that's easy to get wrong

Don't stamp every URL with the same <lastmod>. Mine were all identical from the day I generated the sitemap. Google learns to ignore lastmod if it's not trustworthy, so use real change dates.
Remove duplicates. I had a batch of short-URL variants listed alongside the canonical ones.
Keep it to canonical, indexable URLs only.

What did not help

Hammering "Request Indexing." It's rate-limited and doesn't override crawl priority. Submit once, then stop.
Repeated tiny site tweaks. Each change resets your patience, not Google's schedule. Make the fixes, then leave it alone.

What does move the needle: being linked to

A brand-new domain has no reason to be crawled often. The honest lever is other sites pointing at yours: genuine write-ups, directory listings, community posts, profile links. That's why I'm writing this post at all.

My current checklist:

[x] Raw-HTML internal linking
[x] Clean canonicals and sitemap
[x] About / Contact / Privacy / DMCA pages so the site looks like a real product
[ ] Earn external links to individual tool pages (in progress)

I'll post an update once Search Console moves, and I'll be straight about it if it doesn't.

Takeaways

Verify with the URL Inspection live test, not just the index report.
For SPAs, put crawlable <a href> links and core content in the raw HTML.
Keep your sitemap honest: real lastmod, canonical URLs only.
Stop poking Search Console and make the site worth linking to.

If you've shipped a client-rendered site and got through the "Discovered - currently not indexed" phase, what finally unstuck it for you? I'd like to hear it in the comments.

DEV Community: TechEnthu

Update: one page got indexed in 2 days, but my core tool pages are still "never crawled" after 11 weeks

What happened

What I verified on the stuck pages

What Crawl Stats showed

The Links report gave me a lesson

What I can't explain

What I'm doing now

Takeaways

Why Google ignored my React SPA tool site for weeks (and what I changed)

First, rule out the technical stuff

The SPA-specific problem: links that only exist after JavaScript

Sitemap hygiene that's easy to get wrong

What did not help

What does move the needle: being linked to

Takeaways