I ran an SEO experiment on my side project and broke my own control group

Oleg Kolesnykov — Mon, 05 Oct 2026 13:27:53 +0000

In September I had a weird problem with my side project, a tarot card reference in four languages: https://tarot-magic.com. The Ukrainian card pages sat at positions 6-9 in Google and got almost no clicks. Over three months they had 2,614 impressions and 2 clicks.

On September 21 I rewrote all 78 Ukrainian meta descriptions. The old ones were keyword dumps cut off mid-sentence. The new ones are one plain sentence about what the page actually covers. I left the other three languages alone as a control and promised myself to wait two weeks before looking.

Two weeks are up. Here is what happened, including the part where I ruined the experiment myself.

The numbers

I compared two equal 14-day windows in Search Console, filtered to /uk/card/.

	Sep 7 - Sep 20	Sep 21 - Oct 4
Impressions	231	463
Clicks	0	2
Avg. position	11.2	21

Impressions doubled while average position got worse. That looks contradictory until you check the page list: Google started showing pages that had no impressions before, and they enter at positions 20-35. They drag the average down even though part of the existing pages held their positions.

If I keep only pages averaging position 10 or better, I get 157 impressions and 2 clicks, a CTR of 1.3%. The previous window under the same filter had 197 impressions and zero clicks.

1.3% is roughly what you would expect at positions 6-8 on mobile. But it is two clicks. I am not going to claim anything from two clicks.

How I broke it

Before starting I wrote down one rule: compare CTR only if positions stay stable, because CTR at position 7 and at position 30 are different metrics.

Positions did not stay stable. The single biggest Ukrainian page in the top 10, "Six of Wands", went from 7.4 to 37.5 in exactly this window.

Worse, on the same day the descriptions changed, an FAQ block was added to all 312 card pages in all four languages. It started as a six-page pilot and got rolled out everywhere at once. So the "control" languages were not a control anymore. Russian card pages dropped from 379 impressions to 133 in the same period, and I have no way to tell whether that was the FAQ, normal ranking churn for a young domain, or something else.

Lesson for next time: a control group must not be touched by anything, including changes that look harmless.

The accidental winner

The best result came from a page that was not part of the experiment. On September 20 the French "card of the day" page got the main local query, "Tarot du jour", added to its title. Over the next two weeks it got 18 impressions and 5 clicks. That is half of all clicks on the site for the period.

My current guess is that the title decides whether someone looks at the result at all, and the description only matters after that. That is the next thing I want to test, with a control group I will not touch this time.

Bonus: Yandex

Yandex had excluded 76 of 88 Russian pages as "low value". On October 1 I sent all Russian URLs for recrawl through Yandex Webmaster. Pages in search: 12 on Oct 1, 21 on Oct 2, 30 on Oct 5. No new links, no analytics changes, just a fresh visit from the crawler. The same update on Oct 5 dropped 20 English card pages with the same "low value" status, all last crawled around Sep 21-23. So if you work with Yandex, recrawl requests seem to matter more than I expected.

Next check is mid-October. If anyone has run title A/B tests on a small site, I would love to hear how many impressions you needed before the difference was real.

I prerendered my React SPA for SEO. I forgot the links.

Oleg Kolesnykov — Wed, 16 Sep 2026 12:41:43 +0000

My site had been live for a month. Google had indexed all 352 pages. Titles matched
search queries exactly. Structured data was in place. hreflang covered four locales.

Then I ran a site audit and got 24 errors, all the same:

Page has no outgoing links

The homepage. Every locale homepage. Every tool page. And — the one that actually
hurt — the hub page that is supposed to link to all 78 tarot cards.

That last one made no sense. I could open it in my browser and see 78 links right
there. So I did what I should have done much earlier:

curl -s https://tarot-magic.com/uk/card-meanings | grep -o '<a [^>]*href=' | wc -l
# 0

(Note the grep -o ... | wc -l — not grep -c. Most HTML ships minified on a
single line, so grep -c counts lines containing a match and cheerfully
reports 1 no matter how many links you have. I got that wrong the first time
and briefly convinced myself the fix hadn't worked.)

Zero. Then in the browser console, on the same page:

document.querySelectorAll('a[href]').length
// 56

Fifty-six links that existed for humans and did not exist for crawlers.

What I had actually built

The site is a React SPA. At some point I added a prerender step so crawlers would
get real content instead of an empty <div id="root">. It worked — page text was
in the HTML, word counts were healthy, pages ranked.

Here is what the prerendered body looked like:

<body>
  <noscript>You need to enable JavaScript to run this app.</noscript>
  <div id="root">
    <div class="pre-bg"><div class="pre-wrap">
      <h1>Tarot Card Meanings</h1>
      <p class="pre-lead">A complete encyclopedia of all 78 cards</p>
      <h2>Frequently asked questions</h2>
      <h3>How many cards are in a tarot deck?</h3>
      <p>A standard deck has 78 cards: 22 Major Arcana and 56 Minor Arcana…</p>
      ...

Text, headings, FAQ copy. All of it useful. And not a single <a href>.

I had solved "crawlers can't read my content" and never noticed I hadn't solved
"crawlers can't navigate my site."

The part that made it obvious

The same site has a blog. Articles are generated by a different code path, and
those pages looked like this:

<div class="pre-bg"><div class="pre-wrap"><div class="pre-article">
  <a href="/uk/">Home</a>
  <a href="/uk/blog">Journal</a>
  <a href="#the-element-of-fire">The Element of Fire</a>
  ...

26 links, 20 of them internal. Same wrapper, same mechanism — someone (me, months
earlier) had put links into the article template and not into anything else.

So this was never an architecture problem. The machinery was already there. One
template had links; the other twelve didn't.

Why it mattered more than I expected

Google renders JavaScript, so my links weren't invisible to it. But rendering
is a second, deferred pass. Google crawls the raw HTML first, builds a link graph
from it, and comes back to render later. For a month, the link graph Google built
from my site was: 352 pages, no edges.

Bing and Yandex barely render JS at all. Over the same period:

Search engine	Impressions
Google	10,820
Bing	16
Yandex	11

I had been telling myself that was a new-domain problem. Some of it was. But those
crawlers were also looking at a site where nothing linked to anything, discovering
pages only through sitemap.xml, and reasonably concluding there was no structure
worth indexing.

My average position across all queries was 23.9 — page three. Internal linking is
one of the two levers that move that number. I had turned it off without knowing.

The fix

No architecture change. Put links into the prerendered HTML:

Header and footer nav on every template — the global one, five minutes of real work, fixes every page at once.
The hub page lists its 78 cards as actual anchors — this is the page whose entire job is distributing link equity to 312 card pages across four locales.
Card pages link to their siblings — previously they had 3 links, all breadcrumbs.

Results, same day

I re-ran the audit a few hours after deploying:

	Before	After
Health score	95	100
Errors	24	0
"Only one incoming internal link"	36 pages	4
Internal links discovered	1,020	10,056
Slow pages	11	1

That last row was a surprise — I hadn't touched performance. My guess is the crawler
was spending its budget differently once it could actually traverse the site, but
I can't prove that, so take it as a coincidence until someone shows me otherwise.

Ranking effects are still pending. Google has to recrawl 312 pages and rebuild the
graph; that takes weeks, not hours. I'll know in a month whether it moved anything.

The check that would have saved me a month

If you prerender, statically generate, or SSR a JS app, run this before you trust it:

for path in "" "about" "products" "blog"; do
  printf '%-20s %s\n' "/$path" \
    "$(curl -s "https://yoursite.com/$path" | grep -o '<a [^>]*href=' | wc -l)"
done

Compare each number to document.querySelectorAll('a[href]').length in the browser
on the same URL. If the browser number is much bigger, your navigation is invisible
to anything that doesn't run JavaScript.

I checked my word counts religiously. I never checked my link counts. Word count
tells you a crawler can read the page. Link count tells you it can leave it.

The site in question is tarot-magic.com — a tarot
reference in four languages. If you want to see the check pass, run the command
above against /uk/card-meanings. It returns 105 now.

DEV Community: Oleg Kolesnykov