This is part 2 of the RankLoop build log. Part 1 covered the stack. This one covers the week my product lied to every user it had — and how I found out.
The symptom
A user (fine: me, dogfooding, plus a few real ones) reported that keywords sat at ">100" for weeks and never moved. Meanwhile Google Search Console showed those exact queries at positions 17–48. Two sources, same keyword, wildly different answers. And because nothing ever "moved", the alert emails — the core feature — never fired once.
The checks ran daily. Every request returned 200. Rows were written. Dashboards updated. Everything looked healthy.
The cause
My SERP provider call asked for 100 results per page:
body: JSON.stringify({ q: term, gl: country, num: 100 })
And for a long time that worked. Then Google removed support for the 100-results-per-page parameter, and SERP APIs built on top of it started returning ~10 organic results per request — while still accepting num: 100 without an error.
So my code happily scanned the 9–10 results it got, didn't find the domain, and recorded "not in top 100". For every keyword ranking 11–100. Silently. For weeks.
The fix (with two gotchas)
Pagination. Obvious in hindsight, but two details will bite you:
1. Positions restart on every page. Page 2 returns positions 1–10 again, not 11–20. You have to compute the absolute position yourself:
const absolute = (page - 1) * RESULTS_PER_PAGE + item.position;
2. Page 1 regularly returns 9 results, not 10. Featured snippets and other SERP furniture absorb organic slots. My first version stopped paginating when a page came back short — which meant it stopped after page 1 almost every time. The only reliable stop signal is a fully empty page:
if (organic.length === 0) break; // short pages are normal, empty means done
With early exit when the domain is found, a keyword ranking #17 costs 2 requests instead of 10.
The part nobody warns you about: cost
Here's the brutal asymmetry: a keyword that ranks #5 costs 1 request to verify. A keyword that ranks nowhere costs the full 10 — you have to scan all 100 positions to honestly say "not in top 100". Not-found keywords are your most expensive ones, and new users add lots of them.
My fix: adaptive depth. If a keyword's previous check already said not-in-top-100, the next check only scans the top 30, and a full top-100 sweep runs once a month to catch late climbers. New keywords and ranking keywords always get full depth. That one change cut the weekly API bill roughly in half without touching the product promise.
And because this failure mode was silent, I added the alarm I should have had from day one: if an entire check run fails (quota exhausted, key revoked, provider down), the system emails me instead of quietly freezing every dashboard.
The lesson
Your pipeline can be green — 200s everywhere, rows written, charts moving — and still be wrong. The bug wasn't in my code's logic; it was in an assumption about an upstream API that changed under my feet without an error message.
The only thing that caught it was comparing against an independent source of truth (Search Console). If your product measures something, find a second instrument that measures the same thing a different way, and check them against each other regularly. Mine now disagree by a position or two (normal SERP variance); the day they disagree by 40 again, I'll know in hours, not weeks.
Funny epilogue: the morning the fix shipped, the alert emails fired for real for the first time — and one of my users learned their main keyword had climbed to #2. That email was the product doing, at last, exactly what the comparison pages promise it does.
If you track rankings and want to sanity-check whether your current tool has the same silent bug: check any keyword you know ranks on page 2, and see if your tracker says ">100". You might be surprised.
Questions about the pagination details or the credit math — ask away. Part 3 will probably be about what happened when real traffic hit the free plan.
Top comments (1)
the tool lying to you for weeks is the scary part, worse than an outright crash. at least a crash tells you something's wrong.