Our site claimed four different template counts at the same time.
- 86, in 34 article files
- 82, in nine places including the pricing constants
- "100+", on a landing page
- 96, in the marketing kit
The real number was none of them.
This had happened before on the same codebase: the site once advertised "200+" against
roughly 80, which had to be corrected as an honesty problem rather than a typo. A number
repeated by hand in forty-five places will always drift. The only question is whether you
find out or your customer does.
So we wrote a script that computes the real number and greps the codebase for anything
claiming a different one.
Version one worked, and lied to us
The script counted templates from the config, scanned src, index.html and a couple of
content directories for \d{2,3}\+? templates, and compared.
It printed "All template counts match."
Meanwhile the live page title said "96 ATS-Ready Designs" and the pricing description said
"92 premium templates". Both wrong. Both invisible to the checker, for two separate
reasons.
Reason one: it scanned the wrong directories
The strings a search engine actually displays were not in src. They were hardcoded in a
build script that injects static meta tags, scripts/prerender-head.mjs. The checker's
root list did not include scripts.
And even after adding it, the file extension filter was:
if (!/\.(ts|tsx|html|txt|md|json)$/.test(p)) return;
No mjs. The build scripts are all .mjs. It would have walked into the directory and
skipped every file in it.
Reason two: the pattern assumed a word that was not there
const CLAIM = /(\d{2,3})\+?\s+(?:premium\s+)?templates/gi;
"96 ATS-Ready Designs" never says "templates". The most visible claim on the site (the
<title> element, the thing shown in search results) could not match by construction.
Two patterns fixed it:
const CLAIM = /(\d{2,3})\+?\s+(?:premium\s+)?templates|(\d{2,3})\+?\s+ATS-Ready/gi;
Then it flagged itself
New run, and the script reported mismatches: 86, 82, 100, 96.
Those numbers were in its own docstring. We had written a comment explaining the drift the
script exists to catch, listing every historical wrong value as an example. The scanner
walked its own source and dutifully reported the description of the bug as the bug.
if (path.resolve(p) === fileURLToPath(import.meta.url)) return;
Genuinely funny, and a real category: a checker that scans source files will eventually
scan itself. Any tool with example strings in its comments has this problem. Linters
solve it with ignore directives; a small script just needs one line.
The general shape
Three failures, one theme, and none of them were the matching logic:
- Scanning the wrong scope. The claim lived outside the searched roots.
- An extension filter that excluded the file type. Silent, since a skipped file looks identical to a file with no matches.
- A pattern that assumed phrasing. Marketing copy varies the noun. Code does not.
Every one produced a confident pass. That is the part worth sitting with. A checker
that fails loudly is a minor annoyance. A checker that passes while the thing it checks is
broken is worse than no checker, because it converts an open question into a settled one.
Worth doing anyway
None of this argues against the script. It found four contradictory numbers on the first
real run and now runs in seconds.
The lesson is narrower: when you write a verifier, verify the verifier. Feed it a known
bad value and confirm it screams. We did not, and it told us everything was fine for as long
as we were willing to believe it.
If your product has a number in its marketing (templates, integrations, users, countries)
it is in more places than you think, and at least one of them is already wrong. Ours was
running on CVBooster, and it took a script to find out how wrong.
# start here, then check where the string actually lives
grep -rn --include='*.*' -E '[0-9]{2,3}\+? (templates|integrations|users)' . | grep -v node_modules
Top comments (0)