I ran 1,790 measurements across five AI engines — for my own company and for a devtools category with real money in it. Most of what "AI visibility" dashboards report turned out to be an artifact. Here are the three mirages, and the four numbers worth tracking instead.
The buyer shift is real. G2's latest survey has 51% of B2B software buyers starting vendor research in AI chatbots more often than Google, and chatbots are now the single biggest influence on shortlists. Ahrefs measured what happens on the other side: when an AI Overview shows up, clicks to the #1 organic result drop by more than half.
So now 88% of marketers say they're optimizing for AI search. Ask how they're measuring it and the room gets quiet — most brands don't track AI-search performance at all. The GEO subreddits are full of agency owners asking each other what to even put in a monthly client report.
I've spent the past six weeks measuring instead of asking. Two audits: one on my own company (800 prompts across four engines, run twice, five weeks apart), one on the web-scraping category — Firecrawl, Apify, Bright Data, ScrapingBee, and 22 other vendors (198 buyer queries × 5 surfaces, 990 cells, validated against an independent 10,000-mention dataset).
The measuring was the easy part. The work was not fooling myself. Three mirages, in order of how much money they'll cost you.
Mirage #1: The branded-prompt artifact
My own dashboard told me GTM Labs was ranked #1 on every AI platform, with 35–37% visibility. A number like that goes straight into a board deck.
It was fiction. The mechanism matters, because your GEO tool probably has the same one built in.
My prompt set had 40 prompts. Sixteen of them mention my brand by name — comparison prompts ("GTM Labs vs Kalungi"), evaluation prompts ("is GTM Labs any good"). On those, the engines recognize the brand nearly every time: 58 out of 64 responses. That near-perfect recall score gets averaged together with everything else, and suddenly the aggregate says 35% and #1.
Strip out every prompt that feeds the model my name — leave only the questions a real buyer asks before they have a shortlist, like "best fractional PMM for devtools" — and my organic score is 0 out of 96.
Not 35% — zero. The engines know who I am when asked directly and never think of me unprompted. Those are two different problems, and the branded average hides the one that matters.
So the test for any AI-visibility report, whether it came from a $300/month tool or an agency: does it separate branded prompts from organic ones? If it doesn't, you're reading a recall test dressed up as a discovery measure, and recall was never the hard part.
One consolation from my zero: total organic mentions across all five competitors I track fell from 11 to 6 between runs. Nobody owns these queries yet. The floor is low and winnable — which is why measuring it honestly matters.
Mirage #2: The single-surface headline
The Firecrawl audit gave me my favorite wrong headline of the year, and I wrote it myself.
First pass: Firecrawl is the #1-cited vendor in web-data tooling. True — on Perplexity, at 25% share of citations, well ahead of Apify.
Then the independent dataset arrived. It's ~90% Google AI Overviews, and there Firecrawl is #3 at 20%, behind ScrapingBee at 31% and Bright Data at 28% — the incumbents with a decade of ranked content behind them. On ChatGPT search: a three-way tie for third, at 14%.
Same vendor. Same month. #1, #3, and #3, depending on which engine you ask.
The engines have different supply chains. Perplexity leans hard on what developers currently recommend to each other (Reddit, recent listicles). AI Overviews inherit a decade of Google rankings. ChatGPT search sits somewhere between. A one-number "AI visibility score" averages across sources that disagree with each other and tells you nothing you can act on.
When a tool or a report quotes you a single number, the question is always: which engine, and on which prompts? If the answer takes more than one sentence, the number was marketing.
Mirage #3: llms.txt
On June 1st I deployed the full on-site AI-optimization stack on my own site: llms.txt, llms-full.txt, JSON-LD on every service page, entity schema. The exact checklist every GEO thread recommends.
Thirty-four days later I re-measured. Organic visibility went from 1/96 to 0/96.
That was the honest prediction all along — a top comment in one of the GEO subreddits put it better than any vendor deck: everyone has implemented llms.txt; no crawlers are requesting it. My server logs agree.
Keep the file. It costs nothing, it's tidy, it may matter someday. But treat it as plumbing. The engines cite what other people say about you, and my on-site deploy was me telling the engines about myself. They were not interested. Fair enough — neither are buyers.
What actually moves citations
The Firecrawl data shows where the leverage lives, and it's old-school.
Firecrawl's citation gaps aren't random — 44 queries where a rival is cited and Firecrawl isn't, clustering into one coherent theme: hard-target scraping (anti-bot, Cloudflare, behind-login). On those, engines route buyers to proxy specialists or Playwright tutorials. Citation share on gap themes: 12.6%, versus 23.7% on home turf. The engines have compressed the product into one sentence — "URL-to-markdown for AI agents" — and they hand everything outside that sentence to someone else.
LLMs don't read your feature list. They compress you into one job. Whatever sentence the internet repeats about you most often is your product, as far as the engines are concerned.
And the root cause is almost boring: on 6 of the 9 gap keywords, Firecrawl is absent from Google page 1 entirely. The AI-citation gap mirrors the organic-search gap nearly one-to-one. Engines can't cite what doesn't rank.
The sources that do rank on the gap-theme results: Reddit (27 appearances), GitHub (12), Medium (9), and indie listicles — above any vendor's own blog in nearly every case. Your blog is not in the citation supply chain. The r/webscraping answer, the runnable GitHub cookbook, the third-party "best X for Y" listicle: that's the supply chain.
For all the noise about GEO being a new discipline, the machinery underneath is mostly the old one — rank, get referenced, be the thing communities recommend. What's new is that the interface stopped sending you the click, so the only thing left to win is the citation.
The Four Numbers
If you run a devtools company — or you're the first marketer at one, staring at a "track your AI visibility" vendor pitch — this is the monthly report I'd actually run. Four numbers, one discipline.
1. Organic citation share, per engine, unbranded prompts only.
Build a prompt set from questions buyers ask before they know you exist. Score each engine separately — you now know why. This is the number that tracks discovery, and it will be humbling. Mine is zero. I just published it anyway, which tells you how rare honest baselines are.
2. Branded recall rate — on its own line, never averaged in.
"Do the engines know you when named?" is worth tracking — misses here point to entity work (schema consistency, Wikidata, third-party profile completeness). It's the cheapest fix on this list. It just can't be allowed anywhere near number 1.
3. The gap list, weighted by CPC — not volume.
Every query where a competitor is cited and you aren't, ranked by what the click is worth. "Best web scraping API" gets 140 searches a month and looks like a rounding error in a volume report. It carries a $40 CPC — the market telling you exactly what a citation there is worth. Volume-sorting your gap list deprioritizes the only queries that pay.
4. The citation supply chain.
For your gap queries: which domains do the engines actually cite? That list is your content-placement roadmap. If it's Reddit, GitHub, and two listicles — and it will be — then that's where the work is, not on your own blog.
Plus one discipline that isn't a number: measure monthly, and respect small samples. At my scale, a single stochastic mention is the difference between "up 100%" and zero. A trend needs three points before it's a trend. Anyone selling you daily AI-visibility tracking is selling you noise with a subscription.
The kicker
The entire rig behind this piece — 198-query category audit across five surfaces, plus independent validation — costs about $15–30 per sweep in API calls, plus $2.15 for the validation dataset. The scripts are boring Python.
That's less than a day of most GEO tools' monthly price, and it produces the four numbers most of them don't show you — including the one that told me my own headline number was a mirage.
Your buyers moved into the chat window, and measuring whether you exist there is now part of the job. Just make sure you're measuring the moment before they know your name — because right now, that moment belongs to almost nobody, and it's for sale.
If you'd rather run it than read about it
I open-sourced the honest core of this method as a zero-dependency Python script. It scores organic (non-branded) visibility per engine, keeps branded recall on its own line, and prints the citation supply chain — the four numbers above, on your own machine, for a few cents a run:
github.com/dovzhikova/ai-visibility-scorer
python3 ai_visibility.py --brand "Your Product" --domain yourproduct.com \
--category "what your buyers actually shop for" \
--competitors "Rival:rival.com"
Want the moment-before number for your own product? Start with the free AI Visibility Checker — it takes two minutes. If the number stings, that's what the GTM Diagnostic is for.
Top comments (0)