DEV Community

Saul Fleischman
Saul Fleischman

Posted on

June GEO Study Results: A 30-Day Controlled Measurement Across 6 LLMs

Most founders who talk about "AI visibility" are guessing. I was too, until I decided to stop guessing and start measuring.

The Problem With Vibes-Based GEO

Everyone in B2B SaaS right now has a take on generative engine optimization. The takes range from "just add more FAQ content" to "structure your schema correctly and the LLMs will love you." What almost nobody has is a controlled dataset with actual conversation counts, actual assistants, and actual recommendation rates tracked over time. That gap bothered me enough that I built a study around it. This post covers what I found at Day 0, why the variance across assistants surprised me, and what I think it means for any SaaS company trying to show up when a buyer asks an AI tool for a recommendation.

Quick context: MentionFox is a B2B platform that sits at the intersection of social listening, lead generation, AI-visibility tracking, and investor research. Our buyers are the kind of people who care deeply about where their brand appears - not just on Google, but inside the responses that Perplexity, ChatGPT, Gemini, and the rest of the field are now handing to their own customers and prospects. So GEO is not an academic interest for us. It is the product.

What We Actually Measured

On May 1, 2026 - Day 0 of the study - we ran 853 completed conversations across five AI assistants: Perplexity, Mistral, ChatGPT-4o, Gemini Flash, and DeepSeek. Each conversation was a structured query designed to simulate a real buyer moment. Think: a head of marketing at a mid-size SaaS company asking an AI assistant to recommend tools for tracking brand mentions, monitoring competitor activity, or identifying high-intent leads from social signals. We logged whether MentionFox was recommended, in what position, and with what framing.

The overall recommendation rate across all 853 conversations was 83.1%. That number sounds high, and honestly it is higher than I expected. But the variance underneath it is where the real story lives.

Per assistant, the numbers broke down like this:

  • Perplexity: 95.3%
  • Mistral: 83.6%
  • ChatGPT-4o: 80.1%
  • Gemini Flash: 78.9%
  • DeepSeek: 77.5%

Perplexity at 95.3% is nearly a lock. DeepSeek at 77.5% means roughly one in four conversations where a buyer could have found us - did not. That spread is not noise. It is signal about how differently these models weight sources, recency, and third-party validation.

Why the Variance Matters More Than the Average

The instinct when you see an 83.1% headline number is to feel good and move on. I forced myself not to do that, because the average hides the real competitive risk. If your buyers skew toward Gemini users - and in certain verticals they do - your effective visibility is closer to 78% than 83%. If your category is getting evaluated inside DeepSeek by a segment of the market you care about, you are invisible to nearly one in four of those conversations on Day 0.

What drives the variance? I have hypotheses, not certainties yet. Perplexity weights real-time web retrieval heavily and rewards brands that appear in recent, structured, third-party sources. Mistral appears to weight training data consistency - brands that have shown up in coherent, repeated contexts across its corpus. ChatGPT-4o and Gemini Flash seem more sensitive to how a brand is framed in relation to category terms. DeepSeek is the most opaque, but my working theory is that it underweights English-language SaaS-specific review ecosystems like G2 and Capterra, which is where a lot of our third-party signal lives. We will know more at Day 30.

What I Got Wrong Before Measuring

Before this study, I operated on the assumption that GEO was mostly a content problem. Write good content, get cited in good publications, structure it correctly, and the models will find you. That assumption is not wrong, but it is incomplete in ways that cost you if you do not know about them.

The assistant-level variance tells me that GEO is actually a source-portfolio problem. Different models draw from different pools of evidence. Winning on Perplexity requires a different signal mix than winning on DeepSeek. If you optimize for one and ignore the others, you are essentially doing SEO while only caring about one search engine - which was a reasonable strategy in 2004 and is a fragile strategy now.

The other thing I got wrong: I assumed our recommendation rate would be relatively stable across query types. It is not. Queries that frame the use case around "social listening for B2B" pull higher rates than queries framed around "lead generation from social media." That tells me the category language we have invested in is not evenly distributed across the way buyers actually phrase their problems. Closing that gap is a content and PR strategy, not just a product strategy.

What We Are Doing About It

Between Day 0 and the Day 30 checkpoint, we are running a set of interventions that are specific enough to be worth naming. First, we are increasing structured placements in vertically relevant publications that Perplexity's crawler prioritizes - not generic tech press, but outlets that cover B2B marketing operations and sales intelligence. Second, we are working to normalize our presence in non-English language review ecosystems, specifically to test whether that moves the DeepSeek number. Third, we are expanding the query taxonomy we test against - from 12 query types at Day 0 to 28 at Day 30 - to get a cleaner picture of where the language gaps are.

The GEO study dashboard will show the Day 30 results publicly when they land. I am not going to predict what happens to the overall number, because the honest answer is I do not know. What I expect is that the variance across assistants will narrow if our source-portfolio work is landing, and that the query-type breakdown will reveal something specific enough to act on.

What This Means If You Are Not MentionFox

If you are a SaaS founder or a B2B marketer reading this, the practical takeaway is simpler than the methodology. You need to measure your own AI recommendation rate, and you need to do it broken out by assistant - not as a blended average. A blended average will tell you things are fine when one or two models that matter to your buyers have almost no idea you exist.

The measurement does not have to be as structured as what we did. Start with 50 conversations per assistant, use query language that mirrors how your actual buyers describe their problems, and log whether your brand appears. That baseline is enough to tell you whether you have a GEO problem or not. If you do, the fix is almost always upstream of your website - it is in the citation graph, the review ecosystem, and the publication record that models use to form their opinions about your category.

One More Honest Note

I am aware that publishing a study where MentionFox does well on the metric MentionFox tracks has an obvious credibility problem. I have tried to counter that by publishing the full methodology and the raw conversation logs in the dashboard linked above, and by committing to publishing the Day 30 results regardless of whether they show improvement, regression, or no movement. If the number goes down, I will write about that too. The point is not to market with data. The point is to actually understand what is happening inside the systems that are increasingly mediating the first moment a buyer encounters a new tool.

If you want to see how MentionFox handles AI-visibility tracking and GEO measurement across assistants, the relevant page is the GEO study dashboard. And if you are evaluating whether the platform makes sense for your team, here is MentionFox pricing.


If you found this useful, I write about solo-founder distribution, B2B SaaS, and what's actually working in the AI-search era over on my Substack (one post per week, no spam).

I'm building MentionFox - a B2B intelligence suite that combines brand mention tracking with AI-visibility (GEO) measurement, investor research, and outreach automation. There's a free tier and a 5-day trial of Pro at mentionfox.com/pricing.

Top comments (0)