DEV Community

Cover image for My calibration study found the GEO score doesn't predict AI citations. Here's the data
Juan Camilo Auriti
Juan Camilo Auriti

Posted on AI-assisted

My calibration study found the GEO score doesn't predict AI citations. Here's the data

I built a tool that scores websites 0 to 100 on how visible they are to AI answer engines. Then I ran a study to check whether the score predicts anything real. It does not.

The result: an odds ratio of 0.9625 with a p-value of 0.7172. No significant relationship between the GEO score and whether Perplexity cites a page. I pre-registered the analysis plan before running it, and I am publishing the data anyway.

What I tested

I took 341 pages from sites that publish a sitemap, ran a GEO audit on each one (rubric v2, GEO Optimizer 4.19.3), and then asked Perplexity Sonar two questions built from each page's own headline:

  • Template A: "What are the most reliable sources on: [headline]"
  • Template B: "Explain this topic and cite your sources: [headline]"

I recorded whether the page's domain appeared among the sources Perplexity returned. Then I ran a logistic regression of citation on score, controlling for site popularity (Tranco rank) and page length (word count).

The base rate was 42.23% for domain citation on template A and 44.57% on template B. So Perplexity cited the domain behind a page about 42% of the time when asked about that page's own topic. That is the baseline the score had to beat.

The result

The primary analysis was logistic regression of domain citation (template A) on score divided by 10, log rank, and log words.

  • Odds ratio per 10 score points: 0.9625 (95% CI 0.7827 to 1.1836)
  • p-value: 0.7172
  • n: 341

The odds ratio is below 1.0, meaning higher scores were slightly less likely to be cited, but the confidence interval crosses 1.0 comfortably. The effect is indistinguishable from noise.

The descriptive statistic tells the same story. The mean score of cited pages was 65.5. The mean score of not-cited pages was 65.5. Identical to one decimal place.

I tested every category too

The audit has 9 scoring categories: robots.txt access, llms.txt, schema markup, metadata, content structure, citation signals, brand entity, AI discovery, and Google AI readiness. I ran each one individually against citation, with the same controls and a Holm correction for multiple testing.

None of them were significant. Every single Holm-corrected p-value was 1.0000. The closest was the Google AI readiness category (p = 0.1220 before correction, OR = 0.8345), which trends in the wrong direction.

The sensitivity check

About 26% of the pages in the sample were on sites that block at least one AI bot in robots.txt. I re-ran the primary analysis excluding those pages, leaving 253.

  • Odds ratio per 10 score points: 1.0408 (95% CI 0.8302 to 1.3048)
  • p-value: 0.7288

Still nothing. The odds ratio flipped above 1.0 but the confidence interval is wide and the p-value is unchanged.

What this means

The GEO score measures whether a page is reachable, parseable, and quotable by an AI crawler. It checks robots.txt access, structured data, content structure, metadata, and AI-specific signals like llms.txt. It does a good job of that. The score correctly identifies pages that AI crawlers can read.

But being readable is not the same as being cited. Perplexity's source selection depends on factors the audit does not measure: domain authority, topical authority, third-party mentions, and whatever Perplexity's ranking model does internally. A page with a perfect GEO score and no external authority will not be cited. A page with a terrible GEO score but massive domain authority might be cited anyway, if the crawler can get past the robots.txt.

This is the gap between citability and citation. The first is a prerequisite. The second is an outcome that depends on more than the page itself.

Why I published a null result

I could have not run this study. I could have run it and not published it. Most AI visibility tools have never published a calibration study, and I suspect some of them would not survive one.

I published it because the alternative is worse. If I sell a score and do not test whether it predicts anything, I am doing the same thing as the agencies selling GEO tactics that the data does not support. The whole point of building this tool was to move past "trust me" and toward "check the data." That has to apply to the tool itself.

The pre-registration is at github.com/Auriti-Labs/geo-optimizer-skill/tree/main/research/calibration-2026-10. Its SHA-256 is 441fcbb422964fdf2bf2b63eb02181ccbea9e7c5590d3e770399d567cd373ff8. The results CSV has 341 rows with no domain, address, or headline. You can reproduce the primary result from the published data.

What I am doing about it

The score still measures something useful: whether AI crawlers can reach and understand your page. That is a necessary condition for being cited. It is not a sufficient one.

The problem is that a one-shot audit tells you where you are today. It does not tell you whether you are cited, and it does not tell you whether that changes. Perplexity cited the same domain about 42% of the time in this study, but other research (the Babel42 AI Overviews study, October 2026) found that only 11% of Google AI Overviews cited the same sites two days in a row. Citation is volatile. An audit is a snapshot.

That is why I am building the monitoring layer. The audit tells you if your page is readable. The monitor tells you if it is being cited, and whether that changed since yesterday. The first is a checkbox. The second is the metric that matters.

If you want to run the audit on your own site, it is free and open source at github.com/Auriti-Labs/geo-optimizer-skill. You can also paste a URL into the free web audit on geoready.dev without installing anything. Same engine, no signup.

Top comments (0)