DEV Community

Cover image for I broke my own tool's scoring model and rebuilt it. Here's what changed in rubric v2
Juan Camilo Auriti
Juan Camilo Auriti

Posted on AI-assisted

I broke my own tool's scoring model and rebuilt it. Here's what changed in rubric v2

When I published the calibration study showing the GEO score does not predict citations, one question came up immediately: if the score does not predict citations, why did you just rebuild it?

The answer is that the score was never meant to predict citations. It measures citability, whether an AI crawler can reach, parse, and understand a page. The rebuild makes it better at that, and more honest about what it does not measure. Here is what changed.

The old rubric (v1)

The original rubric had 9 categories adding up to 100 points:

  • robots.txt and AI crawler access (10 pts)
  • llms.txt presence and validity (18 pts)
  • schema markup and structured data (10 pts)
  • metadata and Open Graph (8 pts)
  • content structure and readability (15 pts)
  • citation signals (quotable passages, statistics, quotes) (12 pts)
  • brand entity clarity (8 pts)
  • AI discovery signals (6 pts)
  • negative penalty for blocking, JS-only content, injection surfaces (13 pts deduction)

The problem was the weights. llms.txt carried 18 points, almost a fifth of the total. But Google's own guide, published in May 2026, says llms.txt does nothing for AI Overviews. The Babel42 study confirmed it: only 0.2% of sites have a valid llms.txt file, and AI Overviews cite those sites at the same rate as everyone else. The signal was over-weighted relative to its actual importance.

What changed in v2

Rubric v2 shipped in v4.19.0 (October 2026). Three structural changes:

1. llms.txt dropped from 18 to 6 points. The file is still checked. Some AI crawlers do read it, and 0.2% of sites use it. But it is not a top-3 signal for citability. Six points reflects that: present and valid gets you something, missing does not tank your score.

2. AI discovery dropped from 6 to 3 points. This category checked things like RSS feeds and sitemap references to AI-specific content. Useful but minor. Three points is honest.

3. New category: Google AI readiness (20 points). This is the biggest addition. It checks 8 signals specific to how Google's AI Overviews select and display sources:

  • G-INDEX: is the page indexed and canonicalized correctly
  • G-SNIPPET: does the page have a meta description and structured snippets
  • G-CANONICAL: is there a canonical URL and does it match
  • G-DATES: are there published and updated dates
  • G-BYLINE: is there a named author
  • G-LINKS: are internal links present and crawlable
  • G-VIEWPORT: is the viewport meta tag set (mobile readiness)
  • G-SITEMAP: is the page in a sitemap

These are the signals the WashU study (October 2026, 55,393 queries) found correlate with AI Overview citation: credible sources with clear authorship, dates, and canonical URLs. The category does not guarantee citation. It checks whether the technical prerequisites are in place.

The full v2 category list

Category v1 points v2 points Change
robots.txt access 10 10 unchanged
llms.txt 18 6 -12
schema markup 10 10 unchanged
metadata 8 8 unchanged
content structure 15 15 unchanged
citation signals 12 12 unchanged
brand entity 8 8 unchanged
AI discovery 6 3 -3
Google AI readiness 0 20 new
negative penalty -13 -13 unchanged

Total: still 100 points. The 15 points removed from llms.txt and AI discovery went into the new Google AI readiness category.

You can still use v1

If you have historical data on v1 scores, you do not need to re-audit everything. The CLI supports --score-version 1 for legacy comparisons. Drift and diff comparisons are gated to the same score version, so you cannot accidentally compare a v1 score with a v2 score and get a meaningless delta.

What the calibration study changed

The calibration study did not directly inform v2. The rubric was already designed and shipped in v4.19.0 (October 9). The study ran on October 17. But the study confirms the design philosophy: the score measures citability, not citation. The 9 categories check whether a page is reachable and parseable. They do not claim to predict whether it will be cited.

The null result is actually useful for the rubric. It means I should not inflate the score's claims. A 90/100 does not mean "you will be cited by ChatGPT." It means "your page is technically reachable and parseable by AI crawlers." That is a narrower, more honest claim.

What is next

The rubric will keep evolving. The Google AI readiness category is new and its 20-point weight is a starting estimate, not a final answer. As more calibration data comes in (I plan to run the study again with ChatGPT and Google AI Overviews, not just Perplexity), the weights may shift again.

If you want to see the full scoring rubric, it is in SCORING_RUBRIC.md on GitHub. The tool is free and open source. You can also run a free web audit on geoready.dev without installing anything.

Top comments (0)