How We Score "AI Search Readiness" — The 5-Pillar Method Behind Our Free Checker

lianyang — Fri, 23 Oct 2026 04:40:05 +0000

Over the last year, a new kind of search behavior went mainstream: people ask ChatGPT, Perplexity or Gemini for recommendations — "find me a CPA", "best CRM for a two-person startup" — and act on the shortlist the model gives them. For site owners, this is a third surface next to Google and social. And most websites fail at it quietly: not because their content is bad, but because the machines can't crawl, parse or quote them.

We built a free AI Search Checker to measure exactly that. This post is the methodology: the five pillars we score, what each one actually predicts, and the fixes that move the number. No signup on the tool, and nothing here is behind a paywall — if you want to roll your own audit after reading, that's a win too.

Pillar 1 — AI crawler access (25 points)

Every AI answer is assembled from a corpus. If the crawlers that build that corpus are blocked by your robots.txt, you don't exist in the answer — regardless of content quality.

The boring truth is that most sites have never audited their robots.txt for AI agents. Legacy "bad bot" blocks take out GPTBot, ClaudeBot and PerplexityBot in one line. We check six user agents explicitly: GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and CCBot — for each one, whether it's explicitly allowed, default-allowed, or blocked.

The fix is five lines of config. Declare explicit Allow: / groups per AI bot. Ambiguity is the enemy: crawlers fall back to defaults you don't control.

Pillar 2 — llms.txt (15 points)

llms.txt is a young proposal: a markdown file at your site root that hands LLMs a curated map of your pages — what the site is, which pages matter, in plain markdown. Think robots.txt, but for comprehension instead of permission.

It's not yet a ranking signal anywhere, and we say so on the tool's own results page. We still score it, for two reasons: adopters are early enough to stand out, and writing one forces you to articulate your site's structure — which improves your content either way. If you want to try it, our llms.txt generator takes two minutes.

Pillar 3 — Structured data (20 points)

Schema.org markup is how a machine knows what your page is and who is making the claim. We check for three layers:

JSON-LD present at all — the baseline
Organization / WebSite identity — can the engine attribute content to a real entity?
Rich types — FAQPage, Article, HowTo — the formats answer engines lift most easily

The highest-leverage single block for most sites is FAQPage on pages that answer real questions. It's also the one most teams skip.

Pillar 4 — Citability (25 points)

This is where GEO overlaps content strategy. Answer engines don't rank pages, they extract passages. A passage is liftable when it's self-contained: a question-shaped heading, followed by a direct answer in the first sentence or two.

We measure the proxies: single H1, meaningful H2/H3 structure, question-form headings ("How much does…", "What is…"), a real title and meta description, and enough content depth that a passage has context around it. None of this is exotic SEO — it's writing in a shape that survives extraction.

Pillar 5 — Fundamentals (15 points)

HTTPS, canonical URLs, mobile viewport, OpenGraph. The unglamorous 15 points. They matter because every crawler — classic or AI — treats them as competence signals, and they're all one-hour fixes.

What the score does and doesn't tell you

A 0–100 readiness score measures access and parseability. It does not measure authority — whether engines choose to cite you still depends on content quality, reputation and links. We're explicit about that boundary: the checker tells you if you're invisible for technical reasons; it can't promise citations.

In the first weeks of running it on real sites, the most common profile we see is 40–70, with the same two gaps: AI crawlers blocked by an inherited robots.txt, and zero FAQPage schema. Both are ten-minute fixes that move you 20+ points.

Try it

The AI Search Checker is free, no signup, and returns a per-bot access table plus a prioritized fix list in about fifteen seconds. If you disagree with our weights — good. The pillars are on the results page; build your own rubric and tell us what we got wrong.

DEV Community: lianyang