DEV Community: A.R. Hashir

Why isn't my site cited by ChatGPT? I tested my own blog.

A.R. Hashir — Sun, 08 Nov 2026 15:04:33 +0000

I spent two weeks assuming my website had a technical problem.

That is what every article tells you. Check your robots.txt. Check your schema. Check whether your CMS is stripping your content. So I did all of it, carefully, on my own Blogspot site, expecting to find something broken.

The technical layer came back clean. Seven checks out of seven.

The problem was somewhere I had not thought to look: 282 headings across 93,270 words, and only fifteen of them were questions.

This post is the audit, the numbers, and the three changes I am making as a result.

The site I audited

arhashirbooks.blogspot.com — thirteen published posts, 93,270 words, written between late October and early November 2026. Topics are mostly AI search, structured data, and a few straight how-to guides. Nothing unusual. A standard Blogspot blog with a custom theme.

My hypothesis was that the platform was the problem. Blogspot has a reputation for being technically limited compared to a self-hosted site, and I assumed that reputation had cost me citations.

It had not.

What I measured

Every number below came from the live site, not from a dashboard. Blogspot publishes a full JSON feed of every post and its raw HTML content at a predictable URL, which makes this auditable by anyone.

curl -s "https://arhashirbooks.blogspot.com/feeds/posts/default?alt=json&max-results=500" -o feed.json

That single request returns every post, its publish date, its permalink, and the complete HTML body. No scraping, no rendering, no third-party tool. If you are on Blogspot, you can run exactly the same audit on your own blog in about a minute.

From that feed I extracted seven technical checks and four content checks.

Finding 1: the technical layer was fine

Check	Result	Status
`robots.txt` allows crawling	Yes, only `/search` and `/share-widget` disallowed	✅
XML sitemap exists and is reachable	200, listed in robots.txt	✅
Canonical tag present on posts	Yes, self-referencing	✅
JSON-LD structured data	`BlogPosting`, `Organization`, `Person`, `WebPage`	✅
Single `<h1>` per page	1	✅
Mobile viewport meta	`width=device-width, initial-scale=1`	✅
Open Graph image	Present on posts	✅

Seven for seven. The platform ships a reasonable default template, and nothing was blocking anything.

This matters because it kills the most common diagnosis. If your site is on a mainstream platform, the technical foundation is almost certainly not your problem. You are not being ignored because of a missing tag.

You are being ignored because of what the crawler finds after it gets in.

Finding 2: 282 headings, 15 questions

I pulled every <h2> from every post and counted how many were phrased as an actual question — meaning they ended in a question mark and were asking something a reader might type.

15 out of 282. That is 5.3%.

Here is every post, sorted by length, with its heading count and question count:

Post	Words	H2	Questions
25 AI Workflows for Small Business	20,713	59	0
Personal AI Assistant: 15 Best AI Agents	11,371	25	3
Fractional Chief AI Officer (CAIO)	8,019	19	2
How to Focus Better at Work	7,743	27	0
Structured Data for AI Search	7,066	17	0
What Is an SBOM?	6,823	23	3
How to Read Body Language	6,611	21	3
AI Visibility Audit	6,263	15	1
EU Cyber Resilience Act Deadlines	5,293	17	2
AI Search Optimization	3,914	17	1
Which Task Is a Generative AI Task?	3,707	14	0
Crawled – Currently Not Indexed on Blogger	3,490	17	0
How to Track AI Search Visibility	2,257	11	0
Total	93,270	282	15

Look at the top row. My longest post — 20,713 words, 59 headings — has zero question headings. It is the most thorough thing I have written and it is structurally the least retrieval-friendly.

Six of the thirteen posts had zero question headings. Not one.

Here is what my headings actually look like, taken directly from the feed:

"Start with visible facts, not code"
"What structured data can and cannot do"
"The 100-point AI visibility scorecard"
"Choose schema by the page's job"

These are not bad headings. They are clear, specific, and useful to a human reading top to bottom. I was proud of them.

They are also invisible to a retrieval system that is trying to match a user's question to a passage of text.

Why question headings matter more than they sound

When someone asks ChatGPT or Perplexity a question, the system does not read your whole page. It retrieves passages, and it matches them against the shape of the query.

If the query is "does schema help AI search", a passage sitting under the heading "Does schema help AI search?" is a near-exact match. A passage sitting under "Start with visible facts, not code" might contain the same information and might be equally true — but the system has to infer that. Inference is work. Retrieval systems prefer not to work.

That is the whole mechanism. It is not a trick and it is not a ranking factor in Google's classic sense. It is about making the match obvious instead of making the machine guess.

One post on my blog already did this correctly, by accident. It has a heading that reads "Why is my website not showing up in ChatGPT? Start with this 60-second triage" and it remains the post with the most question-shaped structure I have written. That is the pattern to copy deliberately rather than land on by luck.

The three heading mistakes I was making

Rewriting 282 headings taught me that I had three specific habits, and all three are common.

Habit 1: clever headings instead of plain ones.

"Start with visible facts, not code" is a nice line. It also requires the reader to already know what the section is about. A plain heading — "Should you fix visible content before adding schema?" — tells the system what question lives underneath, in the words the searcher would use.

Clever headings are written for a human who is already reading. Plain question headings are written for a human who is still deciding whether to read.

Habit 2: compound headings that merge two ideas.

Several of mine did two jobs at once: "Structured data, schema and JSON-LD: three different jobs." That is a definition, a comparison, and a heading all crammed into one line. When a passage underneath a compound heading is retrieved, the system cannot tell which part you are answering.

One heading, one question. If a section answers two questions, it should be two sections.

Habit 3: navigational labels dressed up as headings.

"On this page" appeared as an <h2> on more than one post. So did "The 10-minute map: where to start." Those are page furniture. They burn one of your heading slots on something that will never match a query.

Page furniture belongs in a list, or as bold text, or as an actual table of contents component — not as a heading competing with your content headings.

What a matching query actually looks like

Concrete example, using a real section from my own blog.

A searcher types: "does schema help AI search".

My original heading: "What structured data can and cannot do".

The paragraph underneath said something close to: schema does not directly make AI systems cite you, but it removes ambiguity about entities and facts, and the measurable benefit shows up in disambiguation rather than citation count.

Everything needed to answer "does schema help AI search" was in that paragraph. Nothing in the heading matched the query.

The rewrite: "Does schema help AI search? The honest evidence."

Same paragraph. Same claim. Now the heading contains every word of the query, in the same order, ending as a question. A retrieval system scanning for a passage about whether schema helps AI search no longer has to infer anything — the match is explicit.

This is not clever. It is closer to janitorial work. But it is the specific work that most sites skip, because it is boring and because the old headings already felt finished.

The rewrite, before and after

I did not have to change any content. Only the headings.

Before	After
Start with visible facts, not code	Should you fix your visible content before adding schema?
What structured data can and cannot do	What does structured data actually do for AI search?
The 100-point AI visibility scorecard	How do you score AI visibility out of 100?
Choose schema by the page's job	Which schema type should you use for each page?
Validate in five passes	How do you validate JSON-LD without breaking existing markup?

Same content underneath. Same claims, same evidence, same order. The heading now names the question the passage answers.

There is a limit on this. Do not turn every heading into a question, because that reads like a FAQ page and humans hate it. In a 3,000-word post, five or six question headings is enough to give the retrieval system clear anchors without turning the article into a quiz.

Finding 3: eight of thirteen posts never linked to the thing I sell

This one was worse than the heading problem, and it had nothing to do with AI.

Of thirteen posts:

8 had no link to my product page at all
5 had at least one
0 linked to anywhere else I publish

I had 93,000 words of content that people were reading, and in most of it there was no next step. A reader who finished a 7,000-word guide on structured data had nowhere to go.

That is not an SEO problem. That is a basic editorial mistake, and I had made it on sixty percent of my own library.

The fix is not to stuff links everywhere. It is to add exactly one relevant next step per post, placed where the reader has just finished the problem you described. If the post is about AI visibility audits, the next step is the checklist. If it is about schema templates, the next step is the full set of templates.

Finding 4: zero posts pointed at my other platform

Every post linked internally to other posts on the same blog. Good. None of them linked out to where I also publish.

That is a missed connection, not a disaster. Cross-linking between your own properties does two things: it gives readers a second place to find you, and it makes the relationship between your entities explicit to anything reading the web.

What I am changing

Three changes, in this order.

1. Rewrite the headings on the five posts that already rank. Not new content. Five or six question-shaped headings per post, using the actual questions those sections answer. This is a two-hour job across five posts and it touches nothing else — no title changes, no URL changes, no republishing.

2. Add one next step to every post. For the eight posts with no product link, one relevant link, placed after the section that solves the problem. For the five that already have one, check that it is in the right place rather than bolted onto the end.

3. Add a link back from every post to the main blog and to my other writing. Two lines at most. Intrusive link blocks at the bottom of posts do nothing; a sentence in context does something.

I will report back with numbers. The honest expectation is that none of this moves anything for four to twelve weeks, and I would rather say that now than pretend otherwise.

Which posts to fix first

You cannot rewrite 282 headings in an afternoon, and you should not. The order matters more than the volume.

I ranked my thirteen posts on three questions, and fixed them in that order.

1. Does this post already get impressions from Google?

Check Search Console, not your feelings. A post with 400 impressions and a 0.4% click-through rate is a post Google already considers relevant but not compelling. That is the cheapest fix available to you: better headings on a page that already ranks.

A post with zero impressions is a different problem entirely. Rewriting its headings will not create demand that does not exist.

2. Does the post answer a question someone would actually type?

Not "would a reader find this useful" — literally, is there a search query that maps to this section. If the section is "How to Focus Better at Work", the queries are real and specific. If it is "The decision that matters before choosing a tool", the query is probably something like "how to choose an AI tool for my business", which is close but not the same.

3. Does the post end anywhere?

A post that solves a problem and then stops is a dead end. A post that solves a problem and then offers the next step is a funnel, and it does not require you to be pushy. It requires you to notice that the reader who just finished your 7,000-word guide on structured data might want the templates.

My ranking, once I applied all three: the five AI-search posts first, because they have the clearest queries, the highest commercial intent, and they already rank for something. The generic how-to posts (focus, body language, generative AI tasks) last, because they bring traffic that has nothing to do with what I sell, and converting them is a different project.

That last point is worth sitting with. Two of my thirteen posts are about body language and focus at work. They are fine posts. They will never sell a developer checklist, no matter how good the headings are.

What I expect, and when

It would be easy to end this post with the implication that fixing headings will get me cited next week. It will not.

Here is what I actually expect, with the reasoning attached.

Timeframe	What I expect to see
Week 1–2	Nothing. Google has to recrawl. Citations will not move.
Week 3–4	Possible movement on exact long-tail queries where I already rank.
Week 6–8	First honest signal. Either some queries start returning my pages, or they do not.
Week 12	Enough data to say whether this worked, with a real before-and-after.

The reason for the delay is not mysterious. Recrawling is not instant, retrieval indexes are rebuilt on their own schedule, and the queries I care about are ones where I currently rank nowhere. Even a perfect heading rewrite cannot shortcut a site that has no authority yet.

Which is the part I want to be honest about: headings are a fix for a site that is already being found and not being chosen. They will not make a site get found in the first place. If nobody knows your site exists, better headings change nothing, because there is no query reaching your pages at all.

My site has been found — it has traffic and it has indexed pages. My headings were costing me the second step. That is the specific problem I am fixing.

How to run this on your own blog

If you are on Blogspot, the whole audit is three commands.

Get the feed:

curl -s "https://yourblog.blogspot.com/feeds/posts/default?alt=json&max-results=500" -o feed.json

Count your headings and your questions:

import json, re

feed = json.load(open("feed.json"))
total_h2 = total_q = 0

for entry in feed["feed"]["entry"]:
    html = entry["content"]["$t"]
    headings = re.findall(r"<h2[^>]*>(.*?)</h2>", html, re.S)
    total_h2 += len(headings)
    total_q += sum(1 for h in headings if re.sub(r"<[^>]+>", "", h).strip().endswith("?"))

print(f"{total_q} of {total_h2} headings are questions")

Check what your headings actually say:

for entry in feed["feed"]["entry"]:
    print("\n#", entry["title"]["$t"])
    for h in re.findall(r"<h2[^>]*>(.*?)</h2>", entry["content"]["$t"], re.S):
        print("  -", re.sub(r"<[^>]+>", "", h).strip())

That is it. If you are not on Blogspot, the same idea works with any sitemap and a loop over your <h2> tags.

If you want the full version, with the technical checks and the scoring, I put the whole thing into a checklist:

Free AI Search Readiness Checklist (20 pages, no email gate) →

And if you want the complete system — the 238-page handbook, the audit script, the schema templates and the scoring workbook:

Structured for Answers: the full AI Search & SEO Vault →

What I still do not know

Three things I want to be clear about, because the numbers above are easy to over-read.

I have not measured whether question headings increase citations. I changed mine and I am watching. I do not have before-and-after data yet. The reasoning is sound and the mechanism is visible, but reasoning is not evidence, and I would rather say so than dress up a hypothesis as a result.

Thirteen posts is a small sample. One blog, one niche, one author. It is a case study, not a study.

The heading count may not matter at all on its own. It could be that question headings matter only when the content underneath them is genuinely good. My content was fine and my headings were not, which is a very specific starting condition. Yours may be the reverse.

If you run the same audit on your own blog, I would like to see the number. Drop your question-heading percentage in the comments. I will collect them and publish the range, whatever it turns out to be.

I write about AI search and what actually moves citations, mostly by testing things on my own sites and publishing the numbers either way. The longer guides live at arhashirbooks.blogspot.com.

AI Crawler Access: I Tested 11 Top SEO Sites. None Block.

A.R. Hashir — Fri, 06 Nov 2026 16:17:00 +0000

Almost every guide about getting cited by AI opens the same way.

Your robots.txt is blocking GPTBot. Your CDN is returning 403 to Perplexity. Fix that before you do anything else.

I wanted to know whether that was actually true. So I ran the test against eleven sites that already rank well and already show up in AI answers: Ahrefs, Moz, Semrush, Search Engine Journal, Screaming Frog, Stripe, Shopify, GitHub, Cloudflare, Dev.to, and Vercel.

I expected to find a few of them blocking. Maybe the big ones, protecting content behind a wall.

Not one of them was. Zero out of eleven. And the third finding was the one that actually surprised me.

The four crawlers people keep confusing

Before the results, one clarification, because most of the bad advice online comes from people mixing these up. They are not the same thing and they do very different jobs.

User agent	Who runs it	What it does
`GPTBot`	OpenAI	Trains models. Being blocked here does not stop ChatGPT from citing you.
`OAI-SearchBot`	OpenAI	Indexes pages for ChatGPT search. This is the one that decides citations.
`ChatGPT-User`	OpenAI	Fetches a single page when a user pastes a URL or asks a live question.
`PerplexityBot`	Perplexity	Indexes and cites.
`ClaudeBot`	Anthropic	Crawls for Claude.
`Google-Extended`	Google	Controls Gemini training and grounding. Does not control AI Overviews.
`CCBot`	Common Crawl	Public web archive used by many downstream models.

The one that matters most for getting cited is OAI-SearchBot, and it is the one almost nobody mentions. Everyone writes about GPTBot, which mostly affects training, not citations.

How the test works

Two checks per site. Both take under a minute.

Check one — what does robots.txt say?

curl -s https://example.com/robots.txt | grep -iE "gptbot|oai-searchbot|perplexitybot|claudebot|google-extended|ccbot"

If you get output with Disallow: / under a bot's name, that bot is blocked.

Check two — what happens when the crawler actually shows up?

This one matters more, because robots.txt is a polite request. A CDN, a WAF, or a bot-management toggle can ignore it completely and return 403 to anything it does not like. Many managed hosts do exactly this without telling you.

for ua in "GPTBot/1.2" "OAI-SearchBot/1.0" "PerplexityBot/1.0" "ClaudeBot/1.0"; do
  echo -n "$ua -> "
  curl -s -o /dev/null -w "%{http_code}\n" \
    -A "Mozilla/5.0 (compatible; $ua; +https://example.com)" \
    https://example.com
done

A 200 means the crawler gets in. A 403 means it gets the door slammed in its face, no matter what robots.txt promised.

Finding 1: nothing is blocked. Nothing.

Eleven out of eleven sites allow GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, Google-Extended, and CCBot. All of them, all six, no exceptions. No wildcard Disallow: / anywhere.

Then I sent each AI user agent to each home page. Every single combination came back HTTP 200.

Domain	robots.txt	GPTBot	OAI-SearchBot	PerplexityBot	ClaudeBot
ahrefs.com	allow all	200	200	200	200
moz.com	allow all	200	200	200	200
semrush.com	allow all	200	200	200	200
searchenginejournal.com	allow all	200	200	200	200
screamingfrog.co.uk	allow all	200	200	200	200
stripe.com	allow all	200	200	200	200
shopify.com	allow all	200	200	200	200
github.com	allow all	200	200	200	200
cloudflare.com	allow all	200	200	200	200
dev.to	allow all	200	200	200	200
vercel.com	allow all	200	200	200	200

Fifty-five requests. Fifty-five 200s. Not a single 403, not a single challenge page, not a single rate limit.

I want to be careful about what this proves. These are established sites with engineering teams who have looked at this question. It does not mean your host is fine. Custom CDN rules, security plugins, and "block bad bots" toggles are exactly where this breaks, and they break silently. That is why the ninety-second check is still worth running on your own domain.

But it does mean the blanket warning is wrong. The default state of the modern web is permissive.

Finding 2: eight of eleven publish an llms.txt

llms.txt is a plain text file at yourdomain.com/llms.txt. It is not an official standard. No crawler is required to read it. It is just a short Markdown summary of what your site is, what it covers, and where the important pages live.

Eight of the eleven serve one:

Domain	llms.txt	Size
Cloudflare	✅	17 KB (plus a 169 KB full version)
Stripe	✅	69.8 KB
GitHub	✅	28.6 KB
Semrush	✅	14.9 KB
Shopify	✅	12.5 KB (plus 18.8 KB full version)
Vercel	✅	5.4 KB
Dev.to	✅	2.8 KB
Ahrefs	❌ 404	—
Screaming Frog	❌ 404	—
Moz	❌ 403 on that path	—

Here is what they actually say, in their own words:

# Stripe

> Stripe is a technology company that provides financial infrastructure
> for businesses to accept payments, grow revenue, and access capital.

# GitHub

> GitHub is a developer platform for building, shipping, and maintaining
> software.

# Cloudflare

> Cloudflare is a global cloud connectivity company operating in 335+ cities.

# DEV Community

> A Forem community where people share knowledge and participate in
> discussions.

# Semrush

> Semrush is a brand visibility management platform that helps marketers
> audit and improve their online presence.

Every single one follows the same shape. A # heading with the company name. A > blockquote with one or two sentences describing what the company does. Then a list of links.

That is the whole format. No schema, no JavaScript, no build step, no plugin.

Look at what those sentences have in common. Technology company that provides financial infrastructure. Developer platform for building software. Global cloud connectivity company. Brand visibility management platform.

Not one of them says "we empower businesses to unlock their potential." They say what the company is, in a sentence a stranger can repeat back.

Finding 3: the uncomfortable one

If access is not the bottleneck — and it clearly is not, at least among sites that are already succeeding — then something else decides who gets cited.

Look at the eight that publish an llms.txt. It is not that they are big. Ahrefs is big and does not have one. It is not that they are technical. It is that all eight wrote down, in one place, in plain sentences, what they are and what they do.

The newest ones treat it as a product decision, not an SEO trick. Stripe's file is 69 kilobytes of clear statements about what Stripe is, who it is for, and where each piece of documentation lives. That is a business writing down its own facts in public.

Which brings up the part most guides skip.

You can allow every crawler in the world and still get ignored. The crawler arrives, reads your page, and finds a sentence that says "we help businesses unlock their potential." That is not a fact a retrieval system can extract. It is a slogan, and slogans are noise.

Access is a checkbox you can fix once. Clarity is the work, and it never ends.

Crawled is not the same as indexed

One more distinction this test does not cover, and it trips up a lot of people.

Every site above returned 200. That proves the crawler got in the front door. It does not prove the page ended up in an index. Getting crawled and getting indexed are two separate events, and the second one can fail for reasons that have nothing to do with access.

Search Console has a status for exactly this: Crawled – currently not indexed. It means Google fetched the page and then decided not to store it. Nothing was blocked. No 403, no robots.txt rule, no permission problem. The page simply was not considered worth indexing.

That is a different problem, with a different fix, and it comes up constantly on Blogger, where the same article is reachable at ?m=1, with UTM parameters, and through feed URLs, and canonical selection gets messy.

I wrote a separate walkthrough for that specific situation, because it is not something crawler access testing can answer:

Crawled – Currently Not Indexed on Blogger: What to Check (2026) →

The order matters. Check access first, using the commands in this post. Then check canonical selection. Then look at the page itself. Fixing indexing before you have fixed access is wasted effort, because you cannot index what a crawler cannot fetch.

What this test does not prove

Three honest limitations, because the data is only worth as much as its caveats.

The sample is small and biased. Eleven sites, all of them already established, all of them English-language. A startup site on a cheap shared host with an unknown security plugin may behave completely differently.

Home pages are not content pages. I tested the root domain. Individual article pages can have different rules, and article pages are the ones that get cited.

llms.txt has no measured effect here. I did not test whether the eight sites with llms.txt get cited more often. I did not test whether the three without get cited less. That correlation would need far more data. What I can say is that eight of eleven chose to publish one, and all of them are free to do.

What I would check, in this order

1. Actually run the two commands above. Do not assume. Most managed hosts allow everything by default, but custom CDN rules and bot-management toggles quietly break this. Ninety seconds, and it either clears the question or finds a real problem.

2. Write an llms.txt. Upload it to your root. One heading, one blockquote, a short list of your most important pages with plain descriptions. Twenty minutes.

Here is a template you can copy, fill in, and upload:

# Your Company

> One sentence. What you are, who it is for, what you sell.
> Write it so a stranger could repeat it back.

## Main pages
- [Home](https://example.com): what the product does
- [Pricing](https://example.com/pricing): plans and costs
- [Docs](https://example.com/docs): how to use it

## About
- [Company](https://example.com/about): who we are
- [Contact](https://example.com/contact): how to reach us

Replace every placeholder with a real sentence about your own site. Save as llms.txt, upload to your domain root, confirm it loads at yourdomain.com/llms.txt.

3. Read your own home page as if you were a stranger who knows nothing about you. Can you find, in plain sentences: what this is, who it is for, what it costs, and where to go next? If those four facts are buried under adjectives, no crawler permission will save you.

4. Test a content page, not just your home page. Pick your best article or product page and run the same two commands on it. That is where citations actually come from.

I turned all of this into a scored checklist with the exact commands and a 100-point breakdown. It is free and there is no email gate before the download:

AI Search Readiness Checklist →

One last thing

I would genuinely like to know if anyone finds a site that does block. Eleven out of eleven is a small sample, and I would rather my conclusion be wrong in public than quietly wrong in private.

Drop the domain in the comments and I will add it to the table. If someone finds twenty sites that block, the whole premise of this post changes, and I would want to know.

The AI Search Audit: 8 Checks, 100 Points, One Script

A.R. Hashir — Sat, 24 Oct 2026 11:03:18 +0000

Your analytics flattened in March. Organic clicks are down 30%, then 40%. You shipped nothing wrong. Rankings look stable. The traffic just isn't arriving.

Here is what most teams do next: they buy a "GEO audit," add more schema, publish more content. Six months later the numbers are worse and nobody can point at a single decision that caused it.

The reason is almost never the thing being optimized. It's that three technical gates were failing the entire time, and every downstream fix was being applied to a page that a crawler could not fully read.

This is a scored audit for finding that. Eight categories, 100 points, and one script that runs most of it in about thirty seconds.

Why this got harder in 2026, in numbers

Three data points worth internalizing before the checklist:

68% of Google searches now end without a click. That is up from roughly 45% a decade ago. Search volume is at an all-time high and still growing — the capture rate collapsed, not the demand.

When an AI Overview appears, click-through drops by about 61%. AI Overviews trigger on a growing share of informational queries. The top organic result still exists; people just stop there less often.

Only 8% of visits click a traditional result when an AI summary is present, versus 15% when it isn't. Users read the summary. They click the citation occasionally. They click the blue link rarely.

One number moves in the opposite direction, and it's the one that matters for this audit: AI-referred visitors convert at roughly 3x the rate of traditional organic traffic. Similarweb measured ChatGPT-referred sessions at about 15 minutes versus 8 for Google.

So the shape of the opportunity changed. Fewer clicks, better clicks. Which means eligibility is now worth more than volume — a page that can be retrieved and cited is worth several that cannot.

That reframes the work. You are not chasing rankings. You are making your content retrievable, quotable, and unambiguous. That's a technical problem before it's a writing problem, and it's the subject of this audit.

The 40/60 rule: why most audits start in the wrong place

Here is the single most useful thing in this article.

The eight categories below are not weighted equally, and two of them are blocking. If Categories 1 and 2 fail, the other six are decoration.

	Category	Points	Blocking
01	Crawlability	20	Yes
02	Renderability	20	Yes
03	Entity clarity	18	—
04	Structured data	15	—
05	Content chunking	12	—
06	Freshness & trust	8	—
07	Answer formatting	5	—
08	Measurement	2	—

Categories 01 and 02 are 40% of the score and they gate everything downstream. If those two come in under 32 out of 40, stop. Fix rendering and robots.txt this week. Do not spend a single hour on schema.

I have watched teams spend a quarter perfecting sameAs arrays and entity graphs while their documentation body copy was assembled in the browser after hydration. The audit spreadsheet looked excellent. The crawler was receiving an empty shell.

That is the 40/60 rule. Apply it before you read further.

Category 01 — Crawlability (20 points)

Can a non-browser client reach and read your URLs at all? This is binary, and everything else is downstream of it.

Check	Pts
`robots.txt` returns HTTP 200 and contains no wildcard `Disallow: /`	3
`OAI-SearchBot` explicitly allowed (or deliberately blocked, and documented)	3
`Googlebot` allowed on every route that matters	3
`PerplexityBot` explicitly allowed (or deliberately blocked)	2
`Bingbot` allowed — Copilot grounding depends on the Bing index	2
Sitemap exists at `/sitemap.xml` and is referenced from `robots.txt`	2
Sitemap contains only canonical, indexable, 200-status URLs	2
No WAF, bot challenge or CDN rule returns 403/429/503 to AI user agents	3

The silent failure that costs entire channels

A CDN default that 403s PerplexityBot costs you the whole channel and produces zero errors anywhere. No deploy, no alert, no log line you'll notice. This is the most common cause of "we optimized for AI search and nothing happened."

Four lines of shell settles it:

for ua in "GPTBot/1.1" "OAI-SearchBot/1.0" \
          "PerplexityBot/1.0" "ClaudeBot/1.0"; do
  code=$(curl -s -o /dev/null -w "%{http_code}" \
    -A "Mozilla/5.0 (compatible; $ua)" https://yoursite.com/)
  echo "$ua -> $code"
done

Four 200s, or you have found your first problem.

Know which crawler does what

The robots.txt snippets circulating online are two years stale, and most conflate three different jobs into one decision. These are separate agents:

Agent	Operator	Governs	Worth blocking?
`GPTBot`	OpenAI	Model training	Optional
`OAI-SearchBot`	OpenAI	ChatGPT search citations	Rarely
`ChatGPT-User`	OpenAI	User-triggered live fetch	Rarely
`ClaudeBot`	Anthropic	Training + crawling	Optional
`Claude-SearchBot`	Anthropic	Search grounding	Rarely
`PerplexityBot`	Perplexity	Search index	Rarely
`Googlebot`	Google	Search + AI Overviews	Never
`Google-Extended`	Google	Gemini training only	Optional
`Bingbot`	Microsoft	Bing + Copilot grounding	Rarely
`CCBot`	Common Crawl	Upstream model data	Optional

Two corrections that change real decisions:

Blocking Google-Extended does not remove you from Google AI Overviews. AI Overviews are served from the ordinary Google index, governed by Googlebot. Google-Extended only controls Gemini training.

Blocking GPTBot does not remove you from ChatGPT search results. Citations come from OAI-SearchBot, and live URL fetches use ChatGPT-User. Three separate entries, three separate decisions.

A coherent default for most commercial sites is: block training, allow search. That is defensible and, more importantly, deliberate rather than copied.

Category 02 — Renderability (20 points)

Does the content exist in the delivered HTML, or does it appear only after JavaScript runs? This is where modern frameworks quietly lose 15 points.

Check	Pts
Primary body copy present in raw HTML with JavaScript disabled	8
`h1`–`h3` headings present in raw HTML	3
Internal links are real `<a href>` elements in raw HTML	3
JSON-LD present in raw HTML, not injected after hydration	3
Content survives `curl` at 80%+ of the rendered word count	3

# raw word count, tags stripped
curl -s https://yoursite.com/page | sed 's/<[^>]*>//g' | wc -w

Compare that against what the browser shows. Within 10% is fine. A 30% gap is a widget. A 3x gap means your content does not exist as far as retrieval is concerned — no matter how clean your markup is.

The hydration trap

This is the pattern that silently fails:

// ❌ invisible to most AI crawlers
useEffect(() => {
  const s = document.createElement('script');
  s.type = 'application/ld+json';
  s.text = JSON.stringify(schema);
  document.head.appendChild(s);
}, [schema]);

And this is the fix:

// ✅ present in the raw HTML response
const jsonLd = JSON.stringify(schema, null, 2);
return (
  <script
    type="application/ld+json"
    dangerouslySetInnerHTML={{ __html: jsonLd }}
  />
);

Verify it in the only place that counts — not DevTools:

curl -s https://yoursite.com/page | grep -c 'application/ld+json'

0 means everything downstream of this section is theoretical.

The practical answer is SSR, SSG, or ISR for every content route. Client-side rendering belongs to application state, not to published content.

Category 03 — Entity clarity (18 points)

Retrieval systems resolve entities, not pages. Can a machine work out who you are, unambiguously, from any page on the site?

Check	Pts
Exactly one `Organization` (or `Person`) node per page, consistent across pages	4
`@id` values used and cross-referenced between nodes	3
`sameAs` lists 3–8 live, populated profiles (no 404s, no empty accounts)	4
Same entity name in `<title>`, `og:title`, JSON-LD and visible body copy	3
Organization node has a resolvable `logo` ImageObject at an absolute URL	2
No duplicate or conflicting entity blocks from competing plugins	2

curl -s https://yoursite.com/ | grep -o '"@type":"Organization"' | wc -l
# expected: 1

A return of 2 means a plugin and a theme are both emitting an organization with different names or URLs. To a reconciling system that is two companies, and consolidation fails silently — no error, no warning, no dashboard.

On sameAs discipline: every URL must return 200 and carry real content. A dead profile is worse than no profile, because you asserted a fact that then failed verification. Three to eight links is the useful range. Forty reads as manipulation and gets discounted.

Category 04 — Structured data (15 points)

Validators check shape. They do not check meaning, recency, or whether the markup agrees with the page. That gap is where these 15 points live.

Check	Pts
Every JSON-LD block parses with zero syntax errors on priority URLs	3
`datePublished` / `dateModified` in ISO 8601	3
Visible page content matches every claim the markup makes	3
`@type` values correctly cased (`WebPage`, not `Webpage`)	2
All `image` values are absolute `https://` URLs	2
`FAQPage` uses nested `Question` → `acceptedAnswer` → `Answer`	2

Values that parse but degrade output

Property	Common value	Correct value
`datePublished`	`Oct 24, 2026`	`2026-10-24`
`image`	`/cover.png`	`https://yoursite.com/cover.png`
`@type`	`Webpage`	`WebPage`
`name`	`Brand \	Best Tools 2026`
`description`	400-character paragraph	150–160 chars, one sentence

"Oct 24, 2026" is ambiguous enough that some parsers drop the field entirely, and dateModified is one of the few freshness signals you fully control.

The row people skip is row three. If your markup asserts an answer the page does not visibly give, you have manufactured a confident wrong answer attributed to your brand. That is worse than silence, because it will be quoted.

And one tool used for the wrong job: the Rich Results Test validates syntax. Syntax was never the failure mode. Use it as a lint step, not a diagnosis.

Category 05 — Content chunking (12 points)

Retrieval does not read your page top to bottom. It splits the page along heading and paragraph boundaries, embeds each piece, and fetches whichever pieces match. Write for the piece.

Check	Pts
Each H2 section is self-contained and makes sense in isolation	4
Each section opens with a direct answer in the first one or two sentences	3
No context-dependent openers ("as mentioned above", a bare "this")	2
Paragraphs under roughly 120 words	2
Key claims exist as text, not only inside images	1

The isolation test

Copy a single H2 section into a fresh chat with no surrounding page. If it makes sense and answers something, it survives chunking. If it opens with a pronoun pointing three sections up, it does not — and that chunk will never be retrieved no matter how good the page is as a whole.

Pattern	Survives	Does not
Opening	"JSON-LD works by placing a self-contained entity graph…"	"It works by wrapping the entity, as mentioned above…"
Scope	Names the subject in the first clause	Relies on the H1 for context
Length	One idea, under 120 words	Four ideas in one 300-word block

This category costs nothing to fix and is the one most often skipped. It is a writing-discipline change, not an engineering one — no deploy, no ticket, no framework upgrade.

Category 06 — Freshness & trust (8 points)

Check	Pts
Long-lived pages show a visible "last updated" date	2
`dateModified` reflects real edits, not a mechanical bump	2
Author or organization attribution visible on-page	2
Fact-heavy sections cite external sources with working links	1
Nothing material is stale in a fast-moving topic	1

A 2023 page about 2026 crawler behaviour is not merely unranked — it gets retrieved, and it is wrong. Archive or update. Do not leave it up hoping.

Category 07 — Answer formatting (5 points)

Deliberately the lowest-weighted category, because it is the easiest and everyone does it first.

Check	Pts
Summary or TL;DR block near the top of long pages	2
Comparison tables for "X vs Y" queries	2
Procedural content uses ordered steps	1

Formatting makes content easy to extract, so it feels productive. It is worth 5 points and will not rescue a site scoring 20 on the first two categories.

Category 08 — Measurement (2 points)

Check	Pts
Server logs accessible, AI crawler user agents filterable	1
Five target questions probed weekly across ChatGPT, Perplexity and Google	1

grep -iE "GPTBot|OAI-SearchBot|ClaudeBot|PerplexityBot" \
  /var/log/nginx/access.log \
  | awk '{print $9}' | sort | uniq -c

Three things fall out in ninety seconds: whether these crawlers visit at all, which URLs they actually pull, and whether they receive 200 or bounce off a 403. If the answer to the first is no, the rest of your plan is theoretical.

The whole audit in one script

Here is every automatable check above, in a single file. Save it as audit.sh, make it executable, and pass one URL.

It does not cover Categories 03 through 07 completely — those need human judgment — but it will find the blocking failures, which is where you should start.

#!/usr/bin/env bash
# ai-search-audit.sh — blocking checks only
# usage: ./audit.sh https://yoursite.com/page

URL="${1:?usage: ./audit.sh <url>}"
BASE=$(printf '%s' "$URL" | sed -E 's#(https?://[^/]+).*#\1#')
PASS=0; FAIL=0

ok()   { echo "  PASS  $1"; PASS=$((PASS+1)); }
bad()  { echo "  FAIL  $1"; FAIL=$((FAIL+1)); }

echo "== 01 CRAWLABILITY =="

ROBOTS=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/robots.txt")
[ "$ROBOTS" = "200" ] && ok "robots.txt returns 200" \
                      || bad "robots.txt returned $ROBOTS"

if curl -s "$BASE/robots.txt" | grep -qiE '^Disallow: *\*? *$|^Disallow: */$'; then
  bad "robots.txt contains a wildcard Disallow"
else
  ok "no wildcard Disallow in robots.txt"
fi

for ua in "GPTBot/1.1" "OAI-SearchBot/1.0" "PerplexityBot/1.0" \
          "ClaudeBot/1.0" "Googlebot/2.1"; do
  code=$(curl -s -o /dev/null -w "%{http_code}" \
          -A "Mozilla/5.0 (compatible; $ua)" "$URL")
  name=${ua%%/*}
  if [ "$code" = "200" ]; then ok "$name reaches the page (200)"
  else bad "$name got $code — blocked by WAF/CDN?"; fi
done

SM=$(curl -s -o /dev/null -w "%{http_code}" "$BASE/sitemap.xml")
[ "$SM" = "200" ] && ok "sitemap.xml returns 200" \
                  || bad "sitemap.xml returned $SM"

echo
echo "== 02 RENDERABILITY =="

RAW=$(curl -s "$URL")
RAW_WORDS=$(printf '%s' "$RAW" | sed 's/<[^>]*>//g' | wc -w | tr -d ' ')
[ "$RAW_WORDS" -ge 300 ] && ok "raw HTML has $RAW_WORDS words" \
                         || bad "raw HTML has only $RAW_WORDS words — JS-rendered?"

LD=$(printf '%s' "$RAW" | grep -c 'application/ld+json')
[ "$LD" -ge 1 ] && ok "JSON-LD present in raw HTML ($LD block(s))" \
                || bad "no JSON-LD in raw HTML — injected after hydration?"

H1=$(printf '%s' "$RAW" | grep -c '<h1')
[ "$H1" -ge 1 ] && ok "h1 present in raw HTML" \
                || bad "no h1 in raw HTML"

LINKS=$(printf '%s' "$RAW" | grep -o 'href="http' | wc -l | tr -d ' ')
[ "$LINKS" -ge 5 ] && ok "$LINKS absolute internal/external links in raw HTML" \
                   || bad "only $LINKS links in raw HTML"

echo
echo "== 04 STRUCTURED DATA (syntax) =="

ORG=$(printf '%s' "$RAW" | grep -o '"@type":"Organization"' | wc -l | tr -d ' ')
if [ "$ORG" -eq 1 ]; then ok "exactly one Organization node"
elif [ "$ORG" -eq 0 ]; then bad "no Organization node found"
else bad "$ORG Organization nodes — duplicate entity data"; fi

printf '%s' "$RAW" | grep -qE '"date[A-Za-z]+":"[0-9]{4}-[0-9]{2}-[0-9]{2}"' \
  && ok "ISO 8601 dates present" || bad "no ISO 8601 date fields detected"

echo
echo "== RESULT =="
echo "  passed: $PASS"
echo "  failed: $FAIL"
[ "$FAIL" -eq 0 ] && echo "  blocking categories look clear." \
                  || echo "  fix the FAILs above before touching schema."

Run it. If anything prints FAIL in the first two sections, you have your work order for the week and the other six categories can wait.

Your score sheet

Fill this in after running the script and scoring the human-judgment rows.

Category	Max	Yours
01 · Crawlability	20
02 · Renderability	20
03 · Entity clarity	18
04 · Structured data	15
05 · Content chunking	12
06 · Freshness & trust	8
07 · Answer formatting	5
08 · Measurement	2
TOTAL	100

What your score means

Score	Meaning	Action
85–100	Eligible. Remaining gap is content quality, not technical	Stop auditing. Write better pages
60–84	Partially eligible. Citations will be inconsistent	Close 01–02 first, then 03–04
35–59	Mostly invisible. Likely blocked, JS-only, or unparseable	Everything else is wasted until 01–02 pass
0–34	Not reachable. Do not buy tooling yet	Fix rendering and robots.txt this week

Fastest wins, ranked by minutes-to-impact

Fix	Time	Typical points
Unblock AI UAs in robots.txt / WAF	15 min	up to 8
ISO-format all dates	20 min	3
Remove `display:none`-only body content	30 min	3
Deduplicate Organization blocks	45 min	4
Add an `@graph` with `@id` cross-references	2 hrs	7
SSR/ISR the main content routes	1–3 days	up to 20

The last row is the only expensive one, and it is also the only one that can move a site from "not reachable" to "eligible" on its own.

Three things that do not do what you think

1. The Rich Results Test. It validates syntax. Syntax was never the failure mode. Use it as a lint step before deploying, not as a diagnosis of why you are not cited.

2. llms.txt. A proposal published in late 2024, not a standard, with no major AI provider having confirmed publicly that it reads the file. Adding one costs twenty minutes and breaks nothing. Treating it as the fix is a mistake. If someone tells you otherwise, ask which provider confirmed it.

3. Adding more schema types. If you already have a clean Organization node and a correct Article node and are still uncited, more types will not help. At that point the constraint is whether your content is the clearest available answer to a specific question — which is a writing problem, not a markup problem.

FAQ

Does any of this guarantee I'll be cited?
No. It makes you eligible. Selection depends on whether your content is the clearest, most specific answer available for a given question. That is a writing problem, and no audit solves it for you.

Do AI crawlers run JavaScript?
Mostly no, or unreliably. Googlebot renders. The rest generally fetch raw HTML. Build on the assumption that JS-only content does not exist.

Should I block GPTBot?
Decide per purpose, not wholesale. Blocking GPTBot (training) does not affect ChatGPT search. Blocking OAI-SearchBot does. Blocking Google-Extended does not affect AI Overviews. Three separate decisions.

Is llms.txt supported by anyone?
Not confirmed by any major provider as of writing. Harmless to add, wrong to rely on.

How often do these crawlers revisit?
Popular, frequently updated pages within days. Low-traffic pages within weeks to months. Your own server logs are the only reliable answer.

What score should a small site target?
80+. Sites below 60 usually have one or two blocking issues, not a hundred small ones.

How long until structured data shows an effect?
Recrawl cadence is the limiter, not the markup. Expect a quarter, not a week, and track server-log bot hits rather than inferring from traffic.

What to do this week

Score all eight categories honestly, without giving yourself the benefit of the doubt. "Probably fine" is a zero.

If Categories 01 and 02 come in under 32, fix rendering and robots.txt before anything else. Deploy one Organization graph and verify it with curl, not with DevTools.

Then write down five questions your site should be the answer to, and probe them weekly in ChatGPT, Perplexity and Google. Log which URL gets cited. Over a month, that log tells you more than any audit tool will.

If you want this audit as a scored checklist you can actually fill in — 20 pages, all eight categories with every row and point value, plus three production-ready JSON-LD templates (Organization + WebSite, Article + Person, FAQPage) wired with @id cross-references — it's free: *100-Point AI Search Readiness Checklist***

And if you want the full build rather than the audit — the complete template set, the raw-vs-rendered diff tooling, the log-analysis queries, and the rewrite patterns for making each section survive chunking — that's the reference: *Structured for Answers →***

Earlier in this series: JSON-LD for AI Search: 7 Mistakes That Block Citations — why valid markup can still leave you uncited.

JSON-LD for AI Search: 7 Mistakes That Block Citations

A.R. Hashir — Sat, 24 Oct 2026 10:00:21 +0000

Most structured data audits end the same way: the validator reports zero errors, the Rich Results Test passes, and six months later the site has still never appeared in a single ChatGPT or Perplexity answer.

I have been through this on enough sites to say the uncomfortable part plainly — the problem is almost never syntax.

It is that the JSON-LD describes a page while the retrieval system is looking for an entity. And it is that the page's actual content never reaches the crawler in the first place, so the markup is describing something that was never read.

Seven failures, ordered by what they cost you.

The short version, if you read nothing else:

If WebPage is your only node, you described a container, not a publisher.
If sameAs is missing or points at a 404, the entity cannot be resolved.
If your JSON-LD is injected after hydration, most AI crawlers never see it.
If your body copy renders client-side, nothing else in this article matters.

The rest of this is the long version of those four lines.

1. You described the page, not the entity

The default pattern is a single WebPage block. That tells a machine "this is a page of type WebPage" — which is roughly as useful as telling it "this is a file."

Retrieval and grounding systems need to resolve who you are, not which tag wraps the URL. There is no path from WebPage to a knowledge graph node, because nothing in that block claims to be a thing in the world.

Before

{
  "@context": "https://schema.org",
  "@type": "WebPage",
  "name": "Acme Analytics — Product Page",
  "url": "https://acme.dev/product"
}

After

{
  "@context": "https://schema.org",
  "@graph": [
    {
      "@type": "Organization",
      "@id": "https://acme.dev/#organization",
      "name": "Acme Analytics",
      "url": "https://acme.dev/",
      "logo": {
        "@type": "ImageObject",
        "url": "https://acme.dev/logo-512.png"
      },
      "sameAs": [
        "https://github.com/acme-analytics",
        "https://www.crunchbase.com/organization/acme-analytics"
      ]
    },
    {
      "@type": "SoftwareApplication",
      "@id": "https://acme.dev/product#app",
      "name": "Acme Analytics",
      "applicationCategory": "BusinessApplication",
      "operatingSystem": "Web",
      "publisher": { "@id": "https://acme.dev/#organization" },
      "description": "Privacy-first web analytics with cookieless session tracking.",
      "url": "https://acme.dev/product"
    },
    {
      "@type": "WebPage",
      "@id": "https://acme.dev/product#webpage",
      "url": "https://acme.dev/product",
      "about": { "@id": "https://acme.dev/product#app" }
    }
  ]
}

The @id values are the actual work. They turn three disconnected blocks into one addressable graph, so any node can be referenced from anywhere on the site without repeating the data inside it.

Cost of skipping this: the page can be indexed and still be unciteable, because there is no unambiguous subject to attribute the claim to.

2. `sameAs` is missing, or points at dead profiles

sameAs is the cheapest disambiguation signal that exists, and it is the one most often skipped.

An LLM asked "what is Acme Analytics" has no way to know that your domain is the authority on Acme Analytics. It does know what a GitHub organisation, a Crunchbase entry, and a Wikidata item are. sameAs is how you hand it the bridge between the two.

Four rules that matter:

Only link profiles that exist and are populated. A 404, a private repo, or a profile with no bio is worse than no link — it reads as a failed verification.
Wikipedia and Wikidata beat everything else. If you have either, that is your strongest edge and it belongs first in the array.
Three to eight links is the working range. Forty looks like spam and gets discounted.
Never invent one. A broken sameAs is a trust signal pointing the wrong direction.

Cost of skipping this: every query about your brand becomes a coin flip between your site and someone else's, and whoever has the cleaner identity graph wins.

3. Your JSON-LD is injected after hydration

This is the quiet killer on Next.js, Nuxt, React SPAs, and anything using a head-management library.

If this is how your schema reaches the page:

// invisible to most AI crawlers
useEffect(() => {
  const script = document.createElement('script');
  script.type = 'application/ld+json';
  script.text = JSON.stringify(schema);
  document.head.appendChild(script);
}, [schema]);

…then the crawler has to execute JavaScript, and wait for hydration, and wait for your effect to flush before the markup exists at all. Googlebot will sometimes manage it. Most AI crawlers will not.

The fix is boring: serialize the JSON server-side and render it as static text in the document.

// present in the raw HTML response
const jsonLd = JSON.stringify(schema, null, 2);

return (
  <script
    type="application/ld+json"
    dangerouslySetInnerHTML={{ __html: jsonLd }}
  />
);

Then verify it the only way that counts — not in DevTools, in the actual response:

curl -s https://acme.dev/product | grep -c 'application/ld+json'

If that returns 0 while your browser shows perfectly valid schema, you have found your bug.

Cost of skipping this: you keep maintaining markup that no retrieval pipeline has ever parsed.

4. Your content only exists after JavaScript runs

Same root cause as above, much larger consequence. Schema is metadata. Content is what gets embedded and cited.

If your product copy, documentation body, or article text renders client-side, you are serving an empty shell to every crawler that does not run a browser.

# what a crawler sees vs what a user sees
curl -s -A "GPTBot" https://acme.dev/docs/getting-started | wc -w
curl -s https://acme.dev/docs/getting-started | wc -w

A 30-word difference is a widget. A 900-word difference means your documentation does not exist as far as retrieval is concerned — no matter how clean the JSON-LD on top of it is.

The practical answer is SSR, SSG, or ISR for every route you want cited. Client-side rendering is for application state, not published content.

Cost of skipping this: the largest one on this list, and the only one that silently caps your ceiling no matter what you fix afterward.

5. Valid structure, broken values

Validators check shape, not meaning. Every row below passes validation and still degrades the output.

Field	Broken	Fixed
`datePublished`	`"Oct 24, 2026"`	`"2026-10-24"`
`image`	`"/cover.png"`	`"https://acme.dev/cover.png"`
`@type`	`"Webpage"`	`"WebPage"`
`@type`	`"Article "` (trailing space)	`"Article"`
`name`	`"Acme Analytics \	Best Tools"`
`description`	400-character paragraph	one sentence, 150–160 chars

ISO dates matter more than they look. "Oct 24, 2026" is ambiguous enough that some parsers drop the field entirely, and dateModified is one of the few freshness signals you fully control.

Cost of skipping this: a validator that says "0 errors" while the values quietly fail to resolve.

6. `FAQPage` used as a keyword dump

Two separate mistakes get made here constantly.

Mistake A — malformed structure. mainEntity must be an array of Question objects, each with a nested acceptedAnswer of type Answer. A flat {"question": "...", "answer": "..."} is not Schema.org.

{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "Does JSON-LD improve AI search visibility?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "JSON-LD does not directly improve rankings. It removes ambiguity about what a page and its publisher are, which helps retrieval systems select the page as a source."
      }
    }
  ]
}

Mistake B — mismatched pairs. The marked-up question has to be answered by the visible text directly below it. If the markup asks "How much does it cost?" and the visible answer is about onboarding, you have created a false pair. That is worse than no markup, because the system now has a confident wrong answer attributed to you.

One honest caveat: Google restricted FAQ rich results to authoritative government and health sites in 2023, and removed HowTo rich results entirely. This markup will not win you a SERP feature. What it still does is produce clean, self-contained question-answer chunks that retrieval pipelines can lift without parsing your surrounding prose. That is the reason to keep it — not the rich result.

Cost of skipping this: nothing, if you skip it entirely. The cost is in doing it badly.

7. Conflicting entity data across pages

Bolt on a plugin, add a second one a year later, and you end up with two Organization blocks per page saying different things:

{ "@type": "Organization", "name": "Acme Ltd" }
{ "@type": "Organization", "name": "Acme Analytics Inc.", "url": "https://www.acme.dev" }

To a system that reconciles entities, that is two companies. Consolidation fails silently, and you get no error anywhere.

Pick canonical values once — exact legal name, exact canonical URL, exact logo — and use them in <title>, og:title, and every Organization node on the site. Then dedupe and verify:

# should return exactly 1 per page
curl -s https://acme.dev/ | grep -o '"@type":"Organization"' | wc -l

Cost of skipping this: partial credit on every entity query, forever, with no signal that anything is wrong.

The 5-minute audit order

If you do five things after reading this, do them in this order:

curl your three most important URLs and confirm the body copy is there without JavaScript
Count application/ld+json blocks in the raw HTML — zero means everything else is moot
Check that every sameAs URL returns 200 and has real content on it
Confirm dateModified is ISO 8601 and actually recent
Search for "@type":"Organization" and make sure it appears once, with consistent values

Notice that none of these involve the Rich Results Test. It tells you the markup is well-formed, which was never the question.

FAQ

Does adding JSON-LD get my site cited by ChatGPT?
Not on its own. Citation depends on whether the page is crawlable, whether its content is retrievable without JavaScript, and whether it is the clearest available answer to the question. Schema removes ambiguity about the publisher and the entity; it does not create content authority.

Should I block GPTBot?
GPTBot is OpenAI's training crawler. It is separate from OAI-SearchBot, which handles ChatGPT search, and ChatGPT-User, which fires when a user asks ChatGPT to fetch a URL. Blocking GPTBot does not remove you from ChatGPT search results. Blocking OAI-SearchBot does. Make that a deliberate decision rather than a copied robots.txt snippet.

Does FAQPage markup still do anything?
Not for rich results on most sites. It still produces clean question-answer pairs that are trivial to chunk during retrieval, which is a real but modest benefit.

How long until structured data shows an effect?
Recrawl cadence is the limiter, not the markup. Crawlers revisit popular pages within days and low-traffic pages within weeks to months. Expect a quarter, not a week, and track server-log bot hits rather than trying to infer it from traffic.

Start with the free one. The scored version of this audit — eight categories, 100 points, every row and point value, plus three production-ready JSON-LD templates (Organization + WebSite, Article + Person, FAQPage) already wired with @id cross-references: 100-Point AI Search Readiness Checklist

The full build. If you want the complete template set, the raw-vs-rendered diff tooling, the log-analysis queries, and the rewrite patterns for making each section survive chunking: Structured for Answers

Next in this series: The AI Search Audit: 8 Checks, 100 Points, One Script — the full scored audit, including the 40/60 rule that tells you which two categories gate the other six, and one script that runs most of it in about thirty seconds.