A client asks: "Why doesn't ChatGPT know about us?" You open the site it loads fast, ranks okay on Google, looks fine. This is the debugging runbook for exactly that ticket: a triage order that finds the failure in minutes instead of guesswork, with the commands to prove each diagnosis before you touch anything.
The mental model: AI answers pull from two pipelines, a retrieval pipeline (live web search, heavily Bing-flavored for ChatGPT) and a model-memory pipeline (training data). You can only debug the first one, so that's where the runbook lives. Work top-down; each check gates the next.
Check 0: Reproduce the bug properly
Before touching config, document the failure like any bug. Ask ChatGPT (with search enabled) three things and save the outputs: the category question ("best X in CITY"), the brand question ("what is CLIENTNAME"), and a direct URL question ("what's on https://client.com"). The pattern tells you the layer: wrong facts about the brand = data-consistency problem; total absence from category answers = retrieval or authority problem; can't describe the URL = crawl/index problem. Now debug in that order of severity.
Check 1: Can the bots physically fetch the site?
Don't read robots.txt and assume. Fetch as the bots fetch:
bashfor UA in "OAI-SearchBot/1.0" "PerplexityBot/1.0" "ChatGPT-User/1.0"; do
echo "== $UA =="
curl -s -o /dev/null -w "%{http_code} %{size_download} bytes\n" \
-A "$UA" https://client.com/
done
Interpret: 200 with a normal byte size = pass. 403 = something upstream (WAF, Cloudflare bot rules, security plugin) is rejecting the agent even if robots.txt looks clean, check the firewall event logs, not the config files. 200 with a tiny payload = you're serving an empty JS shell (see Check 4). Also confirm robots.txt itself isn't the blocker:
bashcurl -s https://client.com/robots.txt | grep -A1 -iE "gptbot|oai|perplexity|claude"
Remember the split: OAI-SearchBot / ChatGPT-User power live answers; GPTBot is training. A "block AI" tutorial that nuked all of them took the client out of the answers business.
Check 2: Does the retrieval index know the site exists?
ChatGPT search leans substantially on Bing. Google-only SEO histories fail here constantly:
site:client.com <- run on bing.com, not google.com
Zero results is your smoking gun for "invisible in category answers despite a healthy site." Fix: Bing Webmaster Tools (import from GSC in two clicks), submit the sitemap, and wire IndexNow so future updates propagate in hours:
bashcurl "https://api.indexnow.org/indexnow?url=https://client.com/&key=YOUR_KEY"
Log the date; you'll want it for the before/after report.
Check 3: Do the facts survive machine parsing?
Retrieval finding the page is necessary, not sufficient the facts have to be extractable. Grep the raw HTML for the things the client wants AI to know:
bashcurl -s -A "OAI-SearchBot/1.0" https://client.com/contact/ \
| grep -ciE "0335|\+92|@client\.com"
Zero matches on the contact page means the phone/email exist only in an image, an iframe, or post-render JavaScript. Same test on the pricing page for amounts. Every zero here is a fact AI cannot cite, period.
Then validate the structured-data layer at validator.schema.org — and check for the duplicate schema failure mode: multiple plugins each emitting an Organization block with different values. Conflicting JSON-LD on one page is actively worse than none; consolidate to one source of truth per type.
Check 4: Is the content server-rendered?
If Check 1 returned 200 with suspiciously few bytes, or Check 3 greps clean HTML but the browser shows content:
bashcurl -s https://client.com/ | wc -c # raw HTML size
curl -s https://client.com/ | grep -c "<h2" # structural content present?
A 4KB shell with zero headings means the site only exists after client-side rendering, fine for humans, a void for a chunk of retrieval systems. Prescription: SSR/SSG for money pages, or at minimum pre-rendering. On WordPress this failure usually traces to an "optimize everything" plugin preset deferring content itself; on SPAs it's architectural.
Check 5: Is the entity coherent across the web?
The subtle one. Pull the business's NAP (name/address/phone) from the site, the Google Business Profile, Facebook, and the top three directory hits, and diff them mentally. Different phone formats, two name spellings, a stale address each divergence lowers machine confidence in every version. There's no command for this one; it's an audit table and an afternoon of edits. It fixes the "ChatGPT describes us wrongly" class of bug more often than anything technical.
Check 6: The authority floor
If Checks 1-5 all pass and the brand still loses category questions to competitors: the problem isn't parseability, it's evidence. Count Google reviews (volume + last-90-day velocity + owner replies), third-party mentions, consistent citations. Machines making recommendations behave like cautious buyers, a technically perfect site with eight dusty reviews is legible but not recommendable. This layer is marketing's ticket, not yours; hand it over with the data.
The triage table (pin this)
When you're on a call and need the diagnosis order from symptoms alone:
Client saysMost likely checkFirst command"ChatGPT describes us wrongly"5 (entity consistency)NAP diff across surfaces"We never appear for category questions"2 then 6site: on Bing"It can't even summarize our URL"1 (fetch)curl as OAI-SearchBot"Our prices/services never get quoted"3 then 4grep facts in raw HTML"It worked before, stopped recently"1 (regression)check WAF event log + plugin changelogs
That last row deserves emphasis because it's the ticket you'll see most after the first fix: things that were working break silently. The usual suspects, in order of frequency I've encountered: a security-plugin update re-writing robots.txt with fresh AI blocks; Cloudflare's "Block AI bots" toggle getting enabled during an unrelated security review; a caching plugin's preset change that starts serving crawlers the deferred-JS shell; and an SEO plugin update that duplicates the schema output. None of these announce themselves. This is why the weekly snapshot cron from my earlier post or even a monthly manual re-run of Checks 1 and 3 belongs in the maintenance contract, not the "if we remember" pile.
Close the loop
One last professional habit: end every engagement with a one-page handoff report, the three Check-0 outputs from before, the same three after, the dated fix list, and the two recurring checks (bot-fetch snapshot, Bing site: count) whoever inherits the site must keep running. Half the value of this runbook is the fix; the other half is making the invisible layer visible to the client, so the next silent regression gets caught in a week instead of a quarter.
Re-run Check 0's three questions weekly and log who gets named. Retrieval-side fixes typically surface in answers within 2–5 weeks. The log is your regression test and the most persuasive artifact you'll ever attach to an invoice.
Full non-technical breakdown of all seven failure classes (for the client-facing version of this ticket)
I work with DigiMSM (digimsm.com) Pakistan's first AI-powered digital marketing agency. We run this exact runbook as a free AI Visibility Audit. War stories about bots and WAFs welcome in the comments.
Top comments (0)