<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Dejun (Alice) Chen</title>
    <description>The latest articles on DEV Community by Dejun (Alice) Chen (@alicexixi85).</description>
    <link>https://dev.to/alicexixi85</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4237231%2Ffa5100af-6119-426f-b390-b3ff8a407203.png</url>
      <title>DEV Community: Dejun (Alice) Chen</title>
      <link>https://dev.to/alicexixi85</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/alicexixi85"/>
    <language>en</language>
    <item>
      <title>Is your site invisible to ChatGPT Search? Five checks you can run in a minute</title>
      <dc:creator>Dejun (Alice) Chen</dc:creator>
      <pubDate>Tue, 10 Nov 2026 13:55:21 +0000</pubDate>
      <link>https://dev.to/alicexixi85/is-your-site-invisible-to-chatgpt-search-five-checks-you-can-run-in-a-minute-1a18</link>
      <guid>https://dev.to/alicexixi85/is-your-site-invisible-to-chatgpt-search-five-checks-you-can-run-in-a-minute-1a18</guid>
      <description>&lt;p&gt;A site can rank well on Google and still never be cited by ChatGPT Search, Perplexity or Claude. When I audit sites for this, the cause is usually not content quality. It is one of a handful of technical mistakes that take minutes to fix once you see them.&lt;/p&gt;

&lt;p&gt;Here are the five checks I run first, and a free open-source MCP server that runs them for you from Claude Desktop or Cursor.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Are you blocking the &lt;em&gt;search&lt;/em&gt; crawlers while trying to block &lt;em&gt;training&lt;/em&gt;?
&lt;/h2&gt;

&lt;p&gt;AI companies run two kinds of crawlers:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;th&gt;Crawlers&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Live AI search (cites your page in answers)&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;OAI-SearchBot&lt;/code&gt;, &lt;code&gt;PerplexityBot&lt;/code&gt;, &lt;code&gt;Claude-SearchBot&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Model training&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;GPTBot&lt;/code&gt;, &lt;code&gt;ClaudeBot&lt;/code&gt;, &lt;code&gt;Google-Extended&lt;/code&gt;, &lt;code&gt;CCBot&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Blocking training crawlers is a legitimate choice. The problem is a blanket rule, or a copy-pasted list, that blocks the search crawlers too. Then the page simply cannot be cited.&lt;/p&gt;

&lt;p&gt;Check &lt;code&gt;robots.txt&lt;/code&gt; &lt;strong&gt;for the specific path&lt;/strong&gt; you care about (&lt;code&gt;/pricing&lt;/code&gt;, &lt;code&gt;/docs/...&lt;/code&gt;), not just the homepage, because path rules differ.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Is your CDN blocking them anyway?
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;robots.txt&lt;/code&gt; is only a request. Many CDNs and WAFs have bot-management settings that return 403 to AI user agents regardless of what &lt;code&gt;robots.txt&lt;/code&gt; says. If an automated check of your site comes back blocked or challenged, look at your CDN's bot settings before anything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Does your canonical tag point to the right host?
&lt;/h2&gt;

&lt;p&gt;A surprisingly common bug: the site template ships with &lt;code&gt;&amp;lt;link rel="canonical" href="https://example.com/"&amp;gt;&lt;/code&gt; or points every page at the homepage. Search and answer engines then attribute your page to a different URL or drop it. Same for &lt;code&gt;og:url&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Is your structured data in the HTML, or only after JavaScript?
&lt;/h2&gt;

&lt;p&gt;If your JSON-LD (&lt;code&gt;Organization&lt;/code&gt;, &lt;code&gt;Product&lt;/code&gt;, &lt;code&gt;SoftwareApplication&lt;/code&gt;, &lt;code&gt;FAQPage&lt;/code&gt;) is injected client-side, non-rendering fetchers never see it. Look at the raw HTML response. If the page is an almost-empty shell until JavaScript runs, that is worth fixing for every crawler, not just AI ones.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Do you have a sitemap and an &lt;code&gt;llms.txt&lt;/code&gt; that actually return what they claim?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;robots.txt&lt;/code&gt; should declare a real sitemap URL (I keep finding &lt;code&gt;Sitemap: https://your-domain.com/...&lt;/code&gt; placeholders left from templates).&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;/sitemap.xml&lt;/code&gt; should return XML, not your SPA's HTML fallback page.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;/llms.txt&lt;/code&gt; is a newer convention: a short Markdown index of your key pages for AI agents. If you add one, make sure it returns Markdown with a title, sections and links, not an HTML 404 page.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Running the checks from your AI assistant
&lt;/h2&gt;

&lt;p&gt;I packaged these checks as a small open-source MCP server: &lt;a href="https://github.com/alicexixi85/geo-seo-mcp-pro" rel="noopener noreferrer"&gt;GEO &amp;amp; SEO Readiness MCP — Lite&lt;/a&gt;. It is one Python file plus a tiny stdio helper, no dependencies, MIT licensed.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;"mcpServers"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;"geo-seo-readiness"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="nl"&gt;"command"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"python3"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="nl"&gt;"args"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"/absolute/path/to/geo-seo-mcp-pro/server.py"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then ask: &lt;em&gt;"Run a quick readiness report for &lt;a href="https://yoursite.com" rel="noopener noreferrer"&gt;https://yoursite.com&lt;/a&gt; and tell me what to fix first."&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;It returns PASS / WARN / FAIL per check and a short "fix first" list. If a site blocks the request, it reports &lt;code&gt;UNMEASURED&lt;/code&gt; instead of guessing, and it deliberately does not invent an overall score.&lt;/p&gt;

&lt;p&gt;You can also run it without an MCP client:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;python3 server.py --check https://yoursite.com
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Limits:&lt;/strong&gt; it reads the initial HTML only (no JavaScript rendering), and it cannot see CDN bot rules directly. Technical readiness is necessary, not sufficient; it does not guarantee citations or rankings.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I built this tool. There is also a paid edition with rendering and white-label PDF reports for agencies, but the open-source server above is complete on its own.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>seo</category>
      <category>ai</category>
      <category>mcp</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
