<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jerryton Surya</title>
    <description>The latest articles on DEV Community by Jerryton Surya (@jerryton_surya_cfe00892b2).</description>
    <link>https://dev.to/jerryton_surya_cfe00892b2</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4249516%2F9cea0a26-860a-4cc4-996c-3ebe8f548afe.png</url>
      <title>DEV Community: Jerryton Surya</title>
      <link>https://dev.to/jerryton_surya_cfe00892b2</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jerryton_surya_cfe00892b2"/>
    <language>en</language>
    <item>
      <title>Making Your Site Visible to AI Search Engines: A Developer's Checklist for GEO</title>
      <dc:creator>Jerryton Surya</dc:creator>
      <pubDate>Tue, 17 Nov 2026 09:52:59 +0000</pubDate>
      <link>https://dev.to/jerryton_surya_cfe00892b2/making-your-site-visible-to-ai-search-engines-a-developers-checklist-for-geo-4fcg</link>
      <guid>https://dev.to/jerryton_surya_cfe00892b2/making-your-site-visible-to-ai-search-engines-a-developers-checklist-for-geo-4fcg</guid>
      <description>&lt;p&gt;More and more users get their answers from ChatGPT, Perplexity, Gemini or Google's AI Overviews instead of clicking a blue link. Pew Research found people clicked a traditional search result in about 8% of visits with an AI summary, versus 15% without one.&lt;/p&gt;

&lt;p&gt;Marketers call the response &lt;strong&gt;generative engine optimization (GEO)&lt;/strong&gt;. A lot of GEO is content strategy, but a surprising amount is plain engineering: can AI crawlers reach your pages, read them, and understand them?&lt;/p&gt;

&lt;p&gt;Here's the developer side of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Check your robots.txt for AI crawlers
&lt;/h2&gt;

&lt;p&gt;Many sites block AI bots without knowing it, through a blanket rule, a CDN "bot protection" setting, or a copy-pasted robots.txt from 2023.&lt;/p&gt;

&lt;p&gt;These are the common user agents to know:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;User agent&lt;/th&gt;
&lt;th&gt;Operator&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;GPTBot&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;Model training&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;OAI-SearchBot&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;ChatGPT search results&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ChatGPT-User&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;Fetches pages when a user asks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;ClaudeBot&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;Crawling&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;PerplexityBot&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Perplexity&lt;/td&gt;
&lt;td&gt;Search index&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Google-Extended&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Google&lt;/td&gt;
&lt;td&gt;Gemini training opt-out token&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A setup that allows AI search but opts out of training might look like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User-agent: OAI-SearchBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: GPTBot
Disallow: /
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two gotchas:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Google-Extended&lt;/code&gt; does &lt;strong&gt;not&lt;/strong&gt; control whether you appear in AI Overviews. Those come from regular Googlebot crawling.&lt;/li&gt;
&lt;li&gt;Your CDN or WAF can block bots even if robots.txt allows them. Check Cloudflare's (or your provider's) bot settings too.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  2. Don't hide your content behind JavaScript
&lt;/h2&gt;

&lt;p&gt;Googlebot renders JavaScript, but many AI crawlers fetch the raw HTML and move on. If your article body only appears after client-side hydration, some AI engines see an empty page.&lt;/p&gt;

&lt;p&gt;Quick test:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -s -A &lt;span class="s2"&gt;"PerplexityBot"&lt;/span&gt; https://yoursite.com/your-post &lt;span class="o"&gt;|&lt;/span&gt; grep -c &lt;span class="s2"&gt;"&amp;lt;p&amp;gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If that returns 0 (or very few), your content probably isn't in the initial HTML. Fixes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Use SSR or static generation (Next.js, Astro, Nuxt, etc.) for content pages&lt;/li&gt;
&lt;li&gt;Make sure the main text, headings and key facts are in the server response&lt;/li&gt;
&lt;li&gt;Don't lazy-load the actual article body&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  3. Check your server logs: are AI bots visiting at all?
&lt;/h2&gt;

&lt;p&gt;Before optimizing anything, find out who is actually crawling you:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;grep -Ei &lt;span class="s2"&gt;"gptbot|oai-searchbot|chatgpt-user|claudebot|perplexitybot"&lt;/span&gt; access.log \
  &lt;span class="o"&gt;|&lt;/span&gt; awk &lt;span class="s1"&gt;'{print $1, $7, $9}'&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; sort &lt;span class="o"&gt;|&lt;/span&gt; uniq -c &lt;span class="o"&gt;|&lt;/span&gt; sort -rn &lt;span class="o"&gt;|&lt;/span&gt; head -20
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Look for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;No hits at all:&lt;/strong&gt; something is blocking them&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lots of 403s or 429s:&lt;/strong&gt; your WAF or rate limiter is rejecting them&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hits only on your homepage:&lt;/strong&gt; your internal linking or sitemap needs work&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  4. Add structured data
&lt;/h2&gt;

&lt;p&gt;JSON-LD helps machines understand what a page &lt;em&gt;is&lt;/em&gt;: an article, a product, a how-to, an organization. At minimum, add &lt;code&gt;Article&lt;/code&gt; and &lt;code&gt;Organization&lt;/code&gt; schema:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;script&lt;/span&gt; &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s"&gt;"application/ld+json"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="s2"&gt;"@context"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"https://schema.org"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"@type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Article"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"headline"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Your article title"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"datePublished"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-17"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"dateModified"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-17"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"author"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s2"&gt;"@type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Person"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Your Name"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="s2"&gt;"publisher"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="s2"&gt;"@type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Organization"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Your Company"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/script&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Keep &lt;code&gt;dateModified&lt;/code&gt; accurate. Freshness matters for AI answers on fast-moving topics.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Structure content so it can be quoted
&lt;/h2&gt;

&lt;p&gt;AI engines pull out passages, not whole pages. Make yours easy to pull out:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Use question-style headings (&lt;code&gt;## What is X?&lt;/code&gt;) followed by a direct 1–2 sentence answer&lt;/li&gt;
&lt;li&gt;Use semantic HTML: real &lt;code&gt;&amp;lt;h2&amp;gt;&lt;/code&gt;/&lt;code&gt;&amp;lt;h3&amp;gt;&lt;/code&gt;, &lt;code&gt;&amp;lt;table&amp;gt;&lt;/code&gt; for tabular data, &lt;code&gt;&amp;lt;ol&amp;gt;&lt;/code&gt; for steps&lt;/li&gt;
&lt;li&gt;Include specific facts, numbers and cited sources&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That last point has research behind it: a KDD 2024 paper on GEO found that adding citations, quotations and statistics could raise visibility in AI-generated answers by up to 40%.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. Consider llms.txt (but don't count on it)
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;llms.txt&lt;/code&gt; is a proposed standard: a markdown file at your site root that gives LLMs a curated map of your most important content. It's cheap to add, but support from the major AI engines is unclear, so treat it as a nice-to-have, not a fix.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. Measure it
&lt;/h2&gt;

&lt;p&gt;You can't use Search Console to see how often ChatGPT cites you. That's where GEO tools come in. They run a set of prompts across AI engines and track whether your brand gets mentioned and which sources get cited.&lt;/p&gt;

&lt;p&gt;If you're comparing options, I wrote up a detailed breakdown of pricing, engine coverage and team fit: &lt;strong&gt;&lt;a href="https://www.blazly.ai/post/best-generative-engine-optimization-tools" rel="noopener noreferrer"&gt;11 Best Generative Engine Optimization Tools&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;On a budget? Start by checking your server logs and manually asking a few AI engines your target questions every week. It's crude, but it works.&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR checklist
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;[ ] robots.txt allows the AI search bots you want&lt;/li&gt;
&lt;li&gt;[ ] CDN/WAF isn't silently blocking them&lt;/li&gt;
&lt;li&gt;[ ] Content is in the initial HTML (SSR/SSG)&lt;/li&gt;
&lt;li&gt;[ ] Server logs show AI crawlers getting 200s&lt;/li&gt;
&lt;li&gt;[ ] Article + Organization JSON-LD in place&lt;/li&gt;
&lt;li&gt;[ ] Question-based headings with direct answers&lt;/li&gt;
&lt;li&gt;[ ] Facts, stats and sources in your content&lt;/li&gt;
&lt;li&gt;[ ] Some way to track AI visibility over time&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What's your setup? Are you allowing AI crawlers or blocking them? Let me know in the comments.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Disclosure: I work at Blazly, one of the tools covered in the linked comparison.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>seo</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
