<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Nadeem Akram</title>
    <description>The latest articles on DEV Community by Nadeem Akram (@nadeemakram035).</description>
    <link>https://dev.to/nadeemakram035</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4141878%2Fe40cae63-9e1a-4d69-b9c3-9fab941b4027.jpg</url>
      <title>DEV Community: Nadeem Akram</title>
      <link>https://dev.to/nadeemakram035</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/nadeemakram035"/>
    <language>en</language>
    <item>
      <title>Free PDF Tools That Never Upload Your Files</title>
      <dc:creator>Nadeem Akram</dc:creator>
      <pubDate>Mon, 12 Oct 2026 15:08:32 +0000</pubDate>
      <link>https://dev.to/nadeemakram035/free-pdf-tools-that-never-upload-your-files-40ck</link>
      <guid>https://dev.to/nadeemakram035/free-pdf-tools-that-never-upload-your-files-40ck</guid>
      <description>&lt;p&gt;&lt;em&gt;A note for dev.to readers: these tools use &lt;a href="https://pdf-lib.js.org/" rel="noopener noreferrer"&gt;pdf-lib&lt;/a&gt; to edit files, &lt;a href="https://mozilla.github.io/pdf.js/" rel="noopener noreferrer"&gt;pdf.js&lt;/a&gt; to render pages, and &lt;a href="https://qpdf.readthedocs.io/" rel="noopener noreferrer"&gt;qpdf&lt;/a&gt; compiled to WebAssembly for AES-256 encryption, all running client-side. Nothing is uploaded.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Contracts, bank statements, payslips, medical letters, ID scans: the PDFs we most often need to merge, shrink or lock are also the ones we least want to hand to a stranger. Yet the usual way to do it online is to upload the file to a website, let its server process it, and download the result. This guide explains what that actually involves, and how to do the same jobs without your file ever leaving your computer.&lt;/p&gt;

&lt;h2&gt;
  
  
  What happens when you "upload" a PDF
&lt;/h2&gt;

&lt;p&gt;A typical online PDF tool copies your document to its server, works on it there, and keeps the result long enough for you to download it. Reputable services delete files after an hour or so, but you are relying on their word, their security, and every company they use for storage and logging. For a restaurant menu that is fine. For a signed contract or a passport scan, it is a real risk that is easy to avoid.&lt;/p&gt;

&lt;h2&gt;
  
  
  The alternative: tools that run in your browser
&lt;/h2&gt;

&lt;p&gt;Modern browsers are powerful enough to read and rewrite PDF files themselves. SEOpeck's &lt;a href="https://seopeck.com/category/pdf-tools" rel="noopener noreferrer"&gt;PDF Tools&lt;/a&gt; work this way: the page loads the program, and your file is opened, processed and saved by your own browser. Nothing is sent to our server, so there is nothing for us to store, leak or delete. You can check this yourself: open the browser's developer tools, switch to the Network tab and run a tool. You will not see your file go anywhere.&lt;/p&gt;

&lt;p&gt;There is a side benefit, too. Because there is no upload or download round trip, large files are often faster to process locally than online, and the tools keep working on a slow connection once the page has loaded.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you can do
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Combine and rearrange
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/merge-pdf" rel="noopener noreferrer"&gt;Merge PDF&lt;/a&gt; joins several PDFs into one, in the order you choose.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/split-pdf" rel="noopener noreferrer"&gt;Split PDF&lt;/a&gt; breaks a document into separate files by page range.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/organize-pdf" rel="noopener noreferrer"&gt;Organize PDF&lt;/a&gt; lets you drag pages into a new order, and &lt;a href="https://seopeck.com/tool/rotate-pdf" rel="noopener noreferrer"&gt;Rotate PDF&lt;/a&gt; fixes sideways scans.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/delete-pdf-pages" rel="noopener noreferrer"&gt;Delete PDF Pages&lt;/a&gt; and &lt;a href="https://seopeck.com/tool/extract-pdf-pages" rel="noopener noreferrer"&gt;Extract PDF Pages&lt;/a&gt; remove the pages you do not need, or pull out the ones you do.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Make files smaller
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://seopeck.com/tool/compress-pdf" rel="noopener noreferrer"&gt;Compress PDF&lt;/a&gt; reduces file size so a document fits under an email attachment limit or an upload form's cap. Scanned documents usually shrink the most, because they are really a stack of photos.&lt;/p&gt;

&lt;h3&gt;
  
  
  Convert
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/pdf-to-jpg" rel="noopener noreferrer"&gt;PDF to JPG&lt;/a&gt; and &lt;a href="https://seopeck.com/tool/pdf-to-png" rel="noopener noreferrer"&gt;PDF to PNG&lt;/a&gt; turn each page into an image.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/jpg-to-pdf" rel="noopener noreferrer"&gt;JPG to PDF&lt;/a&gt; and &lt;a href="https://seopeck.com/tool/png-to-pdf" rel="noopener noreferrer"&gt;PNG to PDF&lt;/a&gt; turn photos and screenshots into a single PDF.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/pdf-to-text" rel="noopener noreferrer"&gt;PDF to Text&lt;/a&gt; pulls out the text so you can copy, search or reuse it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Finish and sign
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/add-page-numbers-to-pdf" rel="noopener noreferrer"&gt;Add Page Numbers&lt;/a&gt; and &lt;a href="https://seopeck.com/tool/watermark-pdf" rel="noopener noreferrer"&gt;Watermark PDF&lt;/a&gt; for reports and drafts.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/sign-pdf" rel="noopener noreferrer"&gt;Sign PDF&lt;/a&gt; to draw or type a signature and place it on the page.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/pdf-metadata-editor" rel="noopener noreferrer"&gt;PDF Metadata Editor&lt;/a&gt; to change or clear the title, author and other hidden details before you share a file.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Password-protect a PDF
&lt;/h2&gt;

&lt;p&gt;The newest additions are &lt;a href="https://seopeck.com/tool/protect-pdf" rel="noopener noreferrer"&gt;Protect PDF&lt;/a&gt; and &lt;a href="https://seopeck.com/tool/unlock-pdf" rel="noopener noreferrer"&gt;Unlock PDF&lt;/a&gt;. Protect PDF encrypts a document with AES-256, the same standard used by banks and by Adobe Acrobat, so it cannot be opened without the password. You can also switch off printing, copying or editing for people who do open it.&lt;/p&gt;

&lt;p&gt;A few tips for passwords on documents:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Use a long passphrase&lt;/strong&gt;, such as four unrelated words, rather than a short word with a number on the end. The encryption is only as strong as the password. Our &lt;a href="https://seopeck.com/tool/password-generator" rel="noopener noreferrer"&gt;Password Generator&lt;/a&gt; can make one for you.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Send the password separately.&lt;/strong&gt; If the PDF goes by email, send the password by text message or say it on a call. A password in the same email protects nothing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep a copy.&lt;/strong&gt; There is no "forgot password" for an encrypted PDF. If the password is lost, the file is effectively lost too.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Restrictions are a polite request.&lt;/strong&gt; Blocking printing or copying stops casual reuse in normal PDF readers, but it is not a strong lock. The open password is the real protection.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Remove a password you know
&lt;/h2&gt;

&lt;p&gt;Password-protected files are a nuisance when you have to open them every day, such as your own bank statements. &lt;a href="https://seopeck.com/tool/unlock-pdf" rel="noopener noreferrer"&gt;Unlock PDF&lt;/a&gt; takes the file and its password and saves a copy without encryption. It will not guess or crack a password you do not have; that is deliberate. Only remove protection from documents you own or are entitled to open.&lt;/p&gt;

&lt;h2&gt;
  
  
  When a server-based tool is still the right choice
&lt;/h2&gt;

&lt;p&gt;Browser tools are limited by your device. A very large PDF of several hundred megabytes may be slow on an older phone, and some jobs, such as turning a scanned image into editable text with OCR or converting to Word with perfect layout, still need heavier software. For the everyday jobs above, though, there is no need to upload anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;If a document contains anything personal, prefer a PDF tool that works inside your browser. All of SEOpeck's &lt;a href="https://seopeck.com/category/pdf-tools" rel="noopener noreferrer"&gt;PDF Tools&lt;/a&gt; are free, need no account, add no watermark, and never send your file to our server.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Originally published on &lt;a href="https://seopeck.com/blog/free-pdf-tools-without-uploading" rel="noopener noreferrer"&gt;SEOpeck&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>pdf</category>
      <category>privacy</category>
      <category>webdev</category>
      <category>javascript</category>
    </item>
    <item>
      <title>A 5-minute SEO checklist for your homepage</title>
      <dc:creator>Nadeem Akram</dc:creator>
      <pubDate>Sat, 26 Sep 2026 10:16:46 +0000</pubDate>
      <link>https://dev.to/nadeemakram035/a-5-minute-seo-checklist-for-your-homepage-2eeo</link>
      <guid>https://dev.to/nadeemakram035/a-5-minute-seo-checklist-for-your-homepage-2eeo</guid>
      <description>&lt;h1&gt;
  
  
  A 5-minute SEO checklist for your homepage
&lt;/h1&gt;

&lt;p&gt;Your homepage is usually the page with the most links pointing at it, so it is the one search engines and social networks look at first. You don't need an audit tool with a monthly fee to catch the most common problems. Five checks cover most of them, and each takes about a minute.&lt;/p&gt;

&lt;p&gt;Open your homepage, right-click, choose &lt;strong&gt;View page source&lt;/strong&gt;, and search (Ctrl+F) for the tags below.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. One H1 that says what the page is about
&lt;/h2&gt;

&lt;p&gt;Search for &lt;code&gt;&amp;lt;h1&lt;/code&gt;. You want exactly one, and it should describe the page in plain words.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;h1&amp;gt;&lt;/span&gt;Free SEO and developer tools, no sign-up&lt;span class="nt"&gt;&amp;lt;/h1&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Common problems: the H1 is missing because the logo is the only heading, there are several H1s because a theme reused the tag for styling, or the H1 just says "Welcome" or "Home". Headings help search engines and screen-reader users understand the page, so they should be about the content, not the layout.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. A meta description you wrote yourself
&lt;/h2&gt;

&lt;p&gt;Search for &lt;code&gt;name="description"&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;meta&lt;/span&gt; &lt;span class="na"&gt;name=&lt;/span&gt;&lt;span class="s"&gt;"description"&lt;/span&gt; &lt;span class="na"&gt;content=&lt;/span&gt;&lt;span class="s"&gt;"130+ free SEO and developer tools: check a page's SEO, test robots.txt, compress images and more. No sign-up."&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Google doesn't use the description for ranking, and it sometimes rewrites the snippet. A good description still helps, because it is often what people read in the search results before deciding to click. If the tag is missing, Google picks some text from the page itself. Keep it to about 150 to 160 characters and describe what the visitor gets.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. A canonical URL, and no stray "noindex"
&lt;/h2&gt;

&lt;p&gt;Search for &lt;code&gt;rel="canonical"&lt;/code&gt; and for &lt;code&gt;noindex&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;link&lt;/span&gt; &lt;span class="na"&gt;rel=&lt;/span&gt;&lt;span class="s"&gt;"canonical"&lt;/span&gt; &lt;span class="na"&gt;href=&lt;/span&gt;&lt;span class="s"&gt;"https://example.com/"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The canonical tag tells search engines which version of the URL is the main one (https or http, with or without &lt;code&gt;www&lt;/code&gt;, with or without tracking parameters). It should point to the page's own preferred address.&lt;/p&gt;

&lt;p&gt;A &lt;code&gt;noindex&lt;/code&gt; in a robots meta tag tells search engines to leave the page out of their results. It often gets left behind after a site moves from staging to production, and it will quietly keep a homepage out of Google. It can also come as an &lt;code&gt;X-Robots-Tag&lt;/code&gt; HTTP header, which doesn't appear in the page source, so check the response headers too.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Alt text on every image that carries meaning
&lt;/h2&gt;

&lt;p&gt;Search for &lt;code&gt;&amp;lt;img&lt;/code&gt; and look at the &lt;code&gt;alt&lt;/code&gt; attributes.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;img&lt;/span&gt; &lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"dashboard.png"&lt;/span&gt; &lt;span class="na"&gt;alt=&lt;/span&gt;&lt;span class="s"&gt;"SEO Checker report with a score of 92 out of 100"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;img&lt;/span&gt; &lt;span class="na"&gt;src=&lt;/span&gt;&lt;span class="s"&gt;"divider.svg"&lt;/span&gt; &lt;span class="na"&gt;alt=&lt;/span&gt;&lt;span class="s"&gt;""&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Alt text is read by screen readers and helps image search understand the picture. Describe what the image shows. Purely decorative images should have an empty &lt;code&gt;alt=""&lt;/code&gt; so screen readers skip them. Leaving out the attribute entirely is the one option to avoid.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Open Graph tags, so shared links look right
&lt;/h2&gt;

&lt;p&gt;Search for &lt;code&gt;og:&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;meta&lt;/span&gt; &lt;span class="na"&gt;property=&lt;/span&gt;&lt;span class="s"&gt;"og:title"&lt;/span&gt; &lt;span class="na"&gt;content=&lt;/span&gt;&lt;span class="s"&gt;"SEOpeck: free SEO and developer tools"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;meta&lt;/span&gt; &lt;span class="na"&gt;property=&lt;/span&gt;&lt;span class="s"&gt;"og:description"&lt;/span&gt; &lt;span class="na"&gt;content=&lt;/span&gt;&lt;span class="s"&gt;"130+ free tools, no sign-up."&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;meta&lt;/span&gt; &lt;span class="na"&gt;property=&lt;/span&gt;&lt;span class="s"&gt;"og:image"&lt;/span&gt; &lt;span class="na"&gt;content=&lt;/span&gt;&lt;span class="s"&gt;"https://example.com/og-image.png"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Facebook, LinkedIn, Slack, WhatsApp and many other apps use these tags to build the preview card when someone shares your link. Without them you get a bare URL or a random image from the page. An image of 1200 × 630 px works well almost everywhere.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check all five at once
&lt;/h2&gt;

&lt;p&gt;If you'd rather not read page source, I built a free &lt;a href="https://seopeck.com/tool/seo-checker" rel="noopener noreferrer"&gt;SEO Checker&lt;/a&gt; that tests all five of these on any URL. It also checks HTTPS, redirects, response time, subheadings, word count, the mobile viewport and structured data, and gives a score out of 100 with a short fix for each problem. No sign-up needed.&lt;/p&gt;

&lt;p&gt;Run it on your homepage, fix anything marked as a problem, and then run it on your two or three most important pages. That's usually the fastest SEO win a small site can get.&lt;/p&gt;

</description>
      <category>seo</category>
      <category>webdev</category>
      <category>beginners</category>
      <category>html</category>
    </item>
    <item>
      <title>Should You Block AI Crawlers? A Plain Guide to robots.txt and AI Bots</title>
      <dc:creator>Nadeem Akram</dc:creator>
      <pubDate>Thu, 24 Sep 2026 21:07:43 +0000</pubDate>
      <link>https://dev.to/nadeemakram035/should-you-block-ai-crawlers-a-plain-guide-to-robotstxt-and-ai-bots-2ham</link>
      <guid>https://dev.to/nadeemakram035/should-you-block-ai-crawlers-a-plain-guide-to-robotstxt-and-ai-bots-2ham</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://seopeck.com/blog/should-you-block-ai-crawlers-robots-txt" rel="noopener noreferrer"&gt;seopeck.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Alongside Googlebot and Bingbot, most websites are now visited by crawlers from AI companies. Some collect text to train language models; others fetch pages so an AI assistant can answer a question and cite its source. Site owners are increasingly asking the same question: should I let them in?&lt;/p&gt;

&lt;p&gt;There is no single right answer. It depends on what you publish and what you want in return. This guide explains the main crawlers, what they are used for, and how to control them.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two kinds of AI crawler
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Training crawlers&lt;/strong&gt; collect content to build and improve AI models. Examples are OpenAI's &lt;code&gt;GPTBot&lt;/code&gt;, Anthropic's &lt;code&gt;ClaudeBot&lt;/code&gt;, Meta's &lt;code&gt;Meta-ExternalAgent&lt;/code&gt; and Common Crawl's &lt;code&gt;CCBot&lt;/code&gt;, whose open dataset is used by many AI projects. Blocking these keeps your future content out of new training runs. It does not remove anything that was collected before.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Search and user crawlers&lt;/strong&gt; fetch pages in response to what people ask right now. Examples are &lt;code&gt;OAI-SearchBot&lt;/code&gt; and &lt;code&gt;ChatGPT-User&lt;/code&gt; from OpenAI, &lt;code&gt;Claude-SearchBot&lt;/code&gt; and &lt;code&gt;Claude-User&lt;/code&gt; from Anthropic, and &lt;code&gt;PerplexityBot&lt;/code&gt;. These are the bots that can show your page as a source and send visitors to it.&lt;/p&gt;

&lt;p&gt;There are also &lt;strong&gt;control tokens&lt;/strong&gt; that are not separate crawlers at all. &lt;code&gt;Google-Extended&lt;/code&gt; lets you opt out of your content being used for Google's Gemini models without affecting Google Search. &lt;code&gt;Applebot-Extended&lt;/code&gt; does the same for Apple's AI training.&lt;/p&gt;

&lt;h2&gt;
  
  
  How robots.txt rules work
&lt;/h2&gt;

&lt;p&gt;robots.txt is a plain text file at the root of your site, for example &lt;code&gt;https://example.com/robots.txt&lt;/code&gt;. It is made of groups. Each group names one or more crawlers with &lt;code&gt;User-agent&lt;/code&gt; lines, followed by &lt;code&gt;Allow&lt;/code&gt; and &lt;code&gt;Disallow&lt;/code&gt; rules:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User-agent: GPTBot
Disallow: /

User-agent: *
Disallow: /admin/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two details trip people up:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A crawler follows only the most specific group that names it.&lt;/strong&gt; In the example above, GPTBot obeys its own group and ignores the &lt;code&gt;*&lt;/code&gt; group completely. A crawler that is not named anywhere falls back to &lt;code&gt;*&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Within a group, the longest matching rule wins.&lt;/strong&gt; &lt;code&gt;Disallow: /&lt;/code&gt; with &lt;code&gt;Allow: /blog/&lt;/code&gt; means the blog is allowed and everything else is blocked.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Common setups
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Block AI training, allow AI search.&lt;/strong&gt; This is a popular middle ground for publishers who want citations and visits but do not want their work used to train models:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: Meta-ExternalAgent
Disallow: /
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Allow everything.&lt;/strong&gt; If you want the widest possible reach, including being part of the data AI tools learn from, you do not need any AI-specific rules at all.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Block everything AI.&lt;/strong&gt; Add the search and user crawlers to the list above as well. Be aware that you will no longer appear as a source in those assistants' answers.&lt;/p&gt;

&lt;h2&gt;
  
  
  What robots.txt cannot do
&lt;/h2&gt;

&lt;p&gt;robots.txt is a request, not a wall. The large AI companies say their crawlers respect it, but a badly behaved bot can ignore it. If you need to stop a crawler for certain, block it at the server or firewall level as well. Also remember that robots.txt is public: anyone can read it, so never use it to hide private areas. Protect those with a login.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check what your site allows today
&lt;/h2&gt;

&lt;p&gt;It is easy to get a rule slightly wrong, for example by adding a crawler to a group you did not intend, or leaving a typo in its name. A few free resources:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/ai-crawler-checker" rel="noopener noreferrer"&gt;AI Crawler Checker&lt;/a&gt;: reads your live robots.txt and shows, crawler by crawler, whether it is allowed, blocked or partly blocked.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/Nadeem035/ai-crawlers-robots-txt" rel="noopener noreferrer"&gt;ai-crawlers-robots-txt on GitHub&lt;/a&gt;: an open list of 14 AI crawlers with their operator, purpose and official docs link, plus copy-paste templates (CC0).&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://seopeck.com/tool/robotstxt-generator" rel="noopener noreferrer"&gt;Robots.txt Generator&lt;/a&gt;: builds a full robots.txt with per-crawler rules.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Disclosure: I build SEOpeck and maintain the GitHub list. Corrections and new crawlers are welcome as PRs.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>seo</category>
      <category>webdev</category>
      <category>ai</category>
      <category>tutorial</category>
    </item>
  </channel>
</rss>
