<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Yap Pat Yih</title>
    <description>The latest articles on DEV Community by Yap Pat Yih (@yappatyih).</description>
    <link>https://dev.to/yappatyih</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4156054%2F5c2c40fe-ee04-4de0-b91e-14f222fad980.jpg</url>
      <title>DEV Community: Yap Pat Yih</title>
      <link>https://dev.to/yappatyih</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/yappatyih"/>
    <language>en</language>
    <item>
      <title>Which AI bots visit my website? I counted for 30 days</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Thu, 22 Oct 2026 00:56:17 +0000</pubDate>
      <link>https://dev.to/yappatyih/which-ai-bots-visit-my-website-i-counted-for-30-days-2cnj</link>
      <guid>https://dev.to/yappatyih/which-ai-bots-visit-my-website-i-counted-for-30-days-2cnj</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/which-ai-bots-visit-my-website/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;AI crawler detection means finding out which bots that collect pages for AI companies visit your website, how often, and what they read, usually from the name each bot gives in its requests. In the 30 days to 20 October 2026, Cloudflare counted about 4,040 requests to this site from the AI crawlers it recognises. Google sent the most, then Anthropic, OpenAI, Apple and Microsoft. About half of the successful requests were for training AI models. Only 26 were an AI assistant fetching a page because a person asked it something. The crawlers read my sitemaps more than any post, some "AI crawlers" were scanners hunting for password files, and all that reading did not get me cited: ChatGPT named this site in 0 of 12 answers on 17 October. Below are the numbers, and how to see the same on your own site.&lt;/p&gt;

&lt;h2&gt;
  
  
  Key takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;In the 30 days to 20 October 2026, the AI crawlers Cloudflare recognises made about 4,040 requests to this site, a personal blog with eight posts.&lt;/li&gt;
&lt;li&gt;About 1,720 of the roughly 3,540 successful requests were for training. Only 26 were fetches for a person asking an assistant.&lt;/li&gt;
&lt;li&gt;The sitemaps were read 687 times. No single post was read more than 66 times.&lt;/li&gt;
&lt;li&gt;Being crawled is not being cited. OpenAI's crawlers made 508 requests in the month; ChatGPT cited this site in none of 12 answers.&lt;/li&gt;
&lt;li&gt;If your site is on Cloudflare, even on the free plan, you can see all of this under AI Crawl Control in a few minutes.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Who visited: ten companies
&lt;/h2&gt;

&lt;p&gt;In the 30 days to 20 October 2026, ten companies sent most of the AI crawler traffic to this site. Cloudflare groups each company's crawlers together, so OpenAI's row covers GPTBot and two more of its bots.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Company&lt;/th&gt;
&lt;th&gt;Main crawler&lt;/th&gt;
&lt;th&gt;Requests allowed, 30 days&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Google&lt;/td&gt;
&lt;td&gt;Googlebot, and one more&lt;/td&gt;
&lt;td&gt;961&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;ClaudeBot, and two more&lt;/td&gt;
&lt;td&gt;680&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;GPTBot, and two more&lt;/td&gt;
&lt;td&gt;508&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Apple&lt;/td&gt;
&lt;td&gt;Applebot&lt;/td&gt;
&lt;td&gt;377&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Microsoft&lt;/td&gt;
&lt;td&gt;BingBot&lt;/td&gt;
&lt;td&gt;309&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Huawei&lt;/td&gt;
&lt;td&gt;PetalBot&lt;/td&gt;
&lt;td&gt;238&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Meta&lt;/td&gt;
&lt;td&gt;Meta-ExternalAgent, and two more&lt;/td&gt;
&lt;td&gt;212&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Common Crawl&lt;/td&gt;
&lt;td&gt;CCBot&lt;/td&gt;
&lt;td&gt;138&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Amazon&lt;/td&gt;
&lt;td&gt;Amazonbot&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Internet Archive&lt;/td&gt;
&lt;td&gt;archive.org_bot&lt;/td&gt;
&lt;td&gt;111&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Googlebot is the crawler behind Google Search, and Google's AI answers draw on the same index, which is why Cloudflare lists it here. Anthropic came second, ahead of OpenAI. Perplexity, whose answers did cite this site on 17 October, is not in the top ten at all.&lt;/p&gt;

&lt;p&gt;A week earlier, on 13 October, the same 30-day view showed about 3,000 requests: Googlebot 602, ClaudeBot 464, GPTBot 356, Applebot 221. On 20 October it showed Googlebot 889, ClaudeBot 642, GPTBot 373, Applebot 377. The two windows share 23 days, so this is not a clean month on month figure. The direction is still plain: more crawling, most of the growth from Google, Anthropic and Apple.&lt;/p&gt;

&lt;h2&gt;
  
  
  What they came for
&lt;/h2&gt;

&lt;p&gt;Cloudflare sorts every AI crawler by purpose, and on this site training came first by a wide margin. Of about 3,540 successful requests in the 30 days to 20 October 2026:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Purpose, as Cloudflare names it&lt;/th&gt;
&lt;th&gt;Requests&lt;/th&gt;
&lt;th&gt;Share&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;AI Crawler: collecting pages to train models&lt;/td&gt;
&lt;td&gt;about 1,720&lt;/td&gt;
&lt;td&gt;about half&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Search Engine Crawler: building a search index&lt;/td&gt;
&lt;td&gt;about 1,190&lt;/td&gt;
&lt;td&gt;about a third&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AI Search: building an AI search index&lt;/td&gt;
&lt;td&gt;482&lt;/td&gt;
&lt;td&gt;about one in seven&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Archiver&lt;/td&gt;
&lt;td&gt;110&lt;/td&gt;
&lt;td&gt;3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AI Assistant: fetching a page because a person asked&lt;/td&gt;
&lt;td&gt;26&lt;/td&gt;
&lt;td&gt;under 1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That last row is the one I care about most. Twenty six times in a month, someone asked an AI something and it came to read a page here. Everything else was machines filling their stores in case they need it later.&lt;/p&gt;

&lt;h2&gt;
  
  
  What they read
&lt;/h2&gt;

&lt;p&gt;The most read file on this site is not a post. It is the list of posts. In the 30 days to 20 October 2026:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Path&lt;/th&gt;
&lt;th&gt;Requests&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/sitemap-posts.xml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;280&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/sitemap-index.xml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;247&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;/&lt;/code&gt; (home page)&lt;/td&gt;
&lt;td&gt;204&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/sitemap.xml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;94&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;/_image/&lt;/code&gt; (images)&lt;/td&gt;
&lt;td&gt;72&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;a href="https://yappatyih.com/writing/can-ai-crawlers-read-your-site/" rel="noopener noreferrer"&gt;my post on whether AI crawlers can read your site&lt;/a&gt;&lt;/td&gt;
&lt;td&gt;66&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/sitemap-0.xml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;66&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;my post on the Brain and Hand workflow&lt;/td&gt;
&lt;td&gt;60&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;my post on the Chinese web font&lt;/td&gt;
&lt;td&gt;51&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The four sitemap files took 687 requests between them. The busiest post took 66. Crawlers spend most of their visits finding out what exists, and much less reading it. 271 different paths were requested in all. In bytes, the whole month came to about 74 MB, Googlebot 18.63 MB and GPTBot 11.24 MB of it. On Cloudflare's free plan that cost me nothing.&lt;/p&gt;

&lt;p&gt;So a correct, current sitemap matters more than any special file made for AI. In the 30 days to 13 October, the same crawlers fetched &lt;a href="https://yappatyih.com/writing/does-llms-txt-work/" rel="noopener noreferrer"&gt;my llms.txt zero times&lt;/a&gt; and my robots.txt 605 times.&lt;/p&gt;

&lt;h2&gt;
  
  
  The scanners that borrow a crawler's name
&lt;/h2&gt;

&lt;p&gt;Not every request under an AI crawler's name came from an AI company. Among the ten missing paths requested most often in the 30 days to 20 October 2026, eight were files that hold passwords and keys:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;/web.config&lt;/code&gt; and &lt;code&gt;/credentials.json&lt;/code&gt;, 4 requests each&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;/.env.local&lt;/code&gt;, &lt;code&gt;/.env.bak&lt;/code&gt;, &lt;code&gt;/terraform.tfvars&lt;/code&gt;, &lt;code&gt;/.git/config&lt;/code&gt;, &lt;code&gt;/application.yml&lt;/code&gt; and &lt;code&gt;/.git/HEAD&lt;/code&gt;, 3 each&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A real training crawler follows links and sitemaps. It has no reason to ask for a &lt;code&gt;.env&lt;/code&gt; file that nothing links to. These were scanners using a crawler's name in their user agent, and on the free plan Cloudflare recognises crawlers by that name, so they were counted as AI crawlers. Every one got a 404, because none of those files exist here. Borrowing a name is that easy: in &lt;a href="https://yappatyih.com/writing/can-ai-crawlers-read-your-site/" rel="noopener noreferrer"&gt;my first check of this site&lt;/a&gt; I did it myself, from my own computer, to see what each crawler gets back.&lt;/p&gt;

&lt;p&gt;If someone else built your website, ask them to check this today. Open &lt;code&gt;yoursite.com/.env&lt;/code&gt; and &lt;code&gt;yoursite.com/.git/config&lt;/code&gt; in a browser. Both should give an error page, never a file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Being crawled is not being cited
&lt;/h2&gt;

&lt;p&gt;All that reading has not yet turned into AI answers that name this site. The same month, side by side:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;OpenAI's crawlers made 508 allowed requests in the 30 days to 20 October. On 17 October I asked ChatGPT twelve questions this site answers, from my name to the topic of each post. It cited this site in 0 of 12.&lt;/li&gt;
&lt;li&gt;Microsoft's BingBot made 309. Bing Webmaster Tools' AI Performance report showed 0 citations in three months.&lt;/li&gt;
&lt;li&gt;Google sent 961. In Search Console's 28 days to 17 October, this site had 14 impressions in Google's AI features.&lt;/li&gt;
&lt;li&gt;Perplexity is not in the top ten senders, yet it cited this site in 4 of 8 answers on 17 October.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A crawler visit means my page is in the pile. Whether an answer quotes it depends on what the page says, who else says it, and how far the engine trusts the site. The number of visits does not predict the number of citations.&lt;/p&gt;

&lt;p&gt;I also cannot tell you whether any of those answers sent me a visitor. Cloudflare's Referral Traffic column showed a dash on every path. Its documentation says referral counts are for paid plans, so on the free plan that dash means "not counted", not zero.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to see which AI bots visit your own site
&lt;/h2&gt;

&lt;p&gt;There are three ways to check, and the first takes a few minutes if your site already runs on Cloudflare, free plan included.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Cloudflare AI Crawl Control.&lt;/strong&gt; Log in to the Cloudflare dashboard, pick your account and your domain, and open AI Crawl Control.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Overview&lt;/strong&gt; shows the total and the companies.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Crawlers&lt;/strong&gt; lists each crawler, its company, its purpose, how many requests were allowed or failed, and whether it is allowed or blocked.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Metrics&lt;/strong&gt; has the charts. Set the range to "Last 30 days", the longest it offers. Group by Category to see the purposes, read "Most crawled paths", or filter by a path to see who read one page.&lt;/li&gt;
&lt;li&gt;robots.txt is not in Metrics. Cloudflare counts it on the &lt;strong&gt;Signals&lt;/strong&gt; page.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Two limits: on the free plan a crawler is recognised by the name it gives, so a scanner wearing GPTBot's name counts as GPTBot; and there is nothing older than 30 days.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Your server log.&lt;/strong&gt; If your site is not on Cloudflare, ask your host for the raw access log; on cPanel hosting it is under "Raw Access". Then count the crawler names:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-oiE&lt;/span&gt; &lt;span class="s2"&gt;"GPTBot|OAI-SearchBot|ChatGPT-User|ClaudeBot|Claude-User|PerplexityBot|Googlebot|Bingbot|Applebot|PetalBot|Meta-ExternalAgent|CCBot|Amazonbot|Bytespider"&lt;/span&gt; access.log | &lt;span class="nb"&gt;sort&lt;/span&gt; | &lt;span class="nb"&gt;uniq&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; | &lt;span class="nb"&gt;sort&lt;/span&gt; &lt;span class="nt"&gt;-rn&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A name is only a name, so check the address of any request that matters: Google and OpenAI publish the address ranges their crawlers use. Google Analytics will not show any of this. It leaves known bots out, and most of these crawlers never run its script.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Your robots.txt.&lt;/strong&gt; Logs tell you who came. robots.txt tells you who is allowed. My &lt;a href="https://yappatyih.com/tools/ai-crawler-checker/" rel="noopener noreferrer"&gt;free AI crawler checker&lt;/a&gt; reads your robots.txt for every major AI crawler at once and shows which ones it lets in. To see what a crawler really gets back, request a page with its name, as I did in my first check.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I let them all in
&lt;/h2&gt;

&lt;p&gt;My robots.txt lets every one of these crawlers in, training crawlers included. I write here so that people, search engines and AI answers can find what I think, and an answer cannot quote a page its crawler never read. A month of data has not changed that. It has only shown me that letting them in is the start, not the result. Most big Malaysian websites leave the door open too: when I read the robots.txt of 100 of them, &lt;a href="https://yappatyih.com/writing/malaysian-websites-block-ai-crawlers/" rel="noopener noreferrer"&gt;only 13 blocked any AI crawler&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  How do I check which AI bots visit my website?
&lt;/h3&gt;

&lt;p&gt;If your site is on Cloudflare, open AI Crawl Control for your domain: it lists every AI crawler it recognises, its company, its purpose and the pages it read, for up to 30 days, on the free plan too. Otherwise, search your server's access log for crawler names such as GPTBot, ClaudeBot and PerplexityBot. To see which bots your robots.txt allows, use an AI crawler checker.&lt;/p&gt;

&lt;h3&gt;
  
  
  Which AI crawlers visit websites the most?
&lt;/h3&gt;

&lt;p&gt;On this site, in the 30 days to 20 October 2026, Google sent the most allowed requests (961), then Anthropic (680), OpenAI (508), Apple (377) and Microsoft (309). By single crawler, Googlebot led with 889 requests, then ClaudeBot 642, Applebot 377 and GPTBot 373. Other sites will differ, but these names are the ones to look for first.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is the difference between an AI training crawler and an AI search crawler?
&lt;/h3&gt;

&lt;p&gt;A training crawler, such as GPTBot or ClaudeBot, collects pages to train AI models. An AI search crawler, such as OAI-SearchBot, builds the index an AI searches when it answers. An AI assistant fetch, such as ChatGPT-User or Claude-User, happens when a person asks a question and the assistant reads a page for them. On this site the third kind was 26 requests in 30 days, out of about 3,540.&lt;/p&gt;

&lt;h3&gt;
  
  
  Can a bot pretend to be an AI crawler?
&lt;/h3&gt;

&lt;p&gt;Yes. Any program can put GPTBot or ClaudeBot in its user agent. On this site, in the 30 days to 20 October 2026, requests under AI crawler names asked for &lt;code&gt;.env&lt;/code&gt;, &lt;code&gt;.git&lt;/code&gt; and &lt;code&gt;credentials.json&lt;/code&gt; files, which no real crawler needs. Check the address against the ranges the company publishes, and keep no secret files on your web server.&lt;/p&gt;

&lt;h3&gt;
  
  
  If AI crawlers visit my site, will ChatGPT cite it?
&lt;/h3&gt;

&lt;p&gt;Not necessarily. OpenAI's crawlers made 508 requests to this site in the 30 days to 20 October 2026, and ChatGPT cited it in 0 of 12 answers on 17 October. Being crawled puts your page in the pile; being cited depends on what the page says and how far the engine trusts your site.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>seo</category>
      <category>ai</category>
      <category>cloudflare</category>
    </item>
    <item>
      <title>Does llms.txt work? In 30 days, no AI crawler read mine</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Sat, 17 Oct 2026 16:18:43 +0000</pubDate>
      <link>https://dev.to/yappatyih/does-llmstxt-work-in-30-days-no-ai-crawler-read-mine-1ajn</link>
      <guid>https://dev.to/yappatyih/does-llmstxt-work-in-30-days-no-ai-crawler-read-mine-1ajn</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/does-llms-txt-work/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;llms.txt is a plain text file at the root of a website that names the site, sums it up in a line and lists its main pages in Markdown, proposed in September 2024 so that AI tools can understand a site quickly. This site has served one since August 2026. In the 30 days from 14 September to 13 October 2026, the AI crawlers Cloudflare recognises fetched my robots.txt 605 times, my post sitemap 192 times, and my llms.txt and llms-full.txt zero times. I checked three ways that the zero is real. The file stays anyway, because it costs nothing to serve and gives a person or an AI tool one clean page to read when they ask for it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Key takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;In 30 days, no AI crawler that Cloudflare recognises fetched &lt;code&gt;/llms.txt&lt;/code&gt; or &lt;code&gt;/llms-full.txt&lt;/code&gt; on this site, not once.&lt;/li&gt;
&lt;li&gt;In the same 30 days those crawlers fetched robots.txt 605 times, the post sitemap 192 times and the home page 207 times.&lt;/li&gt;
&lt;li&gt;If you want AI crawlers to find your pages, robots.txt, a sitemap and plain HTML pages are what they read. An llms.txt does not replace any of them.&lt;/li&gt;
&lt;li&gt;The count covers only crawlers Cloudflare recognises, over 30 days. It cannot see a person, or an AI tool that fetches the file as an ordinary browser.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What my llms.txt says
&lt;/h2&gt;

&lt;p&gt;My llms.txt, 6,406 bytes on 13 October 2026, is Markdown that the Cloudflare Worker running this site writes fresh on every request, from the same pages and posts the site shows. A first version went up on 1 August 2026; since 28 September it has been generated on each request. It opens like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# Yap Pat Yih&lt;/span&gt;
&lt;span class="gt"&gt;
&amp;gt; General Manager and co-founder of BlackRevo, founder of Habitus Mind, builder of Lumen. Based in Kuala Lumpur, Malaysia. Works on systems where people and AI work together.&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After that come a short paragraph about me and two lists, "Pages (English)" and "Writing", with one line per page: its title, its address and what it is about.&lt;/p&gt;

&lt;p&gt;Its companion, llms-full.txt, has been there since 29 September 2026. It is the full text of every post in one file, 122,659 bytes on 13 October 2026. Both are served as plain text.&lt;/p&gt;

&lt;h2&gt;
  
  
  How I counted
&lt;/h2&gt;

&lt;p&gt;Cloudflare's AI Crawl Control counts the requests that AI crawlers it recognises make to a site, path by path, and it works on this site's free plan. On 13 October 2026 I filtered it for every path containing &lt;code&gt;llms&lt;/code&gt; over its longest range, the last 30 days, which began on 14 September. The answer was "No requests data found". I read it again the next morning, for 7 days and for 30: still zero.&lt;/p&gt;

&lt;p&gt;A zero only means something if the filter could have seen the file. So I checked three things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The same path filter found 97 requests for &lt;code&gt;/sitemap-posts.xml&lt;/code&gt; in 7 days. The filter works.&lt;/li&gt;
&lt;li&gt;"Path contains &lt;code&gt;.txt&lt;/code&gt;" found one request in 30 days: BingBot fetching the small key file this site uses for IndexNow. Text files are counted.&lt;/li&gt;
&lt;li&gt;The llms files are built on each request by the same Worker that builds the post pages, and the post pages are counted: 57 requests for one post in the same 30 days.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The reading was done by the Hand, one of the &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;two AI sessions I run this site with&lt;/a&gt;, in my Cloudflare account, read only.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the AI crawlers read instead
&lt;/h2&gt;

&lt;p&gt;In the same 30 days the AI crawlers Cloudflare recognises made about 3,000 requests to this site. These are the ones that matter here:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Path&lt;/th&gt;
&lt;th&gt;Requests, 30 days&lt;/th&gt;
&lt;th&gt;Top crawlers&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/robots.txt&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;605&lt;/td&gt;
&lt;td&gt;not split by crawler&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;207&lt;/td&gt;
&lt;td&gt;Googlebot 39, GPTBot 29, OAI-SearchBot 25, BingBot 20, ClaudeBot 11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/sitemap-posts.xml&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;192&lt;/td&gt;
&lt;td&gt;ClaudeBot 128, Googlebot 16, GPTBot 14, BingBot 13, PetalBot 8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;one post&lt;/td&gt;
&lt;td&gt;57&lt;/td&gt;
&lt;td&gt;PerplexityBot 6, GPTBot 5, CCBot 5, Claude-User 4, ClaudeBot 4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/llms.txt&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/llms-full.txt&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;all paths&lt;/td&gt;
&lt;td&gt;about 3,000&lt;/td&gt;
&lt;td&gt;Googlebot 602, ClaudeBot 464, GPTBot 356, Applebot 221, Meta-ExternalAgent 185&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;ClaudeBot made 128 of the 192 sitemap fetches, two thirds of them. Cloudflare counts robots.txt on a separate page, Signals, which does not split it by crawler; all 605 fetches succeeded.&lt;/p&gt;

&lt;p&gt;The crawlers came, and often. They read what crawlers have read for years: robots.txt to learn the rules, the sitemap to find the pages, and the pages themselves.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the people behind llms.txt and the AI companies say
&lt;/h2&gt;

&lt;p&gt;Jeremy Howard proposed llms.txt on 3 September 2024 at llmstxt.org, and the proposal itself does not put crawlers first. It describes a file read on demand, when an AI tool needs to know about a topic while it helps someone. In its own words: "Our expectation was that llms.txt would mainly be useful for inference rather than training". My measurement sees crawlers only, so it tests a use the proposal does not lead with.&lt;/p&gt;

&lt;p&gt;Google's Search Central page on AI features and your website says you do not need new machine readable files or AI text files to appear in Google's AI features. It does not name llms.txt.&lt;/p&gt;

&lt;p&gt;OpenAI, Anthropic and Perplexity each document their crawlers, and each points site owners to robots.txt to control them. None of those crawler pages says the crawler reads a site's llms.txt. All three do publish an llms.txt for their own developer documentation, which fits the proposal's aim: a tool reading docs when asked, not a crawler sweeping the web.&lt;/p&gt;

&lt;h2&gt;
  
  
  What this measurement cannot see
&lt;/h2&gt;

&lt;p&gt;Cloudflare's AI crawler view counts only the crawlers it recognises, so three kinds of visit are invisible here: a person who opens &lt;code&gt;/llms.txt&lt;/code&gt; in a browser, a bot Cloudflare does not recognise, and an AI tool that fetches the file while helping someone but presents itself as an ordinary browser. The last one is exactly the use the proposal describes, and I cannot rule it out.&lt;/p&gt;

&lt;p&gt;I also cannot look back past 14 September. Thirty days is the longest range the dashboard offers, the free plan's other analytics cannot filter by path, and the Worker keeps no logs.&lt;/p&gt;

&lt;p&gt;So the honest claim is narrow. In these 30 days, no AI crawler Cloudflare recognises asked for the file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I keep it anyway
&lt;/h2&gt;

&lt;p&gt;I keep both files, for two reasons. They cost nothing: the Worker builds them from the same posts the site already shows, so there is nothing to maintain by hand. And they give a person, or an AI tool someone points at this site, one clean text file to read instead of a page full of layout.&lt;/p&gt;

&lt;p&gt;If nobody ever asks for them, I have lost nothing. What I will not do is count them as a way to be found.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you want AI to read your site
&lt;/h2&gt;

&lt;p&gt;The numbers above show what AI crawlers actually read on this site: robots.txt, the sitemap and the pages. That is where the work belongs.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Let the crawlers in. Check that your robots.txt does not block them and that your host or firewall does not either. &lt;a href="https://yappatyih.com/writing/can-ai-crawlers-read-your-site/" rel="noopener noreferrer"&gt;This is how I checked mine&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Keep a sitemap that lists every page, and name it in robots.txt.&lt;/li&gt;
&lt;li&gt;Make each page plain HTML that carries its words without waiting for a script.&lt;/li&gt;
&lt;li&gt;If you add an llms.txt, add it as a courtesy to readers and tools, not as a step to be found, and measure whether anything asks for it.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  What is llms.txt?
&lt;/h3&gt;

&lt;p&gt;llms.txt is a Markdown file at the root of a website, at &lt;code&gt;/llms.txt&lt;/code&gt;, that names the site, sums it up in a quoted line and lists its main pages with a short note on each. Jeremy Howard proposed it on 3 September 2024 at llmstxt.org. It is meant to help an AI tool understand a site quickly when someone asks it to.&lt;/p&gt;

&lt;h3&gt;
  
  
  Does Google use llms.txt?
&lt;/h3&gt;

&lt;p&gt;Google's Search Central page on AI features says you do not need to create new machine readable files or AI text files to appear in Google's AI features. The page does not mention llms.txt by name. So adding one is not a step toward Google's AI answers.&lt;/p&gt;

&lt;h3&gt;
  
  
  Do AI crawlers read llms.txt?
&lt;/h3&gt;

&lt;p&gt;Not on this site. From 14 September to 13 October 2026, the AI crawlers Cloudflare recognises fetched &lt;code&gt;/llms.txt&lt;/code&gt; and &lt;code&gt;/llms-full.txt&lt;/code&gt; zero times, while they fetched robots.txt 605 times. OpenAI, Anthropic and Perplexity document robots.txt for their crawlers and do not say those crawlers read llms.txt.&lt;/p&gt;

&lt;h3&gt;
  
  
  Should I add llms.txt to my website?
&lt;/h3&gt;

&lt;p&gt;Add it if it is cheap for you to keep accurate, as a courtesy to people and AI tools that look for it. Do not expect crawler visits or search traffic from it: on this site it had none in 30 days. Spend your effort first on robots.txt, a sitemap and plain HTML pages.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is the difference between llms.txt and robots.txt?
&lt;/h3&gt;

&lt;p&gt;robots.txt tells crawlers which paths they may fetch, and OpenAI, Anthropic and Perplexity all document it as the way to control their crawlers. llms.txt is a summary of the site and a list of its pages, meant for an AI tool to read when someone asks about the site. On this site, AI crawlers fetched robots.txt 605 times in 30 days and llms.txt zero times.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>seo</category>
      <category>ai</category>
      <category>llm</category>
    </item>
    <item>
      <title>I made my analytics wait for the page. Here is what it cost</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Wed, 14 Oct 2026 01:09:23 +0000</pubDate>
      <link>https://dev.to/yappatyih/i-made-my-analytics-wait-for-the-page-here-is-what-it-cost-f8o</link>
      <guid>https://dev.to/yappatyih/i-made-my-analytics-wait-for-the-page-here-is-what-it-cost-f8o</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/delay-google-tag-manager-page-speed/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;Deferring Google Tag Manager means the page asks for Google's analytics scripts only after its own content has loaded, when the browser is idle or when the reader first scrolls, taps or presses a key. On 2 October 2026 I did this on this site. Google's two scripts were 297 KB together, 60% of what a reader downloaded to open one post. They still download, but after the page instead of with it. On the English version of one post, the median Lighthouse mobile score went from 84.5 to 98 and the Largest Contentful Paint from 3.95 s to 1.6 s. Every page view is still counted when the reader stays. The cost is that a visit that leaves within a second or two without touching the page may not be. And part of the better score is the speed test not waiting long enough to see Google at all, which I explain below.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Google was costing a post page
&lt;/h2&gt;

&lt;p&gt;On the morning of 2 October 2026, Google was 297 KB of every post page on this site, out of 492 to 494 KB in total.&lt;/p&gt;

&lt;p&gt;Earlier that day I had removed the cookie banner. Analytics now loaded for every reader, with an off switch behind the Cookies link in the footer, and with analytics off by default for readers in the European Economic Area, the UK and Switzerland. The banner had been holding Google back until a reader clicked Allow. Without it, two scripts started on every page as soon as it began to load: gtm.js, the Tag Manager container, about 122 KB, and gtag/js, the Google Analytics tag, about 174 KB.&lt;/p&gt;

&lt;p&gt;I measured &lt;a href="https://yappatyih.com/writing/the-mamak-table-question/" rel="noopener noreferrer"&gt;the mamak table post&lt;/a&gt; with Lighthouse in its mobile setting, two runs per page. The English page scored 94 and 75. The Chinese page scored 95 and 92. The home page scored 68 and 56.&lt;/p&gt;

&lt;p&gt;The words a reader came for were a small part of what the phone was downloading.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the order matters, not only the size
&lt;/h2&gt;

&lt;p&gt;A script marked &lt;code&gt;async&lt;/code&gt; does not block the page from being drawn, and it still competes with it. On a phone the Google files share one connection with the post's own images and fonts, and when they arrive they run on the same main thread that is trying to lay out the text.&lt;/p&gt;

&lt;p&gt;Analytics does not need to be first. It needs to arrive once, and to arrive in time to record the visit.&lt;/p&gt;

&lt;p&gt;So the question was not whether to keep Google. It was when to ask for it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The change: wait for idle or the first touch
&lt;/h2&gt;

&lt;p&gt;The rule since 2 October is simple. The page loads. Then the first of two things requests gtm.js: the browser saying it is idle, or the reader doing something.&lt;/p&gt;

&lt;p&gt;In detail:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The consent defaults, and a reader's stored choice, are pushed straight away, as before. Nothing is ever sent under the wrong consent.&lt;/li&gt;
&lt;li&gt;After the page's &lt;code&gt;load&lt;/code&gt; event, the script asks the browser to call it when it is idle, with a 2,000 ms limit. Safari has no idle callback, so there it waits 1,500 ms after &lt;code&gt;load&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;If the reader scrolls, taps, clicks, uses the wheel or presses a key first, gtm.js is requested at that moment.&lt;/li&gt;
&lt;li&gt;Whichever comes first wins and cancels the others. It happens once per visit, also when the reader moves to another page on the site.&lt;/li&gt;
&lt;li&gt;A reader who has turned analytics off requests nothing from Google at all.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This is the core of it, shortened from the script that runs on this site:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;INTERACTIONS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;pointerdown&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;keydown&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;touchstart&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;scroll&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;wheel&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;LISTEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;capture&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;passive&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="nx"&gt;loaded&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;loadGtm&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;loaded&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;return&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;loaded&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;INTERACTIONS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;type&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;removeEventListener&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;type&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;loadGtm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;LISTEN&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;dataLayer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;push&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gtm.start&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;Date&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="na"&gt;event&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;gtm.js&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createElement&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;script&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;src&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;https://www.googletagmanager.com/gtm.js?id=GTM-XXXXXXX&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;head&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;appendChild&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;s&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;waitForIdle&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;requestIdleCallback&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nf"&gt;requestIdleCallback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;loadGtm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;2000&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
  &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="nf"&gt;setTimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;loadGtm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1500&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;INTERACTIONS&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;forEach&lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="nx"&gt;type&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;addEventListener&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;type&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;loadGtm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;LISTEN&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
&lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;readyState&lt;/span&gt; &lt;span class="o"&gt;===&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;complete&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="nf"&gt;waitForIdle&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="nb"&gt;window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;addEventListener&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;load&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;waitForIdle&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;once&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt; &lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two details matter. The &lt;code&gt;gtm.start&lt;/code&gt; line is pushed in the same function that adds the script, so Tag Manager's own clock starts when the file is actually requested. And the listeners are passive, so they never delay a scroll.&lt;/p&gt;

&lt;p&gt;I work with two AI sessions, &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;a Brain that plans and a Hand that builds&lt;/a&gt;. The Brain proposed this after &lt;a href="https://yappatyih.com/writing/chinese-web-font-cost-1mb/" rel="noopener noreferrer"&gt;removing the Chinese web font&lt;/a&gt; left Google as the heaviest thing on a post page. The Hand built it with tests for each case, including turning analytics off before Google has loaded and on again on the same page. I agreed to it only after I had read what it would cost me, which is further down.&lt;/p&gt;

&lt;h2&gt;
  
  
  Before and after
&lt;/h2&gt;

&lt;p&gt;These are the medians from Lighthouse mobile on the live site, two runs before the change and three after, on the same day. With two runs, the "median" before is the average of the two.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;English post, score&lt;/td&gt;
&lt;td&gt;84.5&lt;/td&gt;
&lt;td&gt;98&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;English post, Largest Contentful Paint&lt;/td&gt;
&lt;td&gt;3.95 s&lt;/td&gt;
&lt;td&gt;1.6 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;English post, Total Blocking Time&lt;/td&gt;
&lt;td&gt;125 ms&lt;/td&gt;
&lt;td&gt;50 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chinese post, score&lt;/td&gt;
&lt;td&gt;93.5&lt;/td&gt;
&lt;td&gt;97&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chinese post, Largest Contentful Paint&lt;/td&gt;
&lt;td&gt;2.45 s&lt;/td&gt;
&lt;td&gt;2.1 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Chinese post, Total Blocking Time&lt;/td&gt;
&lt;td&gt;165 ms&lt;/td&gt;
&lt;td&gt;60 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Home page, score&lt;/td&gt;
&lt;td&gt;62&lt;/td&gt;
&lt;td&gt;68&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Home page, Largest Contentful Paint&lt;/td&gt;
&lt;td&gt;5.65 s&lt;/td&gt;
&lt;td&gt;5.7 s&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Read the English row with care. Its "before" is the average of 94 and 75, and the 75 was one slow run. The honest summary is this: before, the post pages scored in the 90s with bad runs mixed in; after, they score 97 to 99 on most runs, and the time the page spends blocked has roughly halved or better.&lt;/p&gt;

&lt;p&gt;The home page barely moved, and I know why. It runs a 3D scene, which costs more than analytics does, and &lt;a href="https://yappatyih.com/writing/the-night-my-post-pages-scored-39/" rel="noopener noreferrer"&gt;I wrote about that cost separately&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;In a real browser on a fresh visit, the page finished loading at 692 ms, gtm.js was requested at 718 ms, and the page view reached Google at 1,222 ms. With the old code the same morning, gtm.js was requested at 659 ms, before the page had loaded, and the page view went out at 1,212 ms. On a fast connection the page view leaves at about the same moment. What moved is that Google no longer starts while the page is still arriving.&lt;/p&gt;

&lt;h2&gt;
  
  
  The part a speed test does not see
&lt;/h2&gt;

&lt;p&gt;Lighthouse ends its measurement when the network goes quiet. If that happens before the idle moment, Google never appears in the test.&lt;/p&gt;

&lt;p&gt;That happened in two of my nine runs after the change. One English run weighed 195 KB with nothing from Google in it. Another had only gtm.js, 122 KB. A real reader still downloads all 297 KB. The files just arrive after the page.&lt;/p&gt;

&lt;p&gt;So part of the better lab score is the lab not staying long enough. That is not a reason to undo the change: the reader's page really does finish first now. It is a reason not to quote the best run, and to trust the medians and the browser timings above more than any single score.&lt;/p&gt;

&lt;p&gt;One more run shows why one run means little. The first home page run after the change scored 45, with the first text appearing after 22.9 seconds. Nothing on the page had changed. The network had a bad minute.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it costs
&lt;/h2&gt;

&lt;p&gt;The cost is a short visit. A reader who opens a page and leaves within a second or two, without scrolling, tapping or pressing a key, may leave before Google is asked for, and that visit is not counted.&lt;/p&gt;

&lt;p&gt;On a fast connection the window is small: the idle moment came about 30 ms after the page loaded. On a slow phone it is longer. I do not know how many visits I lose this way, and I cannot find out, because analytics cannot count a visit it never saw. I accepted that. This site is written to be read, and someone who leaves in one second without moving has not read anything.&lt;/p&gt;

&lt;p&gt;What it does not cost:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Events pushed before Google arrives wait in the queue and are read when it does.&lt;/li&gt;
&lt;li&gt;Moving to another page on the site loads nothing twice. The second page's view is still recorded.&lt;/li&gt;
&lt;li&gt;A reader who turned analytics off still sends nothing. I checked that after the change with a reload, a scroll, a click and a key: zero requests to Google.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you run ads and count conversions, your answer may be different from mine. A visit you paid for that leaves in a second is still a visit you paid for.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to do it on your own site
&lt;/h2&gt;

&lt;p&gt;You can try this in an afternoon, and measure it in the same afternoon.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Measure first. Run Lighthouse in its mobile setting three times on a post page and note how many KB come from googletagmanager.com and google-analytics.com.&lt;/li&gt;
&lt;li&gt;Remove the Tag Manager snippet from the page head, but keep the &lt;code&gt;dataLayer&lt;/code&gt; and your consent defaults early in the page.&lt;/li&gt;
&lt;li&gt;Add a trigger like the one above in your own script file, not an inline snippet, if your security policy forbids inline scripts, as this site's does.&lt;/li&gt;
&lt;li&gt;Check in a real browser that the page view still arrives: open DevTools, Network, filter for &lt;code&gt;collect&lt;/code&gt;, and try a fresh visit without touching the page, then one with a scroll.&lt;/li&gt;
&lt;li&gt;Check that turning analytics off still sends nothing.&lt;/li&gt;
&lt;li&gt;Measure again, three runs, and compare medians, not your best run.&lt;/li&gt;
&lt;li&gt;Decide whether the short visits you may lose matter for your kind of site.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Does delaying Google Tag Manager improve page speed?
&lt;/h3&gt;

&lt;p&gt;It usually improves the numbers that measure how fast the page itself appears. On this site the median Largest Contentful Paint of an English post on mobile went from 3.95 s to 1.6 s, and the time the page was blocked fell from 125 ms to 50 ms. The Google files still download, after the page instead of with it.&lt;/p&gt;

&lt;h3&gt;
  
  
  Will I lose data if I delay Google Analytics?
&lt;/h3&gt;

&lt;p&gt;You may lose very short visits: a reader who leaves within a second or two without scrolling, tapping or pressing a key. Visits where the reader stays or touches the page are still counted, and events pushed earlier wait in the queue. You cannot measure how many visits you lose, because analytics never sees them.&lt;/p&gt;

&lt;h3&gt;
  
  
  Should I wait for the browser to be idle or for the first interaction?
&lt;/h3&gt;

&lt;p&gt;Use both and take whichever comes first. Waiting only for an interaction loses every reader who reads without scrolling. Waiting only for idle can be late on a busy phone. This site uses an idle callback with a 2,000 ms limit after the page loads, a 1,500 ms timer where the browser has no idle callback, and the first scroll, tap, click, wheel or key press.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why does Lighthouse sometimes show no Google Tag Manager after deferring it?
&lt;/h3&gt;

&lt;p&gt;Because Lighthouse stops measuring when the network goes quiet, which can happen before the idle moment. In two of nine runs on this site, the test ended before Google loaded. Real readers still download the files. Compare medians of several runs and check real timings in a browser rather than trusting one score.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>performance</category>
      <category>javascript</category>
      <category>analytics</category>
    </item>
    <item>
      <title>I removed my Chinese web font. Every page lost 1 MB</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Sat, 03 Oct 2026 06:18:31 +0000</pubDate>
      <link>https://dev.to/yappatyih/i-removed-my-chinese-web-font-every-page-lost-1-mb-286a</link>
      <guid>https://dev.to/yappatyih/i-removed-my-chinese-web-font-every-page-lost-1-mb-286a</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/chinese-web-font-cost-1mb/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;A system font stack is a list of fonts that are already installed on the reader's device, given to the browser in order of preference, so the page downloads no font at all. On 2 October 2026 I replaced the Chinese web font on this site, Noto Sans SC, with a system font stack. One Chinese post went from 20 font files and 1,107 KB of fonts to 2 files and 54 KB. The whole page went from 1,464 KB to 318 KB. Its Lighthouse mobile score went from 68 and 73 to 84 and 97 across two runs each. The change is four lines of CSS. What I gave up is one identical look on every device. What it did not fix is below too, because the first live measurement was worse than I expected, and the reason turned out to be mostly the measurement itself.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a Chinese page was carrying
&lt;/h2&gt;

&lt;p&gt;Before the change, every Chinese page on this site carried 94,098 bytes of font instructions inside its own HTML, before a single font file was requested. That is 100 &lt;code&gt;@font-face&lt;/code&gt; rules, the same 100 on every page.&lt;/p&gt;

&lt;p&gt;This site is bilingual. Every page exists in English and in Simplified Chinese, so the English page is a fair control for its Chinese twin. Same layout, same pictures, same words in another language.&lt;/p&gt;

&lt;p&gt;The English version of &lt;a href="https://yappatyih.com/writing/the-mamak-table-question/" rel="noopener noreferrer"&gt;the mamak table post&lt;/a&gt; was 51 KB of HTML. The Chinese version was 143 KB. After the HTML arrived, the Chinese page went on to download 20 font files, 1,107 KB in total. The Chinese home page downloaded 16 files, 857 KB.&lt;/p&gt;

&lt;p&gt;On a phone, Lighthouse scored that Chinese post 55 and 65 on two runs. The English one had scored 97 two days earlier.&lt;/p&gt;

&lt;p&gt;A reader in Kuala Lumpur on mobile data was paying for a megabyte of font to read one blog post.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a Chinese font is so heavy
&lt;/h2&gt;

&lt;p&gt;A Latin font needs a few hundred shapes. A Chinese font needs many thousands, and each one is a drawing.&lt;/p&gt;

&lt;p&gt;Google Fonts handles this well for what it is. It cuts a Chinese family into about a hundred slices and tells the browser which characters live in which slice. That is what the 100 rules were. The browser then downloads only the slices that contain the characters on the page.&lt;/p&gt;

&lt;p&gt;The catch is that ordinary Chinese prose is spread across many slices. One post of mine needed 20 of them. The slicing saved me from a font of several megabytes and still left me with one megabyte.&lt;/p&gt;

&lt;p&gt;It also has a cost that is easy to miss: the 100 rules themselves. They were inlined in every Chinese page so the font could start loading early, which made every Chinese page nearly three times the size of its English twin before any font arrived.&lt;/p&gt;

&lt;h2&gt;
  
  
  The change: four lines of CSS
&lt;/h2&gt;

&lt;p&gt;The fix was to stop sending a Chinese font and name the ones readers already have. Every iPhone, iPad and Mac ships with PingFang SC. Android ships with Noto Sans CJK, the same design I had been downloading. Windows ships with Microsoft YaHei.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="nt"&gt;html&lt;/span&gt;&lt;span class="o"&gt;[&lt;/span&gt;&lt;span class="nt"&gt;lang&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;'zh-Hans'&lt;/span&gt;&lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="py"&gt;--font-zh&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;'PingFang SC'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;'Hiragino Sans GB'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;'Noto Sans CJK SC'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;'Noto Sans SC'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;'Source Han Sans SC'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;'Microsoft YaHei'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;sans-serif&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="py"&gt;--font-sans&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;'Geist Variable'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;var&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;--font-zh&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The third line puts the Latin font first, so English words and numbers inside Chinese text are still drawn in this site's own typeface, Geist. The browser falls through to the Chinese list only for Chinese characters.&lt;/p&gt;

&lt;p&gt;That is the whole change in the stylesheet. The rest was deleting things: the font entry in the build configuration, the tag that loaded it, and the 100 rules that came with it.&lt;/p&gt;

&lt;p&gt;I work with two Claude sessions, &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;a Brain that plans and a Hand that builds&lt;/a&gt;. The Brain proposed this. I did not approve it on the idea. I approved it after the Hand had measured it and put screenshots of both versions side by side.&lt;/p&gt;

&lt;h2&gt;
  
  
  Before and after
&lt;/h2&gt;

&lt;p&gt;After the change, a Chinese page requests exactly the same two font files as an English page. These are the numbers for the Chinese mamak table post, measured with Lighthouse mobile on the same machine, the same day, two runs each.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Font files&lt;/td&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Font weight&lt;/td&gt;
&lt;td&gt;1,107 KB&lt;/td&gt;
&lt;td&gt;54 KB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;HTML&lt;/td&gt;
&lt;td&gt;143 KB&lt;/td&gt;
&lt;td&gt;51 KB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;HTML, compressed&lt;/td&gt;
&lt;td&gt;48 KB&lt;/td&gt;
&lt;td&gt;15.6 KB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Whole page&lt;/td&gt;
&lt;td&gt;1,464 KB&lt;/td&gt;
&lt;td&gt;318 KB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Largest Contentful Paint&lt;/td&gt;
&lt;td&gt;3.7 s and 4.4 s&lt;/td&gt;
&lt;td&gt;2.9 s and 2.2 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lighthouse mobile score&lt;/td&gt;
&lt;td&gt;68 and 73&lt;/td&gt;
&lt;td&gt;84 and 97&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The Chinese home page went from 1,796 KB to 901 KB, and from scores of 40 and 42 to 51 and 55. It is still low, and I know why: the home page runs a 3D scene, which is &lt;a href="https://yappatyih.com/writing/the-night-my-post-pages-scored-39/" rel="noopener noreferrer"&gt;a separate cost I wrote about&lt;/a&gt;. The font was half of that page's weight. It was not the whole problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I gave up
&lt;/h2&gt;

&lt;p&gt;What I gave up is control: the same page now looks slightly different on an iPhone, an Android phone and a Windows laptop. With a web font, every reader saw the same shapes.&lt;/p&gt;

&lt;p&gt;One thing had to be checked before I agreed: that bold is really bold. A system font with only one weight makes the browser fake the bold by smearing the regular shape, and it shows. On a Mac, the browser drew the real faces: PingFang SC Regular for body text, Medium for headings, Semibold for bold.&lt;/p&gt;

&lt;p&gt;I have not seen this site on Windows or on an Android phone with my own eyes, as of this writing. Microsoft YaHei has fewer weights than PingFang, so headings there may look lighter than I designed. I accepted that. A page that arrives is worth more than a page that matches.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it did not fix
&lt;/h2&gt;

&lt;p&gt;The first live measurement after shipping scored 76, not 97. The largest element took 5.6 seconds to appear. I had saved a megabyte and the phone score barely showed it.&lt;/p&gt;

&lt;p&gt;I went looking for a cause and found a real one. The same day I had changed how analytics loads, and two Google scripts, 297 KB together, were now starting with every page and competing with the content. I moved them to load after the page is idle or at the reader's first touch. Three live runs on the same Chinese post after that: 92, 97 and 99.&lt;/p&gt;

&lt;p&gt;But the scripts do not explain the 76. Before that fix went out, the same page on the same code was measured twice more and scored 95 and 92. The 76 was mostly one bad run. Moving the scripts lifted the typical score by a few points, not by twenty.&lt;/p&gt;

&lt;p&gt;Two lessons from that. One measurement is not a result: the same page on the same code scored 76 and 95 a few hours apart, and I nearly built a story on the 76. And a page has more than one weight on it. Removing the biggest one mostly reveals the next.&lt;/p&gt;

&lt;p&gt;One small thing is still open. In Chinese, a line should not start with a particle like 的. CSS has a property meant to break lines by phrase. I tried it. In the browsers I tested, it did nothing for Chinese. I left it.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to check your own site
&lt;/h2&gt;

&lt;p&gt;You can find out what your Chinese pages download in about two minutes, with no paid tool.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open a Chinese page of your site in Chrome on a computer.&lt;/li&gt;
&lt;li&gt;Open DevTools, go to Network, choose the Font filter, and reload.&lt;/li&gt;
&lt;li&gt;Count the files and read the total at the bottom. If you see twenty files or a megabyte, you have what I had.&lt;/li&gt;
&lt;li&gt;View the page source and search for &lt;code&gt;@font-face&lt;/code&gt;. If the count is in the dozens, your HTML is carrying the font instructions too.&lt;/li&gt;
&lt;li&gt;Open the same page in your other language and compare the two totals. The difference is what your Chinese readers pay.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If the number is large, try the stack above on one page, look at it on your own phone, and decide with your eyes.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Should I use a web font or system fonts for a Chinese website?
&lt;/h3&gt;

&lt;p&gt;For body text, use system fonts. A Chinese web font costs roughly a megabyte per page even when it is sliced and loaded well, and every modern phone and computer already has a good Chinese font installed. Keep a web font only for a short headline or a logo where the exact shape is the brand, and only after measuring what it costs.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why is Noto Sans SC so slow to load?
&lt;/h3&gt;

&lt;p&gt;Because Chinese needs thousands of characters and each one is a drawing. Google Fonts splits Noto Sans SC into about a hundred slices so a page downloads only what it uses, but ordinary prose still touches many slices. One ordinary post on this site needed 20 font files, 1,107 KB.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is a good CSS font stack for Simplified Chinese?
&lt;/h3&gt;

&lt;p&gt;This site uses &lt;code&gt;"PingFang SC", "Hiragino Sans GB", "Noto Sans CJK SC", "Noto Sans SC", "Source Han Sans SC", "Microsoft YaHei", sans-serif&lt;/code&gt;, with the Latin font placed before it. It covers Apple devices, Android and Windows in that order, and ends in the browser's own default.&lt;/p&gt;

&lt;h3&gt;
  
  
  Do system fonts make a Chinese website look worse?
&lt;/h3&gt;

&lt;p&gt;They make it look slightly different on each device, not worse. PingFang SC, Noto Sans CJK and Microsoft YaHei are the fonts your readers see in every app on their own phone. Check that bold text uses a real bold face, and look at the page on a real device before you decide.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>performance</category>
      <category>css</category>
      <category>frontend</category>
    </item>
    <item>
      <title>Can AI crawlers read your site? I checked mine</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Sat, 03 Oct 2026 03:38:07 +0000</pubDate>
      <link>https://dev.to/yappatyih/can-ai-crawlers-read-your-site-i-checked-mine-p0i</link>
      <guid>https://dev.to/yappatyih/can-ai-crawlers-read-your-site-i-checked-mine-p0i</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/can-ai-crawlers-read-your-site/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;An AI crawler access check is a test of whether the bots behind ChatGPT, Claude, Perplexity and the others can fetch your pages, by requesting a page with each bot's name and reading the status code. On 30 September 2026 I ran one on this site: eight crawler names, eight answers of 200. Nothing was keeping the machines out. What the same audit found instead was a line in my own robots.txt, &lt;code&gt;Disallow: /og/&lt;/code&gt;, that blocked the preview image of every page without a cover, plus a sitemap address that returned 404, one company with two ids, and dates that could claim a post was modified before it was published. All fixed the same day. On 2 October I ran the check again with ten names: ten times 200, 54,570 bytes each. Two of my original eight names turned out not to be crawlers at all. The command, the table and the five checks are below.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why I checked
&lt;/h2&gt;

&lt;p&gt;Since July 2025, Cloudflare has blocked AI crawlers by default on new domains unless the owner says otherwise. This site runs on Cloudflare. I write here so that people, search engines and answer engines can find what I think, and an answer engine cannot quote a page it was never allowed to fetch.&lt;/p&gt;

&lt;p&gt;So before doing anything clever, I wanted the dull question answered: can they get in at all?&lt;/p&gt;

&lt;p&gt;I run this site with two Claude sessions, &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;a Brain that plans and a Hand that builds&lt;/a&gt;. The audit was the Brain's job, read-only: look at the live site from the outside, change nothing, write down what is true.&lt;/p&gt;

&lt;h2&gt;
  
  
  The check, in one command
&lt;/h2&gt;

&lt;p&gt;Every crawler announces itself with a user-agent string. You can send the same string yourself and see what your server answers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-o&lt;/span&gt; /dev/null &lt;span class="nt"&gt;-w&lt;/span&gt; &lt;span class="s2"&gt;"%{http_code} %{size_download}&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-A&lt;/span&gt; &lt;span class="s2"&gt;"Mozilla/5.0 (compatible; GPTBot/1.1)"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  https://yappatyih.com/writing/the-mamak-table-question/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A 200 with the full page size means that name is let in. A 403, a challenge page or a much smaller size means something in front of your site is turning it away.&lt;/p&gt;

&lt;p&gt;Here is the run from 2 October 2026, against one post on this site:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Name&lt;/th&gt;
&lt;th&gt;Whose&lt;/th&gt;
&lt;th&gt;What it does&lt;/th&gt;
&lt;th&gt;Status&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;GPTBot&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;collects pages for training&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OAI-SearchBot&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;builds ChatGPT's search index&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ChatGPT-User&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;fetches a page when a user asks&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ClaudeBot&lt;/td&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;collects pages for training&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude-User&lt;/td&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;fetches a page when a user asks&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PerplexityBot&lt;/td&gt;
&lt;td&gt;Perplexity&lt;/td&gt;
&lt;td&gt;builds Perplexity's index&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CCBot&lt;/td&gt;
&lt;td&gt;Common Crawl&lt;/td&gt;
&lt;td&gt;open web archive many models use&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bytespider&lt;/td&gt;
&lt;td&gt;ByteDance&lt;/td&gt;
&lt;td&gt;collects pages for its models&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Google-Extended&lt;/td&gt;
&lt;td&gt;Google&lt;/td&gt;
&lt;td&gt;not a crawler, see below&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Applebot-Extended&lt;/td&gt;
&lt;td&gt;Apple&lt;/td&gt;
&lt;td&gt;not a crawler, see below&lt;/td&gt;
&lt;td&gt;200&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Every one returned the same 54,570 bytes a browser gets.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the check cannot tell you
&lt;/h2&gt;

&lt;p&gt;A 200 from my laptop proves less than it looks like, and I would rather say so than oversell it.&lt;/p&gt;

&lt;p&gt;First, I was pretending. The request came from my own address with a borrowed name. It shows that nothing in front of the site blocks by name. It does not show what happens to the real crawler, coming from its own network, if a firewall judges it by address or behaviour. The only full proof is your server log: the real bot, the real status code.&lt;/p&gt;

&lt;p&gt;Second, two of my eight names were wrong. Google-Extended and Applebot-Extended are not crawlers. They are words you put in robots.txt to tell Google and Apple whether pages their normal crawlers already fetched may be used for AI. Nothing ever visits your site calling itself Google-Extended, so testing that name with curl tests nothing. My audit of 30 September listed both as crawlers that "receive 200". That line was true and meaningless. For those two, the check is to read your robots.txt, not to send a request.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the log says
&lt;/h2&gt;

&lt;p&gt;After this post went out I looked at the log I said I had not checked. Cloudflare keeps one for AI crawlers. From 28 September to the morning of 2 October 2026 it counted 959 requests to this site from 17 crawlers, search engines among them, and no crawler was blocked.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Crawler&lt;/th&gt;
&lt;th&gt;Whose&lt;/th&gt;
&lt;th&gt;Requests answered&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;GPTBot&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;217&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ClaudeBot&lt;/td&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;158&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;OAI-SearchBot&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;31&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ChatGPT-User&lt;/td&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;27&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CCBot&lt;/td&gt;
&lt;td&gt;Common Crawl&lt;/td&gt;
&lt;td&gt;24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;PerplexityBot&lt;/td&gt;
&lt;td&gt;Perplexity&lt;/td&gt;
&lt;td&gt;14&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bytespider&lt;/td&gt;
&lt;td&gt;ByteDance&lt;/td&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Claude-User&lt;/td&gt;
&lt;td&gt;Anthropic&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;So the knock on the door was honest: the real crawlers get the same 200 my borrowed names got. This post itself was fetched 17 times in its first three hours. And Google-Extended and Applebot-Extended appear nowhere in the list, which is what you would expect of two names that are not crawlers.&lt;/p&gt;

&lt;p&gt;One more thing the log showed. Something calling itself an AI crawler asked this site for files named &lt;code&gt;.env.production&lt;/code&gt; and &lt;code&gt;credentials.json&lt;/code&gt;. It got a 404 each time, because those files are not there. A crawler's name is only a name. Let the real ones in, and keep nothing on your site that you would mind a stranger asking for.&lt;/p&gt;

&lt;h2&gt;
  
  
  What was actually blocked: my own images
&lt;/h2&gt;

&lt;p&gt;The audit's first real finding had nothing to do with AI. My robots.txt said:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight apache"&gt;&lt;code&gt;&lt;span class="nc"&gt;User&lt;/span&gt;-agent: *
&lt;span class="nc"&gt;Allow&lt;/span&gt;: /
Disallow: /og/
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It looked tidy: a folder of utility images, kept away from crawlers. But &lt;code&gt;/og/&lt;/code&gt; held &lt;code&gt;en.png&lt;/code&gt; and &lt;code&gt;zh.png&lt;/code&gt;, the preview image that every page without its own cover hands to anything that asks for one. Preview fetchers that obey robots.txt were being told, by me, not to pick up the picture I had made for them.&lt;/p&gt;

&lt;p&gt;The fix was one deleted line. The file is now four lines long, and &lt;code&gt;/og/en.png&lt;/code&gt; answers 200 as &lt;code&gt;image/png&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The same pass found three more things of the same kind, each a small lie the site was telling machines:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;A sitemap address that did not exist.&lt;/strong&gt; &lt;code&gt;/sitemap.xml&lt;/code&gt; returned 404 while Bing still had it on file. It now redirects to the real index.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One company, two ids.&lt;/strong&gt; In the structured data, my profile called BlackRevo by one id and BlackRevo's own page on this site called it by another. To a machine, that is two companies. Now each organisation has one id everywhere, the same one its own website uses.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dates that ran backwards.&lt;/strong&gt; A post's &lt;code&gt;lastmod&lt;/code&gt; in the sitemap was the day it was last saved, in UTC, so a post saved before its publish time claimed to be modified before it existed. It now reads the publish time until a real edit happens.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The Hand fixed all of it on 30 September and left 102 passing tests behind, including ones that pin the ids and the sitemap dates.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I stopped chasing
&lt;/h2&gt;

&lt;p&gt;Every post here ends with a FAQ, marked up as FAQPage data. I used to think of that as a way to win the expandable questions under a Google result. That prize is gone: Google limited FAQ rich results to government and health sites in August 2023, and its documentation now says they are no longer shown at all from 7 May 2026.&lt;/p&gt;

&lt;p&gt;I kept the FAQ anyway and changed what it is for. Each question is now a real heading with its own link, written the way a person would ask it, with an answer that stands on its own. That is not for a rich result. It is for a reader who skims, and for an answer engine that reads a page in pieces and needs each piece to make sense alone.&lt;/p&gt;

&lt;p&gt;The heavy thing on this site, the three.js world behind the pages, was measured in the same audit. That became its own story: &lt;a href="https://yappatyih.com/writing/the-night-my-post-pages-scored-39/" rel="noopener noreferrer"&gt;the night my post pages scored 39&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  The five checks
&lt;/h2&gt;

&lt;p&gt;This is the whole method. It takes ten minutes and no paid tool.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Read your robots.txt as a stranger.&lt;/strong&gt; Open it in a browser. For every Disallow line, name what lives there. If you cannot, find out before a crawler does.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Knock with each crawler's name.&lt;/strong&gt; Run the curl command above for each real crawler on one page. Same status and same size as a browser, or you have a block to find.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fetch everything your pages point to.&lt;/strong&gt; The &lt;code&gt;og:image&lt;/code&gt;, the sitemap, the canonical address, the feed. Check the status and the content type, not just that something came back.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One thing, one id.&lt;/strong&gt; In your structured data, a person or a company gets exactly one id, the same on every page and on every site you control.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Dates that cannot lie.&lt;/strong&gt; Published, modified and sitemap dates should agree with each other and with what happened.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;None of this makes an answer engine quote you. It only removes your own reasons for it not to. What gets quoted is still a page that says one true, specific thing clearly. This list is how you make sure that page can be reached.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  How do I check if AI crawlers can access my website?
&lt;/h3&gt;

&lt;p&gt;Send a request to one of your pages with the crawler's user-agent string, for example with &lt;code&gt;curl -A "Mozilla/5.0 (compatible; GPTBot/1.1)"&lt;/code&gt;, and read the status code and size. A 200 with the full page means that name is allowed. Repeat for each crawler you care about, then confirm in your server log that the real bots receive 200 too.&lt;/p&gt;

&lt;h3&gt;
  
  
  Does Cloudflare block AI crawlers by default?
&lt;/h3&gt;

&lt;p&gt;For domains added since July 2025, Cloudflare blocks known AI crawlers unless the owner chooses to allow them, and it can also manage your robots.txt. On this site, tested on 30 September and 2 October 2026, nothing was blocked and robots.txt was served exactly as written. If your site is on Cloudflare, look at the AI Crawl Control section of the dashboard and run the check yourself.&lt;/p&gt;

&lt;h3&gt;
  
  
  Is Google-Extended a crawler?
&lt;/h3&gt;

&lt;p&gt;No. Google-Extended is a token for robots.txt that tells Google whether content its normal crawlers fetch may be used for its AI models. Nothing visits your site with that name. Applebot-Extended works the same way for Apple. You control both in robots.txt, not in a firewall.&lt;/p&gt;

&lt;h3&gt;
  
  
  Is FAQ schema still worth adding in 2026?
&lt;/h3&gt;

&lt;p&gt;Not for a rich result: Google no longer shows FAQ rich results as of 7 May 2026. I still write a FAQ on every post, as real headings with self-contained answers, because readers skim to them and answer engines read pages in pieces. The structured data costs nothing to keep.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>seo</category>
      <category>ai</category>
      <category>webdev</category>
      <category>cloudflare</category>
    </item>
    <item>
      <title>The night my post pages scored 39</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Fri, 02 Oct 2026 02:11:16 +0000</pubDate>
      <link>https://dev.to/yappatyih/the-night-my-post-pages-scored-39-2b26</link>
      <guid>https://dev.to/yappatyih/the-night-my-post-pages-scored-39-2b26</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/the-night-my-post-pages-scored-39/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;This site draws a particle world behind every page, tens of thousands of points in the shape of my mark, in three.js. I chose it knowing the cost and wrote the decision down: weak devices are sacrificed by choice. Then I had it measured. A post page on a simulated phone scored 39 out of 100 on Lighthouse, with 3636 ms of blocked main thread and a largest paint at 5.70 s. The same page without the world scored 97. I kept the world where it is the point, the home page, and stopped loading it on post pages unless the screen is wide, the pointer is a mouse and the reader has not asked for less motion. The post page now scores 97 on a phone and still draws the world on a desktop. A day later the world was crossing the text on desktop too, so a veil went behind the column, taking the worst spot from 1.59:1 contrast to 7.33:1. At the end is the test I now run before I sacrifice anything: what is this page for.&lt;/p&gt;

&lt;h2&gt;
  
  
  A decision I made on purpose
&lt;/h2&gt;

&lt;p&gt;When this site was rebuilt in September I made a choice most advice says not to make. The background is a live three.js scene: a single brush stroke, drawn with tens of thousands of points, that takes three forms, a stroke, a circuit, a galaxy. It is the mark of the site in motion. It is heavy, and I knew it.&lt;/p&gt;

&lt;p&gt;So I wrote the decision down, in the handbook my planning session reads first every day: heavy three.js is accepted; weak devices are sacrificed by choice. Content must still be real HTML for search. I did not want to be talked out of it one small compromise at a time.&lt;/p&gt;

&lt;p&gt;I still think that was the right way to decide. Write it down, and you can find out later exactly what you were wrong about.&lt;/p&gt;

&lt;h2&gt;
  
  
  Measure first, change nothing
&lt;/h2&gt;

&lt;p&gt;I run this site with two Claude sessions, &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;a Brain that plans and a Hand that builds&lt;/a&gt;. On 30 September, after &lt;a href="https://yappatyih.com/writing/can-ai-crawlers-read-your-site/" rel="noopener noreferrer"&gt;an SEO audit&lt;/a&gt;, the Brain's prompt to the Hand had one step that was different from the others: measure the world's cost on a post page, and change nothing.&lt;/p&gt;

&lt;p&gt;The Hand ran Lighthouse in its mobile simulation against a local copy of the mamak post, once with the world and once without, same page, same build.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Post page, phone&lt;/th&gt;
&lt;th&gt;With the world&lt;/th&gt;
&lt;th&gt;Without&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Score&lt;/td&gt;
&lt;td&gt;39&lt;/td&gt;
&lt;td&gt;97&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Largest Contentful Paint&lt;/td&gt;
&lt;td&gt;5.70 s&lt;/td&gt;
&lt;td&gt;2.21 s&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total Blocking Time&lt;/td&gt;
&lt;td&gt;3636 ms&lt;/td&gt;
&lt;td&gt;0 ms&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Page weight&lt;/td&gt;
&lt;td&gt;733 KB&lt;/td&gt;
&lt;td&gt;191 KB&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Thirty-nine. Three and a half seconds in which a phone could not respond to a touch, on a page whose only job is to be read.&lt;/p&gt;

&lt;p&gt;The home page told a different story: 95 with the world, 97 without. The same scene cost two points there and fifty-eight points on a post. The difference is not the code. It is what the page is for.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sacrifice by page, not by site
&lt;/h2&gt;

&lt;p&gt;The lazy fix is a switch: world on or world off. The honest one is to ask each page what it is for.&lt;/p&gt;

&lt;p&gt;The home page is the mark. The world is the point of it, the thing a visitor came to see, and on a phone it costs two points. It stays.&lt;/p&gt;

&lt;p&gt;A post page is for reading. Nobody opens a long essay on a phone to watch particles behind it. There, the world is pure cost, and I had been charging every phone reader 3.6 seconds for a decoration they could barely see behind the text.&lt;/p&gt;

&lt;p&gt;So the rule became: a post page draws the world only when all three of these are true. The screen is at least 1024 px wide. The pointer is fine, a mouse or trackpad, which is the cheapest honest signal that this is a laptop and not a phone in landscape. And the reader has not set their system to prefer reduced motion. Fail any one, and the page never downloads the world at all. It gets a still ground instead: the same near-black, one soft crimson glow near the top, not animated.&lt;/p&gt;

&lt;p&gt;The Hand built it, with tests that pin the three conditions and all eight combinations of them. Measured again:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Post page&lt;/th&gt;
&lt;th&gt;Score&lt;/th&gt;
&lt;th&gt;LCP&lt;/th&gt;
&lt;th&gt;TBT&lt;/th&gt;
&lt;th&gt;Weight&lt;/th&gt;
&lt;th&gt;World loaded&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Phone&lt;/td&gt;
&lt;td&gt;97&lt;/td&gt;
&lt;td&gt;2.18 s&lt;/td&gt;
&lt;td&gt;0 ms&lt;/td&gt;
&lt;td&gt;191 KB&lt;/td&gt;
&lt;td&gt;no&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Desktop&lt;/td&gt;
&lt;td&gt;98&lt;/td&gt;
&lt;td&gt;0.98 s&lt;/td&gt;
&lt;td&gt;81 ms&lt;/td&gt;
&lt;td&gt;734 KB&lt;/td&gt;
&lt;td&gt;yes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Same page, same essay. The desktop reader still gets the world. The phone reader gets the essay.&lt;/p&gt;

&lt;h2&gt;
  
  
  The world came back for the text
&lt;/h2&gt;

&lt;p&gt;A day later I was reading the bottom of a post on my own laptop and could barely make out the author box. The ring was crossing the text column. The dark pool behind the article was a soft radial gradient, fine in the middle of a short page, gone by the bottom of a long one.&lt;/p&gt;

&lt;p&gt;Measured at the worst spot, where the ring crossed an excerpt, the text had 1.59:1 contrast against the pixels behind it. The minimum for body text is 4.5:1.&lt;/p&gt;

&lt;p&gt;The fix is a veil: a dark band at 0.8 opacity behind the whole text column, full height, feathered at the sides so there is no box, and only on desktop where the world runs. The same spot now measures 7.33:1. The ring is still there, at the sides and faintly through the band. The words come first.&lt;/p&gt;

&lt;p&gt;Two days, two corrections to one decision, and I would make the decision again. The world is still the identity of this site. It just stopped charging people who came to read.&lt;/p&gt;

&lt;h2&gt;
  
  
  The page's-job test
&lt;/h2&gt;

&lt;p&gt;Before I let any page pay for anything, four questions.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;What is this page for?&lt;/strong&gt; One verb. Watch, read, decide, buy. If the expensive thing is not the verb, it is a decoration, and decorations are the first thing to go on a slow device.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Measure on the phone, not the laptop.&lt;/strong&gt; My laptop said the world was fine. It was fine, on a laptop. Lighthouse's mobile simulation was the first thing that told me the truth, and it took one prompt.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sacrifice by page, not by site.&lt;/strong&gt; Global switches punish the page where the expensive thing was the point. Decide per page, and write down the conditions so they can be tested.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Keep the identity in a still form.&lt;/strong&gt; Removing the world from phones did not mean removing the brand. The still ground is the same colour and the same glow. Nobody who arrives on a phone sees a different site, only a quieter one.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And write the original decision down before you measure it. A decision you cannot find is one you cannot correct.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  What is Lighthouse?
&lt;/h3&gt;

&lt;p&gt;Google's open-source page audit, built into Chrome. It loads a page under a simulated slow phone and network and scores it from 0 to 100 on performance, from metrics such as Largest Contentful Paint, Total Blocking Time and Cumulative Layout Shift. The numbers in this post are its mobile simulation on a local preview, so they compare like with like.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why not just remove three.js from the site?
&lt;/h3&gt;

&lt;p&gt;Because on the home page it is the point, and there it costs two points on a phone. The cost was only unacceptable on pages whose job is reading. So the decision is per page: the home page always draws the world; a post page draws it only on a wide screen with a fine pointer and no reduced-motion preference.&lt;/p&gt;

&lt;h3&gt;
  
  
  How does a page know it is on a phone?
&lt;/h3&gt;

&lt;p&gt;It does not, exactly. It asks three media queries: is the viewport at least 1024 px wide, is the pointer fine, and has the reader not asked for reduced motion. All three must be true before the world is downloaded. A phone in landscape fails the pointer test; a small laptop window fails the width test and gets the still ground, which is acceptable.&lt;/p&gt;

&lt;h3&gt;
  
  
  Did the change hurt the desktop experience?
&lt;/h3&gt;

&lt;p&gt;No. The same post page scores 98 on desktop with the world loaded. What the desktop needed was a reading veil behind the text column, added the next day, which took the worst contrast on the page from 1.59:1 to 7.33:1.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>performance</category>
      <category>threejs</category>
      <category>javascript</category>
    </item>
    <item>
      <title>Why I run my company on a Brain and a Hand</title>
      <dc:creator>Yap Pat Yih</dc:creator>
      <pubDate>Fri, 02 Oct 2026 02:11:06 +0000</pubDate>
      <link>https://dev.to/yappatyih/why-i-run-my-company-on-a-brain-and-a-hand-f2g</link>
      <guid>https://dev.to/yappatyih/why-i-run-my-company-on-a-brain-and-a-hand-f2g</guid>
      <description>&lt;p&gt;&lt;em&gt;First published on &lt;a href="https://yappatyih.com/writing/brain-and-hand-claude-code-workflow/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;I run two kinds of Claude Code session. The Brain reads, investigates, decides and writes a brief. The Hand pastes that brief, builds, commits and reports back. The Brain never touches code. This site was rebuilt that way in two days: one Brain session, six Hand sessions, just over a hundred commits. The same split runs Lumen, the internal system at BlackRevo, and Habitus Mind. It works because it is delegation, and delegation is a craft I already had to learn as a General Manager.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with one session doing everything
&lt;/h2&gt;

&lt;p&gt;The first way everyone uses an AI coding tool is one long conversation. You think out loud, it builds, you correct, it builds again. It works for an afternoon. Then the conversation is holding your strategy, your half-finished decisions, four hundred file reads and every wrong turn, and the model that is supposed to be thinking about what to build is spending its attention on how it built the last thing.&lt;/p&gt;

&lt;p&gt;Two jobs are fighting for one context. Judgement and execution.&lt;/p&gt;

&lt;h2&gt;
  
  
  The split
&lt;/h2&gt;

&lt;p&gt;So I separate them into two sessions with two names.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Brain&lt;/strong&gt; reads the sources, investigates the repository without editing it, proposes, asks me numbered questions when a decision is mine, and then writes one self-contained brief. It runs on the most capable model I can get, because judgement is where quality is decided.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Hand&lt;/strong&gt; is a fresh session. It gets the brief pasted in, and nothing else. It builds, verifies in the browser, commits after each step with a plain-language subject, and ends with a report in the format the brief asked for. It never pushes, never deploys, never touches production, unless the brief says so in those words.&lt;/p&gt;

&lt;p&gt;Then I paste the Hand's report back into the Brain, and the Brain reviews it before writing the next brief.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a brief contains
&lt;/h2&gt;

&lt;p&gt;Every brief has the same shape. It is the shape of a good handover to a person.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Where to work and what to read first, by file name.&lt;/li&gt;
&lt;li&gt;What it must not do. No push, no deploy, no credentials, no production resources.&lt;/li&gt;
&lt;li&gt;What this is: two paragraphs of context, so the Hand understands the why, not only the what.&lt;/li&gt;
&lt;li&gt;Standing rules for the project. No em-dashes in copy. Every string in both languages. No inline scripts because of the security policy. Verify in the browser.&lt;/li&gt;
&lt;li&gt;Numbered steps, each ending in a commit with a given subject.&lt;/li&gt;
&lt;li&gt;Report back: exactly what I want to see, including "anything you changed that this brief did not ask for" and "anything you could not do, with the reason".&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That last section is the whole trick. A Hand that must confess its unasked changes makes better ones. A Hand that must explain what it could not do stops pretending.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two days, in numbers
&lt;/h2&gt;

&lt;p&gt;This site is the freshest example. On 27 September I decided to rebuild it. By the evening of the 28th:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;One Brain session, six Hand sessions.&lt;/li&gt;
&lt;li&gt;Just over a hundred commits, every one with a subject a human can read.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://yappatyih.com/writing/the-night-my-post-pages-scored-39/" rel="noopener noreferrer"&gt;A particle world in three.js&lt;/a&gt;, two languages, a database-backed blog, an owner login built to a written security checklist, a new mark with an identity film, and a go-live runbook.&lt;/li&gt;
&lt;li&gt;Zero lines of code written by the Brain.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I did not review a hundred commits. I reviewed six reports.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where it broke, and why that was fine
&lt;/h2&gt;

&lt;p&gt;The method is not magic, and the honest part is where it bent.&lt;/p&gt;

&lt;p&gt;The Hand discovered mid-brief that the hosting adapter could not build for the platform I had planned. It stopped and asked instead of guessing. I chose. That is the rule working.&lt;/p&gt;

&lt;p&gt;The Brain specified a two-step sign-in with an authenticator app. The Hand built it well. Then I decided I did not want it, so the next brief switched it off and kept the code behind a setting. Decisions belong to the owner, and the method makes that cheap.&lt;/p&gt;

&lt;p&gt;One of the logo's three forms came out wrong. The Brain looked at the frames, said exactly what was wrong, and the next brief rebuilt only that form. A brief that names the fault is cheaper than a conversation about it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a General Manager likes this
&lt;/h2&gt;

&lt;p&gt;Running a company is mostly briefing people well, setting boundaries, and reading reports honestly. Nobody senior does the work and the deciding in one breath. You would not hire a builder and then stand behind them narrating.&lt;/p&gt;

&lt;p&gt;The Brain and Hand split is that discipline, applied to a tool. The brief is the boundary. The report is the feedback loop. The rule that the Brain never codes is the same rule that says a manager should not grab the keyboard: the moment you do, you stop seeing the system.&lt;/p&gt;

&lt;p&gt;There is a line I keep coming back to: fairness is giving people what they need to succeed, not treating everyone the same. A Hand needs a complete brief and clear limits. A Brain needs the full picture and no keyboard. Give each what it needs.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to start
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Open two sessions and name them. Brain, Hand. The names matter more than they should.&lt;/li&gt;
&lt;li&gt;Tell the Brain, in one sentence, that it only writes briefs and never edits code. Tell it what the Hand may not do.&lt;/li&gt;
&lt;li&gt;Make the report section of every brief the longest part. What you can measure, you can delegate again.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Do the Brain and the Hand have to be different models?
&lt;/h3&gt;

&lt;p&gt;No. The split is about context and roles, not models. I put the most capable model on the Brain because judgement decides quality, and a strong model on the Hand because execution still has to be right.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why not let one session plan and then build?
&lt;/h3&gt;

&lt;p&gt;Because the conversation that planned is then carrying every detail of the build, and the next decision gets made with less room to think. A fresh Hand starts with only the brief.&lt;/p&gt;

&lt;h3&gt;
  
  
  What goes wrong most often?
&lt;/h3&gt;

&lt;p&gt;A brief that leaves a decision open. The Hand will make it, and it will be reasonable, and it will be wrong for you. Ask the Brain to send you numbered questions before it writes.&lt;/p&gt;

&lt;h3&gt;
  
  
  Is this only for software?
&lt;/h3&gt;

&lt;p&gt;No. I use the same split for keyword research and content briefs. Anything with a plan and an execution can be run this way.&lt;/p&gt;

&lt;p&gt;I write about running a company and building with AI at &lt;a href="https://yappatyih.com/writing/" rel="noopener noreferrer"&gt;yappatyih.com&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>productivity</category>
      <category>claude</category>
      <category>workflow</category>
    </item>
  </channel>
</rss>
