DEV Community: Mike Viewfy

ChatGPT stopped citing Reddit. The model kept reading the threads.

Mike Viewfy — Sun, 30 Aug 2026 08:42:35 +0000

Most readers saw ChatGPT drop Reddit citations and assumed public thread replies stopped reaching the model.
Promptwatch measured reddit.com at a 3.83% average daily share of all ChatGPT Search citations from July 18 through August 7, 2026, then 0.52% across August 14 to 17, a relative drop of 86.4%.
Trackerly found Reddit at 23 to 24% of the pages ChatGPT consulted in late June and early July, then between 25% and 34% on every single day of the collapse.

Citation share is not the same as being read

The fall came in two steps. On August 8 the share slid from the high 3s into the mid 2s. On August 14 it dropped below 1% and stayed there. Google's surfaces moved differently, with AI Overviews down 11.3% and AI Mode down 30.5% over 30 days, both gradual rather than a cliff.

The measurements say something narrower than the headlines. The link went away. The reading did not.

Measure	Before	After	Who measured it
Reddit share of ChatGPT Search citations	3.83% (Jul 18 to Aug 7)	0.52% (Aug 14 to 17)	Promptwatch
site: operator in ChatGPT fanout queries	0.37%	16.8%	Promptwatch, Aug 8
Searches per ChatGPT response	1.08	1.83	Promptwatch, Aug 8
Reddit share of pages ChatGPT consulted	23 to 24%	25 to 34%	Trackerly
Reddit share of Google AI Overviews citations	2.5%	2.1%	Promptwatch

A helpful reply in a live thread is still text a model reads when it goes looking for an answer to a buying question. It is still the thing a human reads when they land on that thread six months later. Those two readers did not leave when the citation share did.

ChatGPT started querying named domains instead of only sweeping the web

On August 8, Promptwatch measured ChatGPT's use of the site: operator rising from 0.37% to 16.8% of its fanout queries, roughly 46 times over. Average searches per response nearly doubled, from 1.08 to 1.83. ChatGPT started asking named websites directly instead of only sweeping the open web.

Named websites means help centers, docs and company blogs. If an engine walks up to your domain and asks it a question, owning a page that answers that exact question is the difference between getting read and getting skipped. Write the answer on your own site while you ship. Saving the post for after launch now means the named-domain query has nothing to fetch.

OpenAI did not confirm that ChatGPT stopped citing Reddit

Did OpenAI confirm a policy change? No. Promptwatch published the measurement on August 20, 2026 and stated that the data shows when the shift happened, not why, adding that a data-collection issue cannot be ruled out. Search Engine Journal reported OpenAI did not respond to a request for comment, and no Reddit statement has appeared.

Six days sit unexplained between the August 8 fanout change and the August 14 cliff. Hold 86.4% loosely. Two independent trackers, one measurement window, no confirmed cause. Treat it as a tracked observation.

A reply was never a citation play

If ChatGPT will not cite Reddit, is thread outreach dead? No. Trackerly found Reddit still made up 25% to 34% of the pages ChatGPT consulted on every day of the citation collapse, against 23 to 24% before it. The reply is read by the person who asked, by later searchers landing on that thread, and by the model. Only the printed link changed.

A comment that answers the real question still does that work. Citation share is one of those three outcomes, and it is the one you control least. The test does not move. If the draft stops being useful the moment you delete the product line, it was an ad wearing a comment costume. Do not send it.

Buyers do not ask in only one field

Stop treating one platform as the channel. Where your buyers ask has a different answer per category and it keeps moving: Reddit, LinkedIn, X, Threads, DEV, anywhere. Retrieval rules change without notice. That was already true after API restrictions and after AI moderation changed who gets to post.

What should you publish on your own site after this change? Answers to the questions your buyers actually type. ChatGPT's use of site:-scoped queries rose from 0.37% to 16.8% of fanout queries on August 8, 2026, so it now asks named domains directly. The post on your domain is what a later reply can point at as proof, not as empty outreach.

Two changes for a founder who ships. Publish the answer on your own domain. Show up in the threads where the question is already live, in the field they are already in.

This window cannot tell you why the citations dropped

Promptwatch was explicit that the data shows when the shift occurred, not why, and said a data-collection issue cannot be ruled out, calling the size of the drop provisional. No causal story is on the table. The consultation numbers and the citation numbers can diverge again next month.

The finding also does not apply if your buyers never ask in public indexed threads. Closed Slack, Discord, and email will not show up in these shares. Google AI Overviews only moved from 2.5% to 2.1% Reddit citation share, so a ChatGPT-only reading of "Reddit is over" is already too wide.

Ask ChatGPT and Claude the buying questions your customers actually type. Read the names that come back. Usually a competitor's. That gap is the brief for the next post and the next reply. It is a better use of an afternoon than refreshing a citation chart you do not control.

Did your own buying prompts still pull Reddit threads this week, and did any of them print a citation?
Full dataset and method: ChatGPT stopped citing Reddit

A founder who ships a feature in an afternoon will stall for three weeks on growth (the blank editor is the leak)

Mike Viewfy — Mon, 17 Aug 2026 16:46:30 +0000

Most founders think growth fails because they refuse to become a content machine on the side of the shipping job.
Wrong frame: a founder who ships a feature in an afternoon will stall for three weeks on growth, because the task has no first step.
Product has a queue, a diff, and a definition of done, and "do some marketing" has none of that, so it slides.

Product wins the same attention window because marketing has no definition of done

Both jobs want the same block of focus. Product wins by default. A ticket has a repro, a failing test, a merge. Growth has a fog: search for what, post where, write how, and by the time you pick a surface the evening is gone.

I have watched this in my own week and in every tiny team that can ship. The hard part was never the writing. It is finding the eight threads worth writing into, then publishing anything at all while a bug ticket sits open in the other tab.

Reddit, LinkedIn, X, Threads, DEV, anywhere: the buyers are already mid-question. The stall is not a talent gap. It is a missing first step.

Growth works when it lands as a review queue instead of a blank editor

You stop generating and start deciding. Threads arrive with a drafted reply attached. A blog post arrives written. SEO and GEO fixes arrive as pull requests against your repo, so the review is a merge button.

Nothing lands blank, which is the whole point. Reading a draft and killing it costs you almost nothing. Starting from an empty editor at 11pm costs you the evening.

Viewfy is the agent that runs that loop for a solo founder: he finds the thread, drafts the reply in the field you already sell into, and you send it. Same shape for the blog and the git-shaped fixes.

Piece of the loop	What arrives	What you keep
Thread scouting	Live threads where category buyers ask, on Reddit, LinkedIn, X, Threads, DEV, anywhere	Nothing until a draft lands
Community replies	A draft that still helps if you delete the product line	Edit or kill
Blog	A daily post	Read it, spike it if it is wrong
SEO and GEO fixes	Pull requests in your repo	Merge or close
AI citation checks	Who ChatGPT and Claude name for your buying questions	Decide what to publish about it
Meta ads	Only after the organic wedge works	Say when

That split is the whole product bet. Recommendations create work. Pull requests remove it.

Buyers ask in public, and those threads go cold in hours

They phrase it as a tool question. "What do you use for X." "Is Y worth the money." "How did you get your first hundred users." Those threads are the shortlist, and they have a half life measured in hours.

The ranking that matters is whether the asker sounds like your buyer instead of a bystander. On the plumbing side, official Reddit OAuth beats third-party scraper middlemen: search costs stay sane and you stay inside Reddit's terms. That is one surface. The same buyer question shows up on LinkedIn, X, Threads, and DEV.

If you are still hunting the thread by hand after a shipping day, you will lose to whoever answered while the question was still warm.

If the reply only works as a pitch, do not send it

One test. Delete the sentence that mentions your product and reread the comment. If it still helps the person who asked, send it. If the whole thing falls over, you wrote an ad wearing a comment costume.

That is why a useful draft is built to survive the deletion. Auto-posting bots skip that check, which is how communities end up banning a whole category of tool. Founders comparing this shape of work against ReplyGuy usually pick a side quickly: useful comment vs volume.

Show up carrying proof. An audit of the asker's problem, a published post, a diff. Empty outreach is just a pitch with better manners.

Recommendations create homework. Artifacts close the tab.

A list of "you should do these 40 things" is a second job you assigned yourself. A pull request is a merge button.

Same logic on your own site. SEO and GEO findings should arrive as diffs, not a PDF with 40 items you will never touch. If your instinct is to keep growth work inside the tools you already have open, the git-shaped version is the one that survives a shipping week.

Viewfy writes in their field, as your business. Thread scout leads. The blog, the SEO PRs, and the mentions are artifacts he carries into the conversation, not a separate content calendar you now have to feed.

ChatGPT and Claude name your competitors when they cannot read you

Often because the machines cannot read you, or because nothing on the open web describes what you do in the words buyers type. Both are fixable and neither shows up in your analytics.

A free check that reports which AI crawlers your site allows, and what a model can actually parse, is the first thing to rule out. We ran that read across 156 Show HN launches. Plenty of shipped products are invisible to the assistants their buyers ask first.

That is a different failure than "I did not post enough." You can write every day and still be missing from the answer that happens in a chat window.

A marketer is the wrong alternative before you have revenue

For a solo founder with no revenue yet, no. A marketer needs a brief, a strategy call, and a month before anything appears, and you will still be the one who knows what the product does.

The honest split in this category is whether the work arrives decided or blank. Viewfy is a bad fit if you want a fully autonomous poster or bulk community volume. Agencies selling spam, enterprise demand-gen teams that already have an SDR bench, and anyone shopping purely for Meta ad creative should look elsewhere. Paid ads belong after the organic wedge is already producing conversations.

The smallest version worth running is also the only honest trial: drop a URL, read what comes back about whether the machines can see you, then look at the first batch of threads. If the drafts read like something you would have written on a good day, the loop is real. If they only work as a pitch, kill them.

This is a workflow diagnosis, not a conversion study

I cannot tell you what a useful reply converts at. The three week stall is a pattern I trust, not a measured cohort with a control group. The 156 Show HN launches number is about crawler access and parseability, not about whether a drafted comment gets you a user.

If you already have a marketer with a queue and a definition of done, product is not eating your growth time, and none of this applies. If your buyers are not asking in public, thread scouting will look empty and you should not pretend otherwise.

How many days does a growth task sit in your notes before you open a blank editor?

Method and the longer version of this argument: marketing while building product without becoming a content machine

Reddit dropped karma gates. The LLM still kills the comment that only works as a pitch.

Mike Viewfy — Mon, 17 Aug 2026 08:43:25 +0000

Most people still think karma floors and Automod keyword lists are what block a first Reddit comment. Reddit is dropping those gates and scoring comments with Rules Hub, an LLM that reads rule intent instead of hunting keywords. Hide a URL in a synonym and you will still lose.

Karma used to stop you. Intent matching is the replacement.

Reddit said it is swapping karma and account-age gates for a model that reads whether the comment matches the intent of a community rule. New users used to hit "invisible barriers like account age and karma thresholds, unclear removals, and poor community discovery." Stronger abuse-prevention is the trade they want mods to accept so those thresholds can go away. They wrote the change "will make it easier for genuine new users to participate and easier for mods to welcome them with confidence."

That is good news if the reply still helps after you delete the product line. It is bad news if your first comment is a launch announcement wearing a helpful costume. You can finally leave a useful first comment without farming karma in throwaway subs. You cannot hide a launch announcement inside a synonym. Keyword lists used to catch the clumsy version of the same pitch. The new reader is built for the version that never uses the banned word.

Rules Hub is live for new subs. Most big rooms have not flipped.

Rules Hub was tested in 700+ communities. Newly created subreddits can turn it on now. Wider rollout is planned later in 2026. Reddit still has 130 million daily active users and 100,000+ communities, so most of the big rooms have not flipped the switch. You are not walking into one new stack. You are walking into a mix of intent scoring and the same brittle keyword lists as last year.

Piece	Status	Detail
Rules Hub	Live for new subs	Tested in 700+ communities, wider rollout later in 2026
Karma and account age	Being dropped	Abuse-prevention is why communities can welcome new posters
Public API	Phasing out	New requests restricted, Devvit plus a $1 million migration program
Old Reddit	Login in the short term	Logged-out traffic called a source of abusive scraping
AutoModerator	Staying for now	Kept until replacements are tested with moderators

Mods choose which rules auto-enforce and whether a trigger sends the comment to a queue, filters it, or removes it. Preview mode exists. Logs exist. The model is supposed to catch the comment that never uses the banned word and still violates the rule. Keyword Automod missed that class of junk for years. A founder-shaped failure mode is obvious. You write a "helpful" paragraph that only works as a pitch. The LLM is built to notice that.

Lower karma floors look like an invitation to spray. They are not.

You should not auto-post now that karma gates are dropping. Rules Hub still reads whether you helped. Marketing while you ship still has to fit next to the product. A firehose of unreviewed comments is how you get banned from the only thread that mattered.

ReplyGuy-style tools optimize for volume in comments. Empty outreach is the failure mode. The useful version finds live threads where category buyers already ask (Reddit, LinkedIn, X, Threads, DEV, anywhere), writes a reply that should still help if you delete the product line, and carries proof so the comment is not a landing page in costume. If you want fully autonomous posting, you are not trying to stay in the room.

Scrapers lose the public API and Old Reddit at the same time.

Reddit is gradually restricting new API requests and pushing developers toward its Devvit Developer Platform, with a $1 million migration program. Old Reddit will require login in the short term. Admin boat-botany called the logged-out experience "a significant source of abusive scraping and automated traffic." CEO Steve Huffman said Reddit may "limit access, migrate important uses, or rebuild it on a modern tech stack."

Logged-out harvest is the same class of traffic that dies when a site pastes a crawler wall in robots.txt. If your loop was harvest then spray, the pipe is closing from both ends. People who show up with a real answer and a real artifact still have a path. A rec is homework. A PR is the fix.

Write the comment that still helps after you delete the product mention.

Answer the question they actually asked, including the ugly constraint they mentioned. Skip the signup CTA. If you have seen the failure mode while shipping, say what broke and how you checked it. Carry proof only if a mention still earns its keep. A blog post, an SEO pull request, or a public mention beats a landing-page dump.

If ChatGPT names competitors for your buying questions, commenting more on Reddit will not fix a robots.txt wall. Often the engines cannot even read your site. Ship pages a crawler can see, then use those pages as artifacts when a buyer thread appears. Viewfy finds those threads and drafts in the field you are already in, as your business. Thread scout leads. Blog posts, SEO PRs, and mentions are artifacts he carries in. Free /check shows whether AI engines can even read your site.

Treat every sub as its own stack. A huge community still runs a brittle keyword list. Tiny new subs can already score intent. Write for the stricter reader.

This rollout does not tell you your home sub already flipped.

Rules Hub is not ready to replace AutoModerator in large or complex communities, and Reddit will keep Automod until replacement workflows are "tested and proven in partnership with moderators." One moderator said Rules Hub "can only enforce 2" of their 8 rules and "cannot enforce our version of remember the human." Some mods threaten to resign if Automod is deprecated. That is one moderator report, not a platform-wide score.

Most of the 100,000+ communities have not turned it on. Wider rollout is later in 2026. Reddit's own post is the source for the 700+ test, the 130 million daily active users, and the $1 million Devvit migration program. None of that measures how often a founder-shaped "helpful" comment actually gets removed once a big sub flips. Write as if the stricter reader is already on. A comment that still helps after you strip the product sentence survives keyword Automod and intent scoring. A costume pitch fails both, just on different timelines.

Have you seen Rules Hub remove a comment that never used the banned word, or is your home sub still a keyword list?
The full breakdown and sources are in the original post.

We scanned 383 Show HN products. 27% were unreadable to crawlers, 17% blocked Claude (and nobody wrote that rule)

Mike Viewfy — Mon, 10 Aug 2026 00:59:40 +0000

Most founders assume that if a page loads in a browser and shows up in Google, an AI assistant can read it too. We scanned 383 products from Show HN with a script we keep in the repo. 27% were unreadable to crawlers, and 17% were blocked from Claude outright.

27% of 383 launches could not be read by a crawler

These are shipped products. Landing pages, docs, pricing, the whole thing. They render fine for a human with a Chrome tab. Fetch them the way a crawler does and better than a quarter of them return something a machine cannot use.

The failure is not exotic. It is client-side rendering with no server response worth parsing, or an interstitial, or a bot filter that decides an unfamiliar user agent is an attack. The founder never sees it because the founder always arrives with a real browser and a real cookie jar.

17% blocked Claude, and almost none of it was a decision

Here is the part that surprised me. Of those 383 products, 17% were blocked from Claude. Not throttled, not rate limited on a bad day. Blocked.

If you ask the founders, most of them will tell you they never made that call. There is no board meeting where a two-person team decides to exclude an AI assistant from reading their marketing site. The rule came from somewhere else.

robots.txt says yes, the edge says no

This is the actual mechanism, and it is worth checking on your own domain today.

Your robots.txt is permissive. You wrote it, or your framework wrote it, and it allows everything. Then a CDN bot management rule, a WAF preset, or a "block AI scrapers" toggle someone flipped during a scare month intercepts the request before your app ever hears about it. The two layers disagree, and the layer that wins is not the one in your repo.

So the file you can read in your editor is not the file that governs behavior. That is why reading robots.txt is not a test. Sending a request with the crawler's user agent is a test.

What you check	What it actually tells you
Page loads in your browser	A logged-in human with JS can see it
robots.txt allows the bot	Your repo intends to allow the bot
Google Search Console coverage	Google specifically got through
Fetch with the crawler's user agent	Whether that crawler got through

The fourth row is the only one that answers the question, and it is the row almost nobody runs.

Analytics cannot show you traffic that never arrived

This is a distribution bug that lives in your infrastructure and is invisible in your dashboards. There is no line in Plausible for "assistant tried to read us and got a 403." The request failed upstream, so the visit never existed, so there is nothing to count and nothing to alert on.

Compare that to a normal outage. If your checkout breaks you get support tickets within an hour. If your site is unreadable to an AI crawler you get silence, indefinitely, and you interpret the silence as low demand.

Our own sitemap left out the three pages we most wanted indexed

I would like to report that we found this problem in other people's repos only. We did not.

Our sitemap omitted three of our highest-intent pages, including the free check page and the proof wall. Both were indexable routes with no noindex, no auth, nothing unusual. They were simply absent from the file whose entire job is telling crawlers what exists. Generated sitemap, route filter, silent omission.

That is a one-line class of bug. It is also the kind of thing that stays broken for months because finding it requires someone to diff the sitemap against the route table, and that task never wins a sprint.

Ranking and being citable are different failure modes

SEO aims at a position in a results page. AI visibility aims at being readable and citable by an assistant. They overlap, but they fail independently.

A page can rank perfectly well on Google and still be blocked from an AI crawler by a bot filter, because Googlebot has been on the allowlist since before the filter existed and the newer agents have not. Your rank tracker will show green the entire time. That is the trap: the instrument you trust is measuring a different pipe.

A 60 item audit list is not a fix

The standard remedy here is a crawler that produces a report. You get a PDF, the PDF goes into Notion, and six weeks later the title tags are unchanged. I have done this. Most people reading this have done this.

The reason is not laziness. It is that a recommendation creates work and a diff removes it. Meta descriptions, title tags, structured data, robots and crawler access, sitemap entries: all of these are edits to files that already live in your repo. A pull request against them takes a minute to review. A list of the same edits takes an afternoon to implement and therefore never gets implemented.

We build a tool that opens those as PRs (Viewfy), which is how the sitemap bug above got caught, but you do not need a tool to run the scan. Curl with a user agent string and a loop over your routes will find most of this.

What this scan cannot tell you

Several honest limits.

It is a single snapshot. A bot filter that returned 403 during our scan window might pass on retry, and rate limiting can look identical to a block if you only ask once.

It does not prove intent. We say most of the Claude blocks were not deliberate because founders say so and because the robots.txt files contradict the edge behavior, not because we surveyed all 383 teams.

It does not measure lost revenue. Being readable is a precondition for being cited, not a cause of it. A crawlable page with nothing worth quoting stays uncited.

And Show HN is a skewed sample: early launches, heavy on JS frameworks, heavy on free CDN tiers with aggressive defaults. A cohort of five year old SaaS companies would likely fail differently, probably less on rendering and more on stale structured data.

What the numbers do support is narrow and checkable: for 383 real shipped products, 27% could not be read by a crawler and 17% were closed to Claude, and you can re-run that test against your own domain in about ten minutes.

What does your site return when you fetch it with an AI crawler's user agent, and did you write that rule or did your CDN?

Full method and the scan script live on our blog at viewfy.ai.

I probed 156 Show HN launches: 23 block Claude, 0 block OpenAI (almost none wrote that file)

Mike Viewfy — Fri, 07 Aug 2026 22:30:52 +0000

Most founders assume the rules in their robots.txt are rules they chose. Across 156 Show HN launches from August 2026, 23 sites (14.7%) disallow Anthropic's ClaudeBot while leaving OpenAI's OAI-SearchBot allowed, and 0 sites (0.0%) do the reverse. In a cohort of 156 independent founders shipping independent products, a coin-flip preference does not come out 23 to zero.

23 to 0 is not a preference, it is a paste

The perfect one-sidedness is the tell. Something is writing the same decision into everyone's repo.

That something is Cloudflare's managed robots.txt. Of the 24 sites in the cohort that disallow any AI crawler, 23 (95.8%) serve that managed file, identifiable by its Content-Signal preamble. The file declares search=yes in the same breath that it disallows ClaudeBot, GPTBot and Google-Extended. The site is telling AI search engines it wants to be found and telling three named crawlers to go away.

The written blocks are uniform in a way hand-editing never produces. ClaudeBot, GPTBot and Google-Extended are each disallowed on the same 24 sites (15.4%). OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot and PerplexityBot are each disallowed on 1 site (0.6%). Exactly one site in 156 runs a blanket disallow, and it is the only block in the whole dataset that reads like a deliberate choice.

Cloudflare wrote the block, not the founder

Cloudflare authored the blocks on 33 of the 156 sites (21.2%). That is 50.8% of the 59 Cloudflare-fronted launches, and Cloudflare fronts 37.8% of the cohort overall.

The cost is concrete. atlasmotion.com launched motors for drones and robotics with 406 words of readable homepage HTML and one point on Hacker News. That page is unreachable to ChatGPT, Claude, Google AI Overviews and Perplexity at once. zipbox.ai shipped 2,141 words about Firecracker VMs for agents and has the same four-engine verdict.

One founder in r/SEO put it plainly: "Then I ran a curl on /robots.txt and saw this block that I definitely didn't write".

Half of the blocks are theater

On 12 of the 23 managed-file sites, the origin still hands ClaudeBot a 200. The block exists only on paper, which is worse than either alternative: crawlers that respect robots.txt stay away from a page the server was happy to serve. You get the traffic loss of a block with none of the protection, and nothing in your logs looks broken.

Twelve rows you can read line by line

Rows where robots.txt names GPTBot, ClaudeBot and Google-Extended but the 403 column is empty are paper-only blocks. Rows where the 403 column is long and the robots column is empty are silent CDN blocks. Rows with both are invisible to all four engines.

host	server	CF managed robots.txt	robots.txt disallows	HTTP 403 to	engines unreachable	words in raw HTML
atlasmotion.com	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot	ChatGPT, Claude, Google AI Overviews, Perplexity	406
zipbox.ai	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot	ChatGPT, Claude, Google AI Overviews, Perplexity	2141
portfoliovideo.com	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot	ChatGPT, Claude, Google AI Overviews, Perplexity	2554
fosbury.ai	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	none	Claude, Google AI Overviews	46
cochat.ai	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	none	Claude, Google AI Overviews	1571
imagetovideoai.tools	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	none	Claude, Google AI Overviews	1215
scalequest.io	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	none	Claude, Google AI Overviews	5
usecharming.com	cloudflare	yes	none	none	Claude, Google AI Overviews, Perplexity	1241
spacescience.tech	cloudflare	yes	none	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot	ChatGPT, Claude, Perplexity	5
today.spqrk.net	Apache	no	none	ClaudeBot	Claude	1163
reelang.com	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Google-Extended	Claude, Google AI Overviews	3
modelplane.dev	cloudflare	yes	GPTBot, ClaudeBot, Google-Extended	OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot	ChatGPT, Claude, Google AI Overviews, Perplexity	393

reelang.com is the row to stare at: three readable words in the un-executed HTML, a managed robots.txt disallowing three crawlers, and a server refusing seven user-agents including Google-Extended. A language-learning product with speech-analyzed native speaker video, and an assistant asked about it has nothing to read.

The ChatGPT block is silent; the Google block is written down

The two layers block opposite engines. robots.txt in this cohort forbids ClaudeBot, GPTBot and Google-Extended. The CDN and WAF layer refuses OAI-SearchBot, ChatGPT-User, Claude-User, Claude-SearchBot and PerplexityBot at the network layer, where nothing is declared.

15 sites (9.6%) returned a non-200 to at least one AI crawler. OAI-SearchBot and PerplexityBot were refused on 14 each (9.0%). Google-Extended was refused on only 2 (1.3%), and on 13 of those 15 sites (86.7%) Google-Extended got a 200 while every other AI crawler got a 403.

Stack the layers and honesty goes lopsided:

Google AI Overviews: unreachable on 25 sites (16.0%), only 4.0% of those blocks silent.
ChatGPT: unreachable on 15 sites (9.6%), 93.3% silent.
Perplexity: unreachable on 15 sites (9.6%), 93.3% silent.
Claude: worst off at 28 sites (17.9%), at least 14.3% silent, the rest declared.

A commenter in r/SEO described the intent behind the split: "Cloudflare blocks the training bots, not the ones that handle search/appearance on the platform. For example, Cloudflare will block GPTBot and allow OAI-SearchBot". The measurement says the bot-fight rules underneath do not honor that split.

Your host decides your visibility more than your craft does

Cloudflare-fronted launches are invisible at 47.5% of 59 sites. Vercel-fronted launches are invisible at 2.6% of 39. That is a ratio of 18.5, and Cloudflare accounts for 82.4% of every invisible site while making up 37.8% of the cohort. nginx sat at 10.0% invisible across 20 sites.

Each stack fails differently. Cloudflare's failure is the block: 8.5% JavaScript-only, only 6.8% missing robots.txt. Vercel's failure is absence: 28.2% serve no robots.txt at all, nginx 35.0%. The 10 launches on shared app-platform subdomains were 20.0% invisible, 20.0% JavaScript-only shells, and 80.0% with no robots.txt.

Cohort-wide: 21.8% of the 156 are invisible to at least one engine and 71.8% are fully clean. 15.4% blocked in robots.txt, 9.6% refused at the network layer, 5.8% ship an empty JavaScript shell, 19.2% serve no robots.txt, 9.0% are thin HTML. Median readable words in raw HTML: 687. Sitemap declared: 67.3%.

Four checks you can run before your coffee cools

curl https://yoursite.com/robots.txt and look for a Content-Signal preamble. 21.2% of the cohort has one; 50.8% of Cloudflare-fronted sites do. If it disallows ClaudeBot, GPTBot and Google-Extended while declaring search=yes, you did not write that.
curl -A "OAI-SearchBot" -I https://yoursite.com, then repeat for ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot and Google-Extended. 9.6% of the cohort returns a non-200 to at least one, and 93.3% of the ChatGPT blocks appear nowhere in robots.txt.
curl -s https://yoursite.com | wc -w on the un-executed HTML. 5.8% of launches ship a shell that is empty until JavaScript runs. Median was 687 words. scalequest.io shipped five, reelang.com three.
Confirm robots.txt and sitemap.xml exist at all. 19.2% serve no robots.txt, only 67.3% declare a sitemap. On Vercel that miss rate was 28.2%, on nginx 35.0%.

What this data cannot say

Homepage only, one request per user-agent, one two-day slice of Show HN, so a rate limit or transient 403 can read as a permanent block. We made no attempt to verify that a user-agent string belonged to the vendor it claims. We did not test Cloudflare defaults, only outcomes.

This study also counts ClaudeBot among retrieval crawlers and GPTBot as training-only, and blocking ClaudeBot does not remove a site from Claude entirely: the 23 sites disallowing ClaudeBot still allow Claude-User and Claude-SearchBot, so exposure is reduced, not zero. Claude still came out worst overall because the network layer refused Claude-User and Claude-SearchBot on 14 sites each, separately from anything robots.txt said.

Method, briefly: Hacker News Algolia search_by_date with tags=show_hn, newest first, no upvote filter, deduped by host, code hosts and app stores and socials and doc hosts dropped. 274 posts scanned down to 170 hosts, 166 reachable, 156 on their own domain and 10 on shared app-platform subdomains. Nine requests per homepage, no crawl beyond it. Spot checks reproduced by hand with curl -A on atlasmotion.com and zipbox.ai.

Run check 1 on your own domain and tell me what came back: did you write that file, or did your CDN?
Full dataset, per-host rows and the recompute script live on the Viewfy blog.

Correction, added after publication

A founder in the cohort pushed back on the network layer numbers, and he was right. Our probes sent crawler user agents from an ordinary IP, so a 403 in that setup measures how a CDN treats an impersonator, not the real crawler. Verified bots calling from their published IP ranges typically pass. The robots.txt findings in this post are unaffected, that file is public text anyone can read. But treat every "unreachable" and WAF number above as "unmeasurable from the outside", not "blocked". We have changed the methodology so outbound claims only use the robots.txt layer.