DEV Community: brainbootdev

OpenAI sends three bots. The one that precedes a citation is the smallest.

brainbootdev — Tue, 01 Sep 2026 08:17:14 +0000

If you go looking for AI traffic in your own server logs, you probably grep for GPTBot and stop there. That is the wrong number, and it is wrong in a way that flatters you.

OpenAI runs three separate agents. They mean three different things.

GPTBot is the training crawler. It feeds model weights. It will never send you a visitor.
OAI-SearchBot is the retrieval crawl. This is the fetch that runs before an answer cites a page. If you care about being cited, this is the one to watch.
ChatGPT-User is not a crawler at all. It fires when a real person clicks a citation or asks ChatGPT to open a URL. That is a human who already arrived.

Collapse them into a single "OpenAI" row and you are adding a trainer, a retriever and a reader together, then calling the total your AI traffic.

What it looks like across 14 sites

Seven days to 31 August 2026, one Cloudflare account, 14 properties:

Agent	Purpose	Fetches
GPTBot	training	11,391
ChatGPT-User	a human who clicked	9,656
OAI-SearchBot	retrieval, precedes a citation	2,930

Retrieval is 12.2% of OpenAI's footprint on the network. Training outweighs retrieval 3.89 to 1.

The bar that matters is the short one. Everything upstream of a citation is happening in that 12.2%, and most dashboards never separate it out.

This is not an OpenAI quirk

The same split shows up at every vendor that runs a separate user agent for human-initiated fetches. Same network, same window:

Vendor	Crawler	Human
OpenAI	GPTBot 11,391	ChatGPT-User 9,656
Anthropic	ClaudeBot 6,881	Claude-User 1,191
Perplexity	PerplexityBot 1,749	Perplexity-User 204

Three vendors, three crawler-versus-human pairs, and almost nobody separates those either.

How to run it on your own logs

If you are on Cloudflare, the data is already there. Query httpRequestsAdaptiveGroups in the GraphQL analytics API, group by clientRequestHTTPHost and the user agent, and split the buckets by exact agent string rather than by vendor. The whole finding lives in that one grouping decision.

If you are on nginx or Apache, the same split is a grep away. The user agent strings are distinct and stable.

The caveat that matters

These are fetches, not citations. A fetch is two rungs below a citation on the ladder and most fetches never become one. They come from a different instrument than citation counts do, they measure a different event, and the two must never be added together.

Bot identity here is self-reported in the user agent. We are not verifying by IP range, so treat these as what the requester claimed to be.

One more honest bound: this is one network of 14 sites over seven days. The ratio will differ on yours. The point is not our number, it is that "OpenAI" was never one number to begin with.

1,001 pages carry 737,197 citations. Fifty of them carry half.

brainbootdev — Mon, 24 Aug 2026 20:23:46 +0000

Most numbers about AI visibility are network totals, and a network total hides the only thing worth knowing. So we counted ours the other way. Every single page on this fourteen-site network that Bing recorded as cited by Copilot over the ninety days to 22 August, one at a time, each with its own count.

There are 1,001 of them, and together they carry 737,197 citations.

The boring part first, because it belongs in the body

This is the page-level figure. Bing's property-level endpoint reports 768,234 for the same window, a 4.0% gap that is negative on all fourteen properties. Both are correct and they measure different things: pages under Bing's reporting threshold land in the property total but never appear as rows. Page level is the floor, and the floor is the number I am using.

This is Copilot only, because Bing is the one engine that publishes per-property citation counts at all. Citations are not clicks. And two of the fourteen properties recorded zero citations for nine and sixty-one consecutive days respectively, and are sitting inside that total, which makes the ninety-day window a reservoir rather than a rate.

The density is 736 citations per cited page

That average is nearly useless, and watching it fall apart is the actual finding.

The top page took 21,685 citations. One address, edubracket.com/articles/best-free-online-courses-2026, cited roughly 241 times a day, every day, for three months. The median cited page took 95. The top page is 228 times the median, inside one network running one architecture.

slice	citations	share of network
top 1 page	21,685	2.9%
top 10 pages	138,441	18.8%
top 50 pages	369,643	50.1%
top 100 pages	498,638	67.6%
top 200 pages	627,315	85.1%
all 1,001	737,197	100%

Half of everything this network is cited for lives on fifty addresses.

What the top pages have in common

It is structural rather than magical. They are answer-shaped: roundups and tools pointed at questions people are actually asking, in verticals where somebody is asking constantly. Best free online courses. A formulary checker. Small business grants. Prescription discount cards. They are not our cleverest pages and several of them are not our longest. They are the ones that sit exactly where a question lands.

I am not going to tell you what that is worth as a percentage lift, because I do not have a before-and-after instrument and anybody quoting you one does not either. What I can tell you is what was measured, on what, over what window, with which endpoint.

The open question

If half a network's citations live on fifty addresses, then buying a site is a strange way to buy anything. The page is the asset. The site is just where it happens to live.

A note for anyone who wants to be inside one of those pages

We know which fifty. We have the per-URL table, refreshed weekly, with a trajectory on every row.

Our sites name real products in real roundups, and some of those slots are commercial. If you would like to talk about your product being one of the things named on a page that is already being cited at this density, we are easy to reach, and we will tell you exactly what a given page is doing before you decide anything.

deepsynthesis.org/lattice

One engine lets us count AI citations. It sends 15% of the readers.

brainbootdev — Sun, 23 Aug 2026 11:31:28 +0000

Every AI visibility tool on the market prices itself on data from Bing. Not because Bing is where the readers are. Because Bing is the only engine that will tell you it cited you.

The instrument

Bing Webmaster Tools publishes per-property AI citation counts. Nobody else does. Across fourteen live properties we run, summed from daily rows over the ninety days to 22 August 2026, that number is 768,234. Restricted to the eleven properties with a full window, the like-for-like figure is 735,780.

Before going further, the number argues with itself. Bing's own citationstats endpoint reports 768,234 for that window. Its per-URL pages/stats endpoint reports 737,197 for the same window, in the same session. The gap is 31,037, about four percent, and it runs in the same direction on all fourteen properties.

We publish the higher figure because it is what the tool's own chart shows a site owner, and we say the lower one exists rather than burying it. If a vendor quotes you a citation count without telling you which endpoint it came from, that is a four percent question they have not answered.

Then we went looking for the readers

Over twenty-eight days, GA4 recorded 2,605 sessions on the network that arrived from an AI assistant.

engine	sessions	share
ChatGPT	2,053	78.8%
Copilot	387	14.9%
Claude	123	4.7%
Perplexity	42	1.6%
Gemini, DeepSeek, Grok	no matching source string	unmeasurable

Copilot is the Bing surface. So the engine that publishes a citation count for us is the engine responsible for roughly one seventh of the AI readers who actually arrive. The engine responsible for nearly four fifths of them publishes nothing at all about who it cites.

The inversion

That is the whole finding, and it inverts what the category is sold on. Every dashboard, every AI-visibility score, every pitch deck with a citation chart in it is describing the small channel with confidence and the large one not at all.

We built our own measurement on the same foundation. This is a finding about our instrument before it is a finding about anyone else's.

The part that is genuinely unknown

Gemini, DeepSeek and Grok matched no source string in our data. Zero rows. The correct word for that is unmeasurable, not absent.

Gemini referrals in particular commonly arrive with no distinguishing referrer and land in organic or direct, which means a real reader gets filed as something else. Reporting those engines as zero would be the easy lie, and it is the specific lie this category is built to tell, because a zero looks like a measurement and an admission of blindness does not.

A hypothesis, offered as a hypothesis

Cloudflare per-bot logs show GPTBot, which crawls for training, at about 7,477 requests a day against OAI-SearchBot, which fetches for retrieval, at about 500. Roughly fifteen to one. Yet ChatGPT delivers 78.8% of the AI arrivals.

If the dominant channel is answering largely out of what it already absorbed rather than what it fetches live, then surviving in the training corpus is the lever on most of the traffic, and retrieval optimisation is the lever on the remainder. We are not claiming that is settled. We are saying the crawl ratio and the referral share point the same way and almost nobody is pricing on it.

The question

If the biggest channel is unmeasurable by construction, what is anyone actually selling: the placement, or the measurement of the one place that happens to report?

Figures: Bing Webmaster AI performance, fourteen properties, 23 May to 22 August 2026, summed from daily rows. GA4 Data API by session source, twenty-eight days to 20 August 2026. GA4 is JavaScript based, so ad blockers and privacy browsers are missed and every session figure here is a floor, not a ceiling.

What happens when you ask 8 AI models the same buying question every month

brainbootdev — Sat, 11 Jul 2026 23:54:13 +0000

A while back I got annoyed at a specific genre of blog post: "we asked ChatGPT what the best CRM is and here's the answer." One screenshot, one run, treated as if the model holds a stable opinion. It doesn't. So I built a small harness to measure that instead of hand-waving about it.

The setup is boring on purpose. Eight models. Sixteen B2B software categories (CRM, project management, email marketing, that kind of thing). For each category I ask every model the same plain question: what is the single best tool here. One pick, no hedging allowed. I log the raw response, the parsed pick, the model, the timestamp, and the exact prompt into a JSONL file. Then I do it again next month.

Two things fell out that I did not fully expect.

First, across all sixteen categories the eight models never once agreed on the same tool. Not close-but-different. Zero unanimous picks out of sixteen. I assumed there would be at least a couple of categories where everyone converged on the obvious incumbent. Nope.

Second, and this is the one I keep chewing on: the models do not even agree with themselves. Ask the same model the same question in a fresh session and it swaps its own top pick around 74% of the time. Same model, same prompt, nothing changed but the session. Roughly three-in-four odds it contradicts what it told you yesterday.

Here is the part I want other people to break. I do not know how much of that self-disagreement is temperature and how much is genuine ranking instability. My instinct is that dropping temperature to 0 will not actually fix it, it will just hide the wobble behind a deterministic-looking facade while the underlying ranking stays mush. But I have not run that ablation cleanly yet, and I would rather someone who does eval work for a living tell me I am wrong.

The reason I am posting the method and not only the numbers: a "models disagree" claim is worthless if you cannot reproduce it. So everything is open. The per-run JSONL, the prompt text, the model list, all of it, DOI'd under CC-BY so you can cite it or tear it apart: https://data.deepsynthesis.org/. I re-run the whole thing monthly, which means you can watch a specific category drift over time rather than trusting a frozen screenshot from whenever the author happened to hit send.

A few implementation notes if you want to build your own version:

Force a single pick. If you let the model give you a ranked list it will hedge and you lose the signal. One tool, no runner-ups, is the whole trick.
Parse defensively. Models love to answer the question and then immediately undercut it with "but it depends on your needs." Strip that.
Store the raw response next to the parsed pick. You will second-guess your parser and you will want the original text.
Timestamp everything and never overwrite a run. The monthly diff is the actual product.

The uncomfortable takeaway for anyone doing AI-search or GEO work: if a model's pick is this unstable, "we rank first when you ask ChatGPT" is a coin that lands differently every flip. Optimizing for a single snapshot is optimizing for noise. Optimizing to show up across many runs and many models is the only thing that survives.

If you poke at the data and find a category where the parser is wrong, or a model I should add, tell me. That is the point of putting it out in the open.

Why I stopped writing prompts and started compiling them

brainbootdev — Thu, 16 Apr 2026 23:49:11 +0000

I've been building AI-powered tools for a while and I kept hitting the same wall. Not the models — the models are fine. The problem is how we use them.

Every interaction with an AI starts from zero. You open a new conversation, paste your system prompt, explain your context, get partway through something useful, watch the model drift off course, hit the context limit, and then start over. I tracked my own usage for a month and roughly 40% of my tokens were spent re-establishing context that
the model already knew five minutes ago.

That's not a model problem. That's an architecture problem. We're treating prompts like disposable messages when they should be treated like software.

What "prompts as software" actually means

Think about what makes code reliable. Types enforce structure. Tests verify behavior. Modules compose into larger systems. Error handling catches failures before they propagate.

Now think about what prompts have. None of that. A prompt is a string you paste into a text box and hope works. There's no type checking on what comes back. There's no test suite. There's no way to compose two prompts together with a contract between them. If the output is garbage you just try again and burn more tokens.

I built Brainboot to close that gap. The core primitive is a "brain" — a prompt wrapped in actual engineering.

The anatomy of a brain

A brain declares three things beyond the prompt itself.

Typed inputs and outputs. The brain specifies what shape of data it accepts and what shape it returns. The runtime validates both directions. If you ask for JSON and the model returns markdown, the runtime catches it and retries automatically. You never burn tokens processing malformed output downstream.

Invariants. These are rules enforced on every execution at the wrapper layer. Not instructions in the system prompt that the model might follow. Actual guardrails that the runtime checks against the output before it's returned. Things like "never include placeholder text" or "all URLs must be valid" or "output must be valid TypeScript." The model cannot drift past these.

Test suites. Every brain has tests that run across multiple models — currently GPT-5, Claude Opus, and Gemini Ultra. Pass rates are public. You know whether a brain works reliably before you use it in production. You don't have to re-test it every time.

Composition is where it gets interesting

Small brains chain into pipelines. A research brain feeds an outline brain feeds a drafting brain feeds a quality gate. Each brain in the chain gets exactly the context it needs — not your entire conversation history. The output of brain A is type-checked before it becomes the input of brain B.

This solves the token problem at a fundamental level. A monolithic conversation carrying 50K tokens of history to maintain context gets replaced by a chain of focused brains each using maybe 2-5K tokens of precisely scoped context. Same outcome, fraction of the token cost, and no drift because invariants catch it at every step.

It also makes failures cheap. If brain 4 in a 6-brain pipeline fails, you retry brain 4 with its specific input. You don't re-run brains 1 through 3. In a monolithic conversation, any failure means starting the whole thing over.

The compiler

The most interesting feature is the compiler. You describe what you want in plain English — something like "weekly content pipeline for B2B SaaS" — and it runs a four-stage process.

Stage 1, Decompose — breaks your intent into atomic capabilities. "This needs keyword research, topic clustering, outline generation, drafting, SEO optimization, and quality review."

Stage 2, Map — searches the existing brain catalog for brains that match each capability. "Keyword Research Brain covers capability 1. Cluster Architecture Brain covers capability 2."

Stage 3, Synthesize — wires the matched brains into a composition graph with typed connections between them. The output schema of each brain is validated against the input schema of the next.

Stage 4, Audit — runs the complete composition through a quality check and grades it A through F with a deploy recommendation.

The output isn't a prompt. It's a deployable system. Describe once, compile once, run forever.

What this looks like in production

I have two circuits running in production right now.

Content Empire runs 6 brains in a pipeline — Keyword Research, Cluster Architecture, Outline, Draft, On-Page SEO, Quality Gate. It produces 150 SEO-optimized pages per quarter, 13 weekly newsletters, 12 authority essays, 3 full SEO audits, and 500+ platform adaptations. It runs on a schedule via Vercel Cron with zero manual intervention. $299/month versus roughly $25K/month for the equivalent human team.

Sales Engine runs 5 blueprints handling prospecting, research, personalized outreach, reply handling, and pipeline intelligence. 500 qualified prospects per month with individualized research and multi-step sequences. $399/month versus roughly $6,600/month for an SDR.

To validate the architecture on something with objectively verifiable results I also pointed a circuit at MLB predictions — a 6-layer probability engine predicting scoreless innings. It's been running autonomously since opening day. 150+ verified picks at 89% accuracy. Completely unrelated to the core business but it proved that composed brains with invariants work on real-world problems with measurable outcomes. Full public track record at
brainboot.dev/labs/nrfi/results.

The stack

Next.js 16 with App Router and React 19. Supabase for auth and Postgres. Vercel for deployment and cron jobs. Stripe for billing. Resend for transactional email. Vercel AI SDK v6 with AI Gateway for multi-model routing.

The brain runtime handles type validation, invariant enforcement, retry logic, and telemetry. Every execution produces a cognitive trace showing which models, rules, and brains contributed to which decisions.

Total infrastructure cost is roughly $50/month. Solo built, no funding.

Try it

The platform is live at https://brainboot.dev. Free tier has 200+ curated prompts with no account required. The compiler, brains, and circuits are where it gets interesting.

The manifesto explaining the full "prompts are software" philosophy is at https://brainboot.dev/manifesto.

I'm interested in how other people are approaching the reliability and token efficiency problem. If you've built
something similar or taken a different approach I'd genuinely like to hear about it.