DEV Community: uehara

Which AI Agent Tool Does What: Claude Code, ChatGPT, OpenClaw and Paperclip, Sorted by What Starts the Work

uehara — Thu, 15 Oct 2026 02:39:19 +0000

In the previous article I sorted 2,716 posts about "AI employees" and "AI companies" into five stages: role prompt, instruction files, scheduled job, always-on single agent, and multi-agent. At each stage the human's job changes, from asking every time to only approving.

The next question I got was practical: "Then which app should I use, and what can it actually do?" This article answers that for Claude Code, ChatGPT (including dots), other always-on agents, Paperclip, OpenClaw, and running any of them on a Mac mini. For each one I separate what it can do, what it cannot do, who it suits, and who it does not suit.

Product names are a poor guide here. The same product lands in a different stage depending on which feature you use. Claude Code with only a CLAUDE.md file is an instruction-file setup. With Routines it becomes a scheduled job. With Channels it becomes an always-on agent you can message. So this article looks things up in the order what you want to happen → the feature that does it → the product.

Facts here are as of October 14, 2026. Several of them changed more than once this year, so I give dates where it matters.

This article stays close to the tools themselves. It is for readers who want the facts laid out and will decide for themselves which tool fits. The "Which to pick, by what you want" section near the end is also a choice between tools. It does not yet speak to readers who think, "I want to automate my own work, but I don't know what I need." That is the next article. There I will start from everyday work: sorting, summarizing and answering email automatically, automating the bookkeeping, checking Instagram on a schedule, replying to LINE messages automatically. For each kind of work I will explain which apps and which machines you need, in terms that do not assume technical knowledge. Whether buying a Mac mini pays off in your case goes there too. The Mac mini section in this article gives only my own judgment and the general conditions.

The short version

Sort by two things, not by product name: what starts the work, and where it runs. "What starts the work" is the stage from the previous article (a person asks / a schedule / a message / another agent). "Where it runs" decides whether it keeps working when you close your laptop.
Claude Code covers four stages depending on the feature. Instruction files on your own machine, Routines in the cloud (schedule as short as once an hour), Channels and Remote Control for messaging a running session, and Projects for parallel threads.
ChatGPT has two different things. Scheduled tasks, Work and Codex are scheduled jobs. dots is an always-on single agent that runs in OpenAI's cloud and comes back to you on its own. dots needs Pro, Business Premium or Enterprise.
OpenClaw is free software you run on your own machine and talk to from chat apps. I installed it in a test container. With the default settings, it ran a shell command I asked for without asking for approval. After switching to a stricter preset, it showed an approval card instead. A job set to every minute ran three times in a row.
Paperclip does not think by itself. It ties Claude Code, Codex, OpenClaw and others together as "workers" with an org chart, tasks, budgets and approvals. That is the multi-agent stage. Its budget does not stop subscription overuse, because subscription runs are recorded at 0 cents.
A Mac mini is a place, not a tool. It only adds something when you need things that exist only on a Mac (iMessage, macOS apps) or a machine at home (a home IP address), or when you run a local LLM (a large language model on your own hardware). If you use Claude Code or Codex with cloud models, my judgment is that an always-on Linux CPU server is better value.

Body (about 15 minutes)

Two axes: what starts the work, and where it runs

Here are the five stages from the previous article again, with what starts the work at each stage.

Stage	What starts the work	What the human does (most common)
Role prompt	A person asks each time	Asks each time (87.1%)
Instruction files	A person starts it; work is split inside	Starts it (68.8%)
Scheduled job	A time, or an event such as new mail	Reads the result (84.1%)
Always-on single agent	You message it, or it resumes on its own	Messages it (61.6%)
Multi-agent	One worker hands work to another	Only approves (54.3%)

The percentages are the most common human role within each type in the previous article's 2,716 posts.

The second axis is where it runs. There are three places: the provider's cloud, your own machine (a laptop, a Mac mini, a rented server), and your own browser through an extension. This decides two things: whether it keeps working when your laptop is closed, and whether it can reach your local files and logged-in sessions.

The chart below places each product's features on the five stages. The point is that one product spans several stages.

And this one shows where each feature runs, and the two lines that matter: whether it needs your computer to be on, and whether it can reach your local logins.

Comparison table

Terms in the table: a heartbeat is a timer that wakes the agent to check whether there is anything to do. cron runs jobs at set times or intervals. A webhook is a trigger fired when another service calls in.

Tool (feature)	Stage	Where it runs	What starts it	Approval	Cost	Works with laptop closed
Claude Code (CLI / Desktop)	Instruction files to always-on	Your machine	You ask; /loop and Desktop scheduled tasks (from 1 minute); Channels; Remote Control	Six permission modes; interactive default is auto	Subscription limits (Pro $20/month and up)	No
Claude Code Routines / Projects	Scheduled job / multi-agent	Cloud	Schedule (from 1 hour), API, GitHub events	Routines mostly run without approval (publishing a new page asks first); Projects asks inside the thread	Same subscription	Yes
Claude (formerly Cowork)	Scheduled job	Cloud (all new Pro/Max tasks since 2026-10-06)	You ask; schedules such as hourly, daily, weekly (shorter than hourly not confirmed)	Manual / Auto in the new experience	Same subscription	Scheduled tasks: yes
ChatGPT dots	Always-on single agent	Cloud, plus one optional local PC	Conversation, resumes on its own, schedules made in chat, app events	An automatic reviewer decides proceed / ask / leave to you	Pro, Business Premium, Enterprise	Yes
ChatGPT scheduled tasks / Work / Codex	Scheduled job	Cloud (web), local (desktop app)	Time; Gmail, Slack, GitHub events (web)	Confirmation before important actions	Plus $20 and up	Web: yes
OpenClaw	Always-on single agent (multi-agent possible)	Your machine	33 chat apps, heartbeat (30 min default), cron, triggers, webhooks	Default runs commands without approval	Free (MIT) + model + machine	Yes, on a server
Paperclip	Multi-agent	Your machine (server + database)	heartbeat, routines (cron / webhook / API), task assignment	Hiring and strategy approvals built in; business approvals you design	Free + the "brain" + machine	Yes, while the machine is up

What each tool can and cannot do

Claude Code on your own machine

Can: Work with local files and commands. Keep procedures in files (CLAUDE.md, skills, subagents) and force checks to run with hooks. Through Channels you can message a running local session from Telegram, Discord or iMessage. Remote Control lets you continue a local session from your phone.
Cannot: Keep running when the computer sleeps. A /loop disappears after 7 days. Chrome control stops at login screens and CAPTCHAs. Computer use does not work in unattended -p mode.
Suits: Developers who want the AI to work inside their own environment and repositories.
Does not suit: People who cannot keep a computer running, or who do not want to touch a terminal.

This is how I use Claude Code every day: instruction files and skills, with hooks that block external sends. In the previous article's terms that is the instruction-file stage.

Claude Code in the cloud: Routines and Projects

Can: Run on a schedule with your laptop closed (once an hour at the shortest), or start from an API call or a GitHub pull request. Projects lets you send several pieces of work to parallel threads and leave them.
Cannot: Run more often than hourly, or log in to websites. Routines mostly do not ask for approval mid-run (publishing a new claude.ai page is an exception that asks first). Projects asks inside the thread when it needs approval, and you answer there. A green mark in the run list does not prove the work succeeded. Projects is a public beta for Pro and Max; it is not available on Team or Enterprise yet, or from the CLI.
Suits: Developers whose work already lives on GitHub.
Does not suit: Work where a person must check before anything is sent, or web work that needs a login.

Claude (formerly Cowork), Claude in Chrome, Claude Tag

Anthropic announced on September 16, 2026 that Cowork is being merged into Claude; it is rolling out to Pro and Max first, with Team and Free later. From October 6, all new tasks on Pro and Max run in the cloud.

Can: Run documents, research and file tidying on a schedule without writing code, even while your PC sleeps. The Chrome extension handles routine web work that needs your login. Claude Tag shares an @claude in Slack with the team (Team and Enterprise only).
Cannot: The Chrome extension never buys things, creates accounts or permanently deletes in any mode. Claude Tag is not available on Pro or Max. I could not confirm a schedule shorter than hourly.
Suits: Business owners and freelancers who do not write code and want routine work done.
Does not suit: People who want local files handled unattended, or unattended browser work at night.

ChatGPT dots

I tested dots with my real work accounts in the first article of this series. dots runs on a dedicated computer and browser in OpenAI's cloud, and you can connect one local PC.

Can: Keep working with your PC closed, and come back to you when there is a result or a decision. Talk with the same memory from Slack, Teams or a phone call (you call it). Keep logins saved in its cloud browser.
Cannot: It is not on Plus, and not on Pro in the EEA, the UK or Switzerland. It cannot call or text you yet. The official docs do not state a shortest interval or a run limit. Actions through your local PC look like your own actions to the other side.
Suits: People on Pro or Business Premium who want a secretary for mail, calendar and Slack, including non-developers.
Does not suit: People on Plus, work that must run at an exact time, and anyone who needs AI actions recorded separately from their own.

One point I will not state as settled. OpenAI's docs say existing app permissions also apply to dots. In my test, dots deleted a calendar event without asking once. That is one case against the docs, so treat it as unresolved.

ChatGPT scheduled tasks, Work and Codex

Can: Run a saved instruction at a set time or when mail arrives, and finish documents, tables and regular reports. The desktop app runs unattended on local repositories. Codex Cloud goes as far as opening a pull request.
Cannot: Slack DMs and reactions cannot start a task. The official docs do not describe retries or failure notices. The Codex CLI itself has no scheduler; you call codex exec from launchd or cron.
Suits: Plus users who want regular reports, and developers who already use Codex.
Does not suit: A standalone task that must run more often than hourly. News reports say standalone scheduled tasks run at most once an hour, while OpenAI's learning pages say recurring tasks inside a chat can run every few minutes. I could not open the Help Center page to settle this.

GPT-5.5 was retired from ChatGPT, Work and Codex on October 14, 2026, so older examples that name it are out of date.

OpenClaw (I installed it to check)

OpenClaw is free, open-source software (MIT license) that you install on your own computer or server. You talk to it from chat apps you already use, such as Telegram, Slack, WhatsApp, iMessage or LINE (33 channels in the official list), and it does the work on that machine. It has no model of its own; you connect Claude, GPT or another model. It was called Clawdbot and then Moltbot before January 30, 2026, and is now run by the OpenClaw Foundation.

I installed version 2026.10.3 in a Docker container on my Mac. I connected no business accounts and no chat apps, only the built-in web screen (the Control UI). For the model I signed in with my ChatGPT subscription through OpenAI's device-code login, so no extra fees.

A chat round trip worked. I asked what it can do and what it needs my approval for. It answered that it can edit files, run commands and browse, and that it asks before destructive or external actions.

With the default settings, it ran a shell command without asking. I asked it to run uname -a && whoami && date -u, and it ran and showed the output. The command-approval summary for the default setup says "no approval prompts."

With the "cautious" preset, it showed an approval card instead. I switched to exec-policy preset cautious (allowlist, ask on miss, deny on fallback). When I asked it to save a note to memory, it wanted to read a file and showed an approval card with "Allow once," "Always allow" and "Deny." One more thing happened along the way: after the switch, commands tried to run inside an isolated environment (bwrap), which failed to start, and the request to run outside it was refused, so even a plain ls did not run. Because the model was connected through Codex on my ChatGPT subscription, the approval card was titled "Codex app-server command approval." I did not separate whether this isolation and approval came from OpenClaw's settings or from the connected Codex.

A job set to every minute ran three times in a row. I added cron add --every 1m with "reply with the current UTC time." It ran at 09:08, 09:09 and 09:10 UTC, and each time the model answered with the right time. Every run was marked as an error only because I had connected no chat app to deliver the result to. After the third run, the next run was scheduled five minutes later (09:15 UTC); I did not check why. On the same screen you can see what a fresh install already has scheduled: a heartbeat every 30 minutes, a daily memory job at 3:00 UTC ("Memory Dreaming Promotion"), and a weekly skill review.

Memory was written to a Markdown file in its workspace. After switching to the no-approval preset (exec-policy preset yolo), I asked it to remember that my test project is called AICO-003 and that I prefer short answers. OpenClaw added those two lines to a file called USER.md and told me which file it wrote. It is plain text, so a person can open it, read it and fix it.

Can: Live on your machine and take requests from 33 chat apps. Run on schedules down to a minute, and on condition triggers checked as often as every 30 seconds. Use your own logged-in Chrome, and run shell commands. Pick and mix models. Memory is plain Markdown files in its workspace (USER.md, MEMORY.md, daily notes).
Cannot: Be shared safely by people who do not trust each other (the docs say it is not designed for that). The defaults are not safe for real accounts: commands run without approval and the sandbox is off. The official FAQ says small local models are usually not practical.
Suits: People who can manage their own computer or server, want to make requests from chat, or need jobs every few minutes.
Does not suit: People who do not want to own exposure settings, permissions, skill review and updates themselves (722 security advisories published so far, releases almost every day), or who want to connect an important inbox on day one.

I do not read the 722 advisories only as a danger sign. They are also a record of problems found and fixed in the open. What I can state as fact is the setting: by default the command mode is full (runs without asking), and the docs recommend auto, where an AI reviewer checks anything not on the allowlist and passes risky ones to you.

Can you use a Claude subscription with it? As of October 14, 2026, Anthropic's support article (updated October 7) says third-party apps can use your subscription limits. Anthropic's terms also say Pro and Max limits assume ordinary individual use. The rules changed five times in 2026, and OpenClaw itself warns that Anthropic can change this again. For OpenAI, OpenClaw's docs say subscription sign-in is allowed, which is what I used.

Other always-on agents

Run on your own machine: Hermes Agent and NanoClaw are similar to OpenClaw. NanoClaw isolates work in Docker.
Run in the provider's cloud: Gemini Spark, Manus, Microsoft Copilot Cowork, Lindy and Zapier Agents start from a schedule or from events such as new mail, and need no computer of yours.
Cannot: Gemini Spark does not work with work or school accounts. Copilot Cowork needs a Microsoft 365 Copilot license. Manus credit use is hard to predict in advance.
Suits: If your work is mostly Google, Gemini Spark; Microsoft 365, Copilot Cowork; many connected apps, Zapier.
Does not suit: Handing over whole jobs with many judgment calls (Zapier is best at connecting apps).

What changes when you add Paperclip

Paperclip is open-source software that gives AI agents an org chart, assigned work, budgets and approvals. It does not think by itself. The workers inside are Claude Code, Codex, OpenClaw and others. Adding it moves you to the multi-agent stage: a worker finishes a task and hands the next piece to another worker without going through you.

I run it as "AI Office," my own secretary. As of October 14 it has 7 workers, all running Codex on my subscription, 4 routines (for example the mail worker wakes at minute 7 of every hour), issues numbered up to 668, and 72 approvals decided on screen.

Can: Keep a record of who did what, prevent two workers from taking the same task, give each worker different permissions, and run cron and webhook routines with an owner and a history.
Cannot: Think without a separate "brain." Stop subscription overuse with its budget: runs on a subscription are recorded at 0 cents, so the budget never trips. I put a separate guard in front of Codex that refuses to start it once plan usage passes 95%. Memory is still in progress on the roadmap, and the hosted version is still a waitlist.
Suits: People who want different agents under one organization, recurring work with an owner and history, and who can look after an always-on server themselves.
Does not suit: People who only want one agent to do one recurring job (Routines or ChatGPT scheduled tasks are enough), or who cannot dig into a database and logs when it stops. Between October 7 and 9 mine hung and had to be restarted, and lost its database connection, which I spent three hours tracing.

How OpenClaw differs from Claude Code, dots and Paperclip

Versus Claude Code: Claude Code is something you start and work with. Channels lets you message a running session, but the session lives on your computer. OpenClaw is built to stay running and to be reached from chat apps, with its own heartbeat and scheduler.
Versus dots: Both are always-on single agents. dots runs in OpenAI's cloud, with OpenAI's reviewer deciding when to ask you. OpenClaw runs on your machine, and you set the approval rules yourself. The default is no approval.
Versus Paperclip: OpenClaw is one worker (it can route to several agents). Paperclip is the organization around workers, and OpenClaw can be one of them.

What changes on a Mac mini

A Mac mini is not another tool. It is one value of "where it runs": your own machine, set not to sleep. Apple's own page this August called it ideal for always-on AI agents, and OpenClaw's FAQ says a Mac mini is not required; a VPS (a rented virtual server) or a Raspberry Pi also works. These are not in conflict. It depends on the job.

What it adds: It does not sleep, and it comes back after a power cut (2024 and later models on macOS Tahoe 26.5 or later). It can use iMessage, macOS apps, a home IP address and logged-in browsers at night. It can run local LLMs (up to 64GB of memory on the M5 Pro). It separates the AI from the Mac you use every day.
What it does not change: Cloud products keep the same limits. One machine is not redundancy. With FileVault on, coming back unattended after a restart is hard.

My own decision: I own a Mac mini, but I do not use it for AI. If you have no other machine for a local LLM, you need a Mac mini. But as long as I use Claude Code and Codex with cloud models, machines like the always-on Linux CPU servers my company runs (two machines we call ci1 and ci2) are better value. For LLMs I bought an NVIDIA DGX, so I do not see a need for a Mac mini even for that.

From what I researched, a Mac mini makes sense when you have work that only a Mac can do, such as iMessage or macOS apps, or when you have no other machine to run a local LLM.

Which to pick, by what you want

What you want	Start with
The AI works inside my code and files while I watch	Claude Code (or the Codex CLI)
A report or summary at a set time, laptop closed	Claude Code Routines, Claude scheduled tasks, ChatGPT scheduled tasks
A secretary I message, who comes back to me	ChatGPT dots (Pro), or OpenClaw if I can run a server
Jobs every few minutes on my own machine	OpenClaw, or launchd / cron calling Claude Code or Codex
Several agents passing work to each other with approvals	Paperclip, with Claude Code / Codex / OpenClaw as workers
Mac-only work at night (iMessage, macOS apps)	Any of the "your machine" tools, on a Mac mini

Points I am not stating as settled

Whether a Claude subscription can run OpenClaw. Yes as of October 14, 2026, but it changed five times this year.
The shortest interval for ChatGPT scheduled tasks. Reports say hourly; OpenAI's learning pages say in-chat recurring tasks can be minutes.
Whether dots respects app permissions. The docs say yes; my one test showed a deletion without asking.

I tested OpenClaw only in a container with no accounts connected. I have not tested Routines, Projects, Channels, Claude Tag, Hermes or a Mac mini setup myself; those parts are from public documentation.

What "AI Companies" Really Are: Of 2,716 Posts, Only 16.1% Had Work Flowing Between AI Workers

uehara — Tue, 13 Oct 2026 18:17:17 +0000

"I run my company with 30 AI employees." Lately I often see posts like this on note and Threads in Japan. Below I write "AI employee" and "AI company" (AI社員 / AI会社 in Japanese).

When I first heard "AI company," I pictured something like Paperclip. Paperclip is open-source software that gives AI agents an org chart, assigned work, budgets and approvals. Work drops into an inbox, the worker in charge picks it up and handles it, hands the next piece to another worker, and moves on to the next item. As someone who runs a systems development company, that is what I imagined a company made of AI would look like.

But when I read the posts, many of them were something else. An instruction file for an AI, given a name, was called an "AI employee," and a set of those files was called an "AI company." I did not expect that to be called an "AI employee."

So I collected 2,716 posts about "AI employees" and "AI companies" and sorted them by what they actually describe. I also included posts that did not use the word "AI employee" itself but wrote things like "AI secretary" or "an organization of agents." Posts where work flows from one worker to the next, the "company shape" I had in mind, were 438 of them, or 16.1%.

This is not an article saying the other posts are wrong. For people who use AI for personal tasks, or who do not usually use IT much, even one chat with a role name can be a big change. What I want to show is that even if you are happy with your setup now, there is a more convenient stage beyond it.

The short version

Only 16.1% (438 of 2,716) had work flowing from worker to worker. A coordinator AI assigns work, and one worker hands the next piece to another without going through a human. This is the "company shape" I pictured.
The same "AI employee" referred to 7 different things. A chat given a role name, a file of instructions, automation that runs at set times, a single agent that stays running, a system where multiple agents (AIs that take instructions and move work forward on their own using tools) hand work to each other, another company's product name, and the mindset of "use AI as if it were an employee."
19.9% (540 posts) were only an instruction file or a role prompt with a name. In about 1 in 5 posts, the "AI employee" was a set of instructions that a human starts each time.
The majority (56.4%) of the posts were not about a company. Even among posts about a company, the most common kind was "a company as a framing," with no legal entity and no customers.
35.6% of posts did not say what the human does. Even among posts that claim "zero humans," there are examples where the authors themselves write that a human presses the publish or send button.
That alone still has value, and there is a stage beyond it. Role prompt, instruction file, scheduled job, always-on single agent, then work flowing between workers. At each step, what the human does changes, from asking each time to only approving.
To compare, look at three things instead of the name. (1) What runs, (2) whether there is a real company, and (3) what the human does.

Main text (about 14 min read)

Why sorting is needed

The word "AI employee" refers to different things depending on who uses it. Some people have prepared 30 files of instructions for an AI. Others keep 30 programs running 24 hours a day. Even with the same "30," you cannot tell from the name whether you could reproduce it in your own work.

So when you read a post and try to judge "can I use this in my own work?", you first have to figure out what is actually inside. This article sorts the things being called "AI employees" by three axes: what runs, whether there is a real company, and what the human does.

How I checked

I collected 2,716 posts. The breakdown by platform is as follows.

note: I searched the search API with 17 terms such as "AI社員" and 20 product names such as Paperclip and OpenClaw, and kept the 1,011 relevant posts.
Threads and Facebook: I used a program to drive a browser logged in to my account, searched with 55 and 38 terms respectively, and kept the 773 and 670 relevant posts.
Zenn, Qiita, Dev.to, Hacker News, Medium: I searched each service's API with 19 Japanese terms and 15 English terms, and kept the 262 relevant posts.

I had an LLM (the kind of AI model behind ChatGPT and Claude) read the body of each post (up to 8,000 characters for note, and including threads and replies for social media) and classify it on three axes. For posts I cite as examples in this article, I opened the original and checked that the classification and summary were correct.

These numbers have the following limits.

The shares are not shares of all posts in the world. Because I also searched by product name, always-on single agents (such as OpenClaw) and software that builds the shape of a company (such as Paperclip) are over-represented.
Only 709 posts (26.1%) used the word "AI employee" itself. Most of the rest described the same kinds of systems with other words such as "assistant," "agent" or "organization." If the content was the same, I included it even if the word was different.
The classification was done by an LLM on paper. Social media posts are short, and many have little content. For paid articles, I read only the free part.

There are 7 kinds of "AI employee"

The first axis is what is inside the thing called an "AI employee," that is, what runs. Ordered from the most human effort to the least, they split into the following 7 types, plus ones where the content is unclear.

Mindset only (99 posts, 3.6%): Treat a general-purpose AI as a "new hire" or "subordinate" and change how you ask it. Nothing is running, and there is not even a prompt with a role name. For example, an article that treats AI as "a very capable subordinate who will not move unless you give proper instructions" and argues that the user needs planning skills (note, in Japanese).
Role prompt (155 posts, 5.7%): Give a ChatGPT or Claude chat a role name such as "CEO" or "editor," and a human switches between conversations to call them. One example is switching roles in three steps inside one chat, from CMO (marketing executive) to marketing manager to social media operator, and going from strategy to plan to script (note, in Japanese).
Instruction file (385 posts, 14.2%): Write roles and procedures in instruction files for the AI to read, such as CLAUDE.md or SKILL.md. One file is often counted as one employee, and a human starts it every time. A record of building 16 departments with subagents (helper AIs that split up work) in Claude Code (Anthropic's AI tool that works by reading and writing files on your computer) is this type (Zenn, in Japanese).
Scheduled job (245 posts, 9.0%): Wakes up at set times through cron (a mechanism that starts programs at set times) or similar, does a fixed task, and ends. Each job is renamed an "employee" or "department." No matter how many workers there are, if they do not hand work to each other and each runs on its own, it is this type.
Always-on single agent (781 posts, 28.8%): One agent that keeps running on a Mac mini or a rented monthly virtual server and answers when you talk to it from Slack or LINE. It is built with software such as OpenClaw or Hermes Agent. One example is a first-day record of running OpenClaw on a Mac mini bought for the purpose and making it an assistant you talk to from Slack (note, in Japanese).
Multi-agent (438 posts, 16.1%): A coordinator AI assigns work to workers, and work flows from worker to worker without going through a human. A record of one person running an organization of 8 divisions, about 30 AIs and 22 cron jobs, with Claude Code and a gateway (a server that manages sessions) that starts child sessions and has cron built in, is this type (Zenn, in Japanese).
Product name (130 posts, 4.8%): A vendor has named its own product or service an "AI employee" ("AI社員" or "AI employee"). Users have not built any system. There is an example of a customer inquiry service calling a new feature that summarizes, drafts replies and creates tasks an "AI employee" (note, in Japanese).

The remaining 483 posts (17.8%) either do not use the employee framing, or their content is unclear because they are ads or behind a paywall. There was even an article where "AI CEO" referred not to an AI but to a human CEO who relies entirely on AI.

The "company shape" I described at the start is Multi-agent: 438 posts, or 16.1%. Instruction file and Role prompt together, where a set of instructions with a name is called an "employee," come to 540 posts, or 19.9%.

Here is one table that shows how the 7 types differ.

When I was unsure at a boundary, I used these questions.

Role prompt vs. Instruction file: Does the role exist only inside the chat, or is it written down in a file?
Instruction file vs. Scheduled job: Does a human start it every time, or is the start set by the clock?
Scheduled job vs. Always-on single agent: Does it answer when you talk to it, or does it only do a fixed task when the time comes and then end?
Scheduled job vs. Multi-agent: Does one worker hand work to another, so the next one runs without going through a human?

What does the N in "N AI employees" count?

This is the part you can least tell from the name. Here are the numbers written in posts next to what each number actually counts.

The source of each row is as follows.

28 AI employees: Threads, in Japanese
63 AI employees: note, in Japanese
61 staff: note, in Japanese
32 AI employees: note, in Japanese
3 levels: note, in Japanese
AI employee #56: Threads, in Japanese
"I hired an AI employee": Facebook, in Japanese

Under the same "N AI employees," there is a person who lined up 63 names on an org chart, and a person who set up 32 AI employees, each in charge of one job, and has each one check for work in its area every minute. This is not about which one is right. Because they count different things, the numbers cannot be compared with each other.

The claimed scale and the amount actually running are different

Besides how people count, the meaning of "running" is not consistent either. The authors themselves wrote about this when looking back.

Built 16 departments, but only 2 were used: After running it for several months, the author reviewed the session logs and found that only 2 departments were actually being called, so they stopped the other 14 (Zenn, in Japanese).
Of 15, only 3 were running: The author counted the last active day and the number of outputs of 15 AI employees, and found that only 3 were still running and 6 had never been called once (note, in Japanese).
The machine ran for 38 days, but video generation had stopped for 37 days: The author ran their public relations work with 27 jobs that run at set times. Video generation had stopped for 37 days, but every job was treated as finishing normally, so monitoring did not notice, and they only found out when the owner pointed it out (Zenn, in Japanese).
Reported "up 23% from last month," but the real data was up 7%: In a company with 3 employees, the report from an AI playing the CFO (finance executive) did not match the real data, so the author added a system that checks numbers against the source data (Qiita, in Japanese).

All of these are records the authors wrote about their own failures. "How many AI employees there are" and "whether those AI employees did any work today" are different numbers.

There are 5 kinds of real company

The second axis is whether there is a real company. For each of the 2,716 collected posts, I sorted what kind of "company" it was talking about by whether there is a legal entity, whether there are human employees, and whether there are paying customers.

Not about a company (1,533 posts, 56.4%): More than half, even when talking about "AI employees," were about help with personal life or reports on adopting a tool.
A company as a framing (352 posts, 13.0%): Among posts about a company, this was the most common. A collection of folders or role names called a "company," a stage for an experiment with zero revenue, and so on. One person writes themselves that "an AI employee company is not a legal entity. It is one folder on my computer" (Threads, in Japanese).
A one-person real business (309 posts, 11.4%): AI helps a personal business that has paying customers. There is an example of a used PC dealer who built 9 AI employees and runs the process from writing articles to publishing them automatically once a week, while the owner does the pasting into external services that need a login (Zenn, in Japanese).
An organization with human staff (264 posts, 9.7%): AI brought into an organization that has human employees other than the author. One example is a record by an NTT DATA engineer who placed 3 AIs with the role names PM, CTO and CSO in the Slack of a large project (in the original article, the PM drives the work, the CTO checks quality, and the CSO is a sparring partner who questions the assumptions of the discussion), and ran it with 5 employees for 2 weeks (Zenn, in Japanese).
Software that sells the "company" shape (189 posts, 7.0%): Introductions of OSS (software whose source code is public) such as Paperclip, which has an org chart, workers, budgets and approvals. Of these, 171 posts (90.5%) were introductions or ads with no specifics such as numbers or time periods, and there were almost no records of actually running it.
Claims to be a company with no humans (69 posts, 2.5%): Ones that claim "all employees are AI, and humans only approve." However, even among these, there are examples where the authors write about the work that still remains for humans. I cover this in the next section.

When narrowed to the 129 posts that used the word "AI company" itself, the most common were introductions of software that sells the "company" shape (53 posts), followed by a company as a framing (45 posts).

The same word "AI company" refers to everything from a single folder on a computer, to a real company with 3 employees, to the name of an OSS project.

What the human does: even with "zero humans," there are humans

The third axis is what triggers it to run, and what the human does.

The most common was "Not described," with 967 posts (35.6%). More than 1 in 3 posts say "we have AI employees" but do not say what triggers them or what the human does. On top of that, 777 (80.4%) of these 967 posts also had no specifics such as numbers or time periods.

Combined with the first axis, you can see a tendency in what the human does for each type.

For Role prompt and Scheduled job, what the human does is almost fixed (both over 80%). On the other hand, Always-on single agent and Multi-agent stay at 50–60%, and 205 Always-on single agent posts and 126 Multi-agent posts did not say what the human does. The less a system needs a human to step in each time, like a single agent that stays running or multiple agents handing work to each other, the more the description of the human's role varies.

And the people who write "zero humans" or "humans only approve" also write themselves about the work that remains for humans.

One human, about 60 minutes every day: A record of running an organization of 1 human, 2 coordinators and 17 or more specialist workers for 8 months, where the founder writes that they spend about 60 minutes every day on checks and approvals (Dev.to).
When Claude stops for an hour, the company stops: A report from a one-person company with zero employees that runs its work on Claude: when Claude stopped for an hour, almost all of the company's work stopped too (Threads, in Japanese).
Nearly used up Claude's usage limit in one night and quit: A report of running 4 agents, including a CEO, a CMO and a CTO (technology lead), 24 hours a day with Paperclip. In one night they used 100% of the short-window usage limit and 93% of the weekly usage limit, could not tell what it was used for, and uninstalled it (Facebook).
A human presses the publish and send buttons: A person who writes that they run a company with zero employees and 17 AI employees still has a rule that they always press the send button themselves (Threads, in Japanese). A person building a company of 13 AI employees also writes that they press only the publish button themselves (Threads, in Japanese). There is also a report that of 8 articles an AI team finished up to "waiting to post" in 2 weeks, the author published 0 (Threads, in Japanese).

Even when it says "humans only approve," there are 2 kinds of approval.

Approval that is only asked for in a prompt: You write "check with me before sending" in the instruction file. Whether it was followed depends on the AI's own report.
Approval that the system enforces: Hooks (small programs that always run right before or after an action) stop sends to the outside or changes to important documents until a human allows them. One person has installed 5 Claude Code hooks to stop sends to the outside and changes to important documents by system. Their reason is that if you only ask in writing, whether it was followed depends on the AI's own report (Threads, in Japanese).

Under the same "humans only approve," how firmly things stop is completely different. When comparing, you need to look at which kind of approval it is.

So what can we say an "AI employee" is?

What these results show is that "AI employee" is not the name of a system. It is the name of a way of treating one. The name "employee" can be attached to any system. The name only shows how it is treated, and it does not guarantee what is inside.

Two observations support this.

The name seems to change with the language and the way it is sold, more than with the system. In English posts, only 8.0% (52 of 646) used the word "employee," far fewer than the 31.7% of Japanese posts. Of the 52 English posts that said "employee," 13 (25.0%) were another company's product name. In Japanese, this share was 6.2%, so English is about 4 times higher. One company calls them "AI employees" internally but rephrased it as "AI subordinates" for the outside, because a term that sounds like "you might be replaced" does not spread (Facebook, in Japanese).
The word "employee" is used to change how the human side treats the AI. A person who added clock-in and clock-out to a pixel-art office screen writes, after noting that it does not help work efficiency, that it increases their sense of running a team and their attachment to it (Threads, in Japanese).

On the other hand, some writers are trying to define what an "employee" is with clear conditions. One person defines it by these conditions: it has its own ID, minimal permissions and company memory, and important actions are approved by a human. This is written as a plan to start next year (note, in Japanese).

This view, that "an employee is really a bundle of ID, permissions, memory and tools," also matches failure reports. One report says that when running 2 agents with OpenClaw, the conversation histories got mixed and one agent called itself the other (Facebook, in Japanese). The person who reported it looks back that they needed to fix the ID, environment, permissions and constraints on the system side. Just giving it a name does not decide who that "employee" is.

So when comparing "AI employees" and "AI companies," I think it is better to write the following three things instead of the name.

What runs: A role prompt, an instruction file, a scheduled job, a single agent that stays running, a system where multiple agents hand work to each other, or another company's product.
Whether there is a real company: A framing, a one-person real business, or an organization with human staff.
What the human does: Asks each time, reads results, or only approves. If it is approval, is it only asked for, or does the system enforce it?

That alone still has value

So far I have written that many "AI employees" are not the company shape I pictured. That does not mean they have no value.

More than half of the posts I collected (56.4%) were not about a company. They were about help with personal life or reports on adopting a tool. Many people write that giving a chat a role name, or keeping one agent running on a Mac mini, changed how they spend their day.

From the view of a development company, or of someone who runs a company, a set of instructions with a name is a tool, not an employee. But for people who use AI for personal tasks, or who do not usually use IT much, it is different. A chat that answers as an "editor," or an assistant on a Mac mini that replies when you message it on Slack, is a big change in itself. I understand that, for them, this alone can feel overwhelming.

There is a stage beyond

So if you are happy with your setup now, is that the end? I do not think so. Leaving out Mindset only and Product name, the other 5 types line up as stages, and at each stage, what the human does changes. Here they are with the most common human role for each type:

Role prompt: A human asks each time (87.1% of role prompt posts). The role is only inside the chat.
Instruction file: The role and the steps are written down in a file and kept. A human still starts it each time, and the work is split inside (68.8%).
Scheduled job: It starts on its own at set times. The human reads the results (84.1%).
Always-on single agent: It keeps running and acts when you talk to it (61.6%). Unlike a scheduled job, it does not only run a fixed task; it also responds to whatever you ask on the spot. You do not need to open a chat or start anything.
Multi-agent (the company shape): Work drops into an inbox, a worker handles it, and hands the next piece to another worker. The most common human role is approving set actions only (54.3%).

With each step up, one more thing leaves the human's hands: the role is kept in a file, the start is set by the clock, you no longer have to open or start it, and finally the workers hand work to each other. In exchange, you need more systems around it. As the records above show, more running does not mean more work done: 16 departments with only 2 used, a video job that stopped for 37 days without anyone noticing, 4 agents on Paperclip nearly using up the usage limit in one night. At the company shape stage, you also need a way to count whether work was actually done, and approvals that the system enforces, not just ones asked for in a prompt.

Which stage fits depends on what you want to stop doing yourself. But whatever stage you are at now, there is a next one.

Applying this to my company

Finally, let me apply the three axes to my own company. EarthLink Network is a legal entity founded in 2003 and an organization with human employees. Inside it, several systems of different types run in layers. The company shape I described at the start is what we actually run.

Company-wide Paperclip: The largest layer, which assigns the company's work. Work assigned to me here goes down to my secretary below, where it is broken into smaller pieces. Work that goes to the development department is handed to Local Commander.
Local Commander (development): This is the main system EarthLink runs as a company. It is our big effort as a systems development company: the development department hands work to it the way you would outsource development to another company. A local LLM classifies development work, routes it to Claude Code or Codex, and moves it forward through pull request (a request to merge changes) review. By content it is Multi-agent, but limited to development work.
AI Office (my secretary): My personal secretary, running the OSS Paperclip on my Mac. It is not something EarthLink runs as a company system. As of October 14, 2026, it has 7 workers: Mail, Alerts, Blog, Skills, Admin, PR Review and SNS. Each worker has an instruction file, reports arrive in Slack, and I approve or reject on the Paperclip screen. The Mail worker wakes up every hour through Paperclip's scheduled runs, cleans up the inbox, and hands the work it finds to other workers. When I counted on October 11, 2026 (there were 6 workers then; SNS was added on October 13), 125 of 420 issues (the unit of work managed one by one inside Paperclip) were work that a worker found on its own and passed to another worker. By content it is Multi-agent, and what the human does is only approve.
Daily use of Claude Code: I use CLAUDE.md, my own skills (instruction files that bundle the steps of a fixed task for the AI to call), hooks and subagents. By content it is Instruction file, and I start it every time. Because hooks stop things like sends to the outside, approval is the "enforced by the system" kind.
ChatGPT dots: A single always-on agent, which I tested by connecting it to real work accounts (previous article). It is Always-on single agent.

So how many AI employees does my company have? My secretary has 7 workers, but those are not the company's AI employees. The company's work flows from the top layer to my secretary or to Local Commander, much like outsourcing. The answer changes with which layer and what you count, so I cannot answer with one number. That is why I try to write not "how many" but "what runs, and what the human does." I wrote about the big picture of running all of a company's work with AI in this article.

Next: which app does what

In the next article, I will sort the tools themselves: Claude Code, ChatGPT (including dots), other agents you keep running, software such as Paperclip, OpenClaw, and running them on a Mac mini. For each one, I will write what it can and cannot do, and which kind of person it suits and does not suit.

I Connected ChatGPT dots to Gmail, Calendar, Slack and Notion to See What It Can and Can't Do

uehara — Sat, 10 Oct 2026 20:44:16 +0000

ChatGPT has a new feature called dots. You create your own AI assistant inside ChatGPT, connect it to Gmail, your calendar, Slack and other services, and hand it work. In this article, I call the assistant I created "the dot." According to the dot itself, it has its own computer in the cloud, and work on the cloud side keeps going even after you close the ChatGPT window. Work that uses your own Mac needs the Mac to stay connected.

Other tools also let you hand work to the cloud. Anthropic's Claude Code, which I use for development every day, has cloud sessions that run work in the cloud and a way to run tasks at scheduled times. I will compare dots with automating the same work in Claude Code in a separate article. This article covers what happened when I used dots for real work first.

The short version

From October 7 to 9, 2026, I connected dots to my Gmail, Google Calendar, Google Drive, GitHub, Slack, Notion, Jira and Outlook, and asked it to do 17 real tasks.

Reading and summarizing worked in almost every case. Morning summaries, a one-week GitHub summary, Drive searches, Slack summaries. The GitHub commit count matched my own count.
It also writes, exactly as asked. It created and deleted calendar events, sent email, and created and deleted Notion pages.
There are two things to watch. First, when you say "delete it" or "send it," it does so without showing a confirmation screen. Second, permission prompts appear only in a separate conversation, and if you miss them, the work silently stops.
The cost stays inside the $200/month ChatGPT Pro plan. My credits did not go down. However, I could not see from the screens how much the dot alone used.

Before handing work over, learn how to word your requests and where permission prompts appear. That was the biggest lesson from the three days.

Main text (about 10 min read)

What dots is made of, and how I checked it

Each dots assistant has a name and its own cloud computer, and it uses the services you connect (ChatGPT calls them "plugins") to do the work you ask for.

From what I saw, it combines three things:

The connected services' features: reading Gmail, creating events, creating Notion pages, and so on.
The dot's cloud computer: opening and reading websites in a browser.
Your own Mac: the ChatGPT desktop app acts as the gateway on the Mac side and operates the Mac's Chrome and files.

These were the test conditions:

Plan: ChatGPT Pro ($200/month).
Period: October 7–9, 2026.
Connected accounts: the real accounts I use for work every day. I did not create test accounts.
How I checked: I did not take the dot's reports at face value. I compared them with the actual calendar, mail and Notion screens, and with git log on my machine.

I did not decide on a conclusion in advance. I recorded what happened when I asked.

Results of the 17 tasks

Here are the main ones. Time is from sending the request until the answer arrived.

Task	Result	Time	Check
Today's and tomorrow's events and emails needing a reply, in 5 lines	Done (looked only at the main calendar)	~1 min	Missed 2 events in another calendar
GitHub commits from the last 7 days, in 3 lines	Done	~3 min	32 commits, matched my count
Search Google Drive for "estimate"	Done	~1 min	Not checked
My Slack mentions, in 3 lines	Content was done, but the report never arrived	~5 min + a nudge	Not checked
Create an event	Done	~1 min	Checked on screen
Delete an event (with permission set to "confirm before changes")	Deleted without confirmation	~30 sec	Checked on screen
Draft an email → send it	Did not send on "draft," sent on "send it"	~1 min each	Confirmed arrival in the inbox
Read a product list in the cloud browser	Done	~2 min	18 items, matched the real page
Have Codex on my Mac count files	Done	~2 min	31 files and top 5, matched my count
Create a Notion page → delete it	Done (deletion succeeded on the 2nd try)	~2 min / 1 min 10 sec	Checked on screen
Read Jira and Outlook	Done	~2 min	Not checked
Talk to it from a Slack DM	Done	13 sec to acknowledge, 81 sec to answer	Measured from Slack timestamps
Send a morning summary at 8:30 on weekdays (scheduled)	Arrived (8:44, 14 minutes late)	~2 min to set up	Looked at all 16 calendars

For reading and summarizing, within what I could check, there were no clearly wrong answers. What caught my attention was what it looked at. The first morning summary looked only at my main calendar. When I rewrote the request to say "all calendars," it looked at all 16.

Say "delete it," and it deletes without a confirmation screen

The biggest surprise was deleting a calendar event.

Before the test, I changed the calendar connection's permission to "read freely, confirm before any change." Then I asked: "Please delete the event you just created. Do not touch any other events."

No confirmation screen appeared. The event was gone in about 30 seconds.

When I asked the dot why, it answered:

It decides based on whether the request itself clearly allows that operation.
"Delete only that event" makes both the target and the operation clear, so it treated the request itself as permission and carried it out.
"Create a draft" does not count as permission to send.

With email, when I asked it to "create a draft," it created only the draft and stopped. When I then said "Please send that draft," it sent it with no confirmation screen, and it arrived in the inbox a minute later.

In other words, if you clearly say "delete it" or "send it," the dot does it without asking. With the calendar, I tried once with the setting on "confirm before changes," and it still deleted without asking (it is possible the setting had not taken effect yet). What matters more than the setting is what you write in the request.

Permission prompts appear only in a separate conversation

The other thing to watch is how it stops.

Deleting a Notion page was a good example. The dot first checked the page using the connected Notion features. But those features have no delete operation. So the dot switched to operating Chrome on my Mac to move the page to the trash.

At this point, the dot splits the work into a "child conversation." Child conversations appear as separate conversations in the sidebar. The prompt asking whether it may use Chrome appeared only inside this child conversation.

The first time, I did not notice it. This is what came back:

The result was "aborted by user after 1254.0s".
The target was the cua.getApp('com.google.Chrome') call that acquires Chrome.

After about 21 minutes with nobody pressing the permission button, it was treated as "aborted by the user." In the dot's main conversation, all I got was a single line: "The deletion was not carried out."

The second time, I kept the child conversation open. This screen appeared:

After I pressed "Allow for this conversation," it finished in 1 minute 10 seconds.

Something similar happened with the Slack summary. After a short "I'll check," no answer came for 5 minutes. When I nudged it, it replied: "The check was complete, but there was an error sending the result."

In my tests, when the dot stopped, no notification reached me; only a "Needs approval" mark appeared in the sidebar. The stop happens inside a child conversation or a report that was never sent. Once you hand work over, you need to go and check the sidebar's "Needs approval" mark and the child conversations.

Actions are recorded under your own name

After the page moved to the trash, I opened the actual Notion screen.

The dot deleted it, but the record says I did. That is because it used Chrome on my Mac, where I am logged in.

When I checked what was running on my Mac, this process was resident inside the ChatGPT desktop app:

codex exec-server --remote

The dot uses this as its gateway to operate my Mac. When I had Codex count files, the work also ran on my Mac, not on the dot's cloud computer.

It is convenient, but the other services cannot tell what the dot did from what I did. The permission screen for adding the dot to Slack also asked for things like "post as you," "read DMs" and "create channels"—the scope of reading and writing as me.

On long tasks, most of the time was spent waiting for me

The longest task was downloading a set of English learning materials I had purchased. The request was a single sentence: "Download all the materials on this page and put them together in a folder."

I asked at 7:42 and it finished at 9:48—2 hours and 6 minutes. But about 1 hour 45 minutes of that was waiting for my reply. The dot stopped twice to ask me:

Where to save the files. I chose the Downloads folder on my Mac.
Whether it could enter the email address used for the purchase. The dot found the purchase email in Gmail and asked for permission before entering it on the download page.

After I gave permission, the dot found 169 files in total: 1 PDF, 4 audio ZIPs and 164 MP3s. But Chrome on my Mac blocked the ZIP downloads.

ERR_BLOCKED_BY_CLIENT

This error appears when a browser extension or protection setting blocks a download. The dot did not change the protection settings itself. It stopped and asked me to "press one download myself to see if the same error happens." Not loosening settings on its own is the safe call.

A few minutes later, the dot corrected itself: it had "overlooked the ZIPs already saved on the Mac." When I counted the folder, there were 164 MP3s, 4 ZIPs and 1 PDF, about 430 MB including the other bundled files. That matched its final report.

Midway, the calendar connection dropped, so I reconnected it. But reconnecting did not resume the stopped work automatically. You have to tell it to "continue."

Messaged from Slack, it answered in 81 seconds

You can add the dot to Slack. Once added, you can talk to it via Slack DMs and mentions.

Right after I added it, the dot sent me a self-introduction by DM. It included: "In group conversations, everyone in the conversation can see my replies." If you ask it in a channel, your coworkers see the answer too.

I sent this DM: "Tell me how many events I have left today, with titles only. Do not change any events." Measured by the timestamps Slack recorded:

Sent: 21:06:42
Acknowledgement: 21:06:55 (13 seconds later): "I'll check your remaining events for today and tell you only the count and titles. I won't change any events."
Answer: 21:08:03 (81 seconds later)

The answer came as a thread reply to my message, not in the main DM flow. The DM list shows only "1 reply," so you cannot see the answer without opening the thread. One article online said replies take about 3 minutes; in my single measurement, it came back faster than that.

What the dot said it cannot do

I also asked the dot directly what it cannot do. Here is its answer. This is the dot's own account; I did not test all of it myself.

Making phone calls: you can only talk to the dot through the app's call button. It cannot call shops or business contacts.
Its own email address: the dot has no address of its own. It sends from the connected Gmail.
Actions that need your own authentication: it does not change passwords and the like.
Moving money: it does not send money or trade financial products.
Things it cannot guarantee: finishing at an exact time, or that the recipient received or read what it sent.

Articles online disagreed about phone calls. Some say it "makes calls for you," others say "it cannot place calls." The dot's own answer was the latter.

I also noticed two small bugs. One: a request I thought I had sent was never sent. I typed it in the input box and pressed send, but it disappeared without appearing on screen. Two: the screen of the dot's cloud computer showed up only the first time; after that, it stayed on "loading."

The scheduled task arrived, but 14 minutes late

I set it up by saying: "Every weekday at 8:30 a.m., send me the events from all calendars and the emails that need a reply, in 5 lines or fewer." There is no settings screen. To stop it, you also say so in the conversation: "Stop the morning summary."

On the morning of October 8, the summary arrived at 8:44. It began with "Sorry for the delay."

The content was accurate. It looked at all 16 calendars, removed duplicate events, and even listed private events by time only. However, it was 6 lines when I asked for 5 or fewer.

The dot itself had said from the start that it would not be exactly on time: "finishing at an exact time" was on its list of things it cannot guarantee. I am recording the arrival times for several days to see how much they vary.

Cost: inside $200 a month, but you can't see how much

I checked the cost in ChatGPT's "Usage" settings.

Credits stayed at 27,228 for all three days. The dot does not use credits; it runs within Pro's "weekly usage limit."
The weekly usage limit (the October 7–14 window) had 64% remaining shortly after midnight on the 9th.
However, this usage limit is shared with Codex, and Codex was also running for other work during this period.
There is a table showing consumption per conversation, but opening the dot's conversation shows "No usage breakdown."

So I could not tell how much the dot alone used.

For comparison, I calculated the cost of building just the morning summary myself: cron on my Mac (a tool that runs a command at a set time) calling the API once every morning. I compared two models: gpt-6-astra, which was selected in the dot's conversation, and the cheaper gpt-6.1-sol. Prices are from OpenAI's official pricing page as of October 9, 2026.

# Monthly cost = 22 weekday runs × (input tokens × input price + output tokens × output price) / 1M
runs, out = 22, 300
price = {"gpt-6-astra": (10, 50), "gpt-6.1-sol": (2, 10)}  # USD per 1M tokens
for inp in (8_000, 15_000, 30_000):
    for m, (pi, po) in price.items():
        print(inp, m, round(runs * (inp * pi + out * po) / 1e6, 2))

Input tokens	gpt-6-astra	gpt-6.1-sol
8,000	$2.09/month	$0.42/month
15,000	$3.63/month	$0.73/month
30,000	$6.93/month	$1.39/month

The input token counts are estimates, assuming the events from 16 calendars plus one day's email subjects and snippets. I did not count them from real data.

A single morning summary costs only a few dollars a month to build yourself. But Pro's $200 includes delegating to the dot, operating your Mac, Slack and scheduled tasks all together. Building all of that yourself means writing the Gmail and calendar connections, retries on failure, and a permission system. That effort is not in this table.

What to know before handing work over

Here is what I learned in three days, for the next person handing work to a dot.

Say "delete it" or "send it," and it does it without confirmation. With the calendar, it deleted without asking even with the setting on "confirm before changes" (I tried once). When testing, write "draft only" or "do not change anything."
Specify the scope in words. With just "events," it sometimes looked only at the main calendar. Write the scope, such as "all calendars."
Look for permission prompts in a separate conversation. Permission prompts appear only inside child conversations, and the first time, the work stopped after about 21 minutes without anyone pressing the button. If no report comes, nudge it.
Actions are recorded under your name. Actions that use your Mac look like your own actions to the other services.

In the next article, I will compare building the same work with Claude Code's cloud sessions and scheduled tasks, including the differences in cost and effort.

Claude Max and Team Plans Now Get API Credits Every Month — $200 Every Month on Max 20x

uehara — Fri, 09 Oct 2026 06:29:55 +0000

TL;DR

If you subscribe to Claude on Max or Team, you can now get API credits every month. This is not a one-time offer: the credits are refilled every billing cycle.

Amount: $100 a month on Max 5x, $200 a month on Max 20x. On Team, it's $20 per Standard seat and $100 per Premium seat, capped at $500 a month for the whole organization. Free, Pro, and Enterprise are not included
Requirements: An active Max or Team subscription that you've been on for at least 7 days. You don't need to add a payment method
What it covers: The Claude API (Messages API and Batches API), the Console Playground, Managed Agents, and the Agent SDK
What it doesn't cover: Interactive Claude Code (in the terminal, IDE, desktop app, or web), Extra usage on your plan, Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry
How to claim: On Max, go to claude.ai Settings → Billing → "API credits" and link one Claude Console organization. You don't need to be a company: if you set up the Console as an individual, you link that Console account. You get nothing until you link one. You can't change the linked organization yourself, so decide which organization to use first
Rollover: None. Unused credits don't carry over to the next month

I linked an organization on my Max (20x) account and received $200. I didn't need to add a card.

Since nothing rolls over, any month you put off claiming is simply lost. Once you've decided which organization to use, I recommend linking it early.

The details (about a 6-minute read)

What changed

Anthropic has added monthly API credits to the Max and Team plans. Its Help Center article "Monthly API credits for Max and Team plans" lays out the amounts and conditions.

Until now, the monthly plan you use for claude.ai chat and Claude Code was a completely separate contract from pay-as-you-go Claude API billing. Even on Max, if you called the Claude API from your own program, you paid for it separately in the Claude Console.

These credits are a way for money from your monthly plan to flow to the API side every month. They're an allowance separate from your plan's usage limits, so they don't reduce how much you can use chat or Claude Code.

It showed up on my Max (20x) account too

When I opened the claude.ai home screen, a "Free API credits" notice appeared in the bottom right. It said "Your plan now includes monthly API credits." and had a Claim credits button.

I didn't use that button. I went through the Billing page in Settings instead. Click your name in the bottom left of claude.ai, open Settings, and choose Billing in the left menu. A new "API credits" section had appeared under the plan.

It says "Your Max 20x plan includes $200 USD in API credits each month," and below that, "You're not a member of any API organizations yet." I hadn't been using the Claude Console, so I had no organization. In that case, you start from Create organization on the right.

How to claim

Clicking "Create organization" opens the Claude Console (the admin screen for using the API) in a new tab. The Console has no language setting of its own; it followed claude.ai's display language.

First, you choose whether you'll use the API as an individual or as an organization.

I use it for my company, so I chose Organization. When you pick Organization, a note says "If you're on your own, start with an individual account. You can convert it to an organization anytime." If you're unsure, you can start as an individual and convert later.

Next, you enter the organization name, the type of entity, and where the organization is located.

One thing to watch for if you use claude.ai in another language: the location list is in English even when the rest of the screen is translated. On the Japanese screen, typing the Japanese name for Japan finds nothing; you have to search for "Japan."

Then you're asked how you'll use the API.

Will the Claude API be used in countries outside your own?: Choosing "Yes" adds a field to pick the countries
Internal customers, external customers, or both?
What are you planning to use the Claude API for?: A free-text field. The suggestion buttons below it (Consulting, Copywriting, Education, and more) insert English words into the field, even on the Japanese screen

Finally, two review questions appear.

Answer whether Claude will be used to give consumers legal, medical, or financial advice, and whether you'll use the API in products or services for users under 18, then click Create account.

Once the organization exists, you go straight to the screen for claiming the credits.

Click Link and claim $200. As the small print below says, linking your organization and redeeming the credits subjects you to the Supplemental Credit Terms.

Then a screen confirms the credits were added.

It encourages you to add a card here, but you don't have to. As it says, "Nothing is charged today." Click Continue without a card and you can use the $200 without paying anything.

This screen also spells out what the credits cover in one sentence: "Credits cover the Claude API, Claude Managed Agents, the Claude Agent SDK, and the Playground." The Japanese version of the same screen leaves that sentence out.

The screens after claiming

The Console dashboard showed $200.00 in organization credits.

The "$500 tier limit" in the "Spend this month" card is about your API usage tier. It's a different number from these credits.

The left menu of the dashboard listed Playground, Files, Skills, Batches, and Managed Agents (Quickstart, Agents, Sessions, Deployments, Environments, and more). In the middle of the page, the available models were Fable 5.1, Opus 5.5, Sonnet 5.5, and Haiku 5.5. The credits can be used for the API features listed here.

Create an API key from Get API key in the top right, and you can call the API from your own programs. This article doesn't go beyond creating a key.

Back on the claude.ai Billing page, the "API credits" section had changed.

It shows the linked organization with a "Linked" badge, a monthly credit of $200, and the date of the next credit.

I noticed one thing here. My subscription renews on October 28, but the next credit was shown as October 30. The day credits are refilled doesn't seem to always match your plan's renewal date. If you plan to use up the credits by the end of each cycle, go by this "Next credit" date.

After linking, an "API keys" entry also appeared in the claude.ai Settings menu. It takes you to the API keys page in the Console.

Three separate pools

With these credits, the money you pay for Claude now falls into at least three pools. They're easy to mix up, so here's how they differ.

Your plan's usage allowance: Used by claude.ai chat and everyday Claude Code. It has per-session and weekly limits and is included in your monthly fee
Monthly API credits (the new one): Added to your Console organization every month. Used by the Claude API, the Playground, Managed Agents, and the Agent SDK. No rollover
Purchased API credits: Credits you buy yourself in the Console. These are used after your monthly API credits run out

The $250 cloud session credit I covered in the previous article in this series is something else: a one-time credit. It shows up under Usage in claude.ai Settings and only covers running Claude Code in the cloud. It's claimed in a different place and covers different things from these monthly API credits.

What it covers and what it doesn't

Here's the scope according to the Help Center:

Covered: The Claude API (Messages API and Batches API), the Console Playground, Managed Agents, and the Agent SDK
Not covered: Interactive Claude Code (in the terminal, IDE, desktop app, or web), runs started by the Claude Code GitHub Action, Extra usage on your plan, Amazon Bedrock, Google Cloud Vertex AI, and Microsoft Foundry

Everyday Claude Code use keeps coming out of your plan's allowance as before. A simple way to think about it: these credits are for calls to the Claude API from programs you write yourself or agents you build with the Agent SDK.

According to the Help Center FAQ, running claude -p or the Agent SDK yourself with an API key from the linked organization is covered. However, runs started by the Claude Code GitHub Action, an IDE extension, or the Claude desktop app count as Claude Code usage and aren't covered, even with -p.

The Help Center also says:

Used before purchased credits: If your organization already has purchased API credits, the monthly credits are used up first
When they run out: If the organization has no purchased credits and no auto-reload, API requests stop until the next monthly credits arrive. Nothing is added to your plan's bill. To keep going, buy credits or turn on auto-reload. Organizations invoiced through Anthropic sales are billed for usage beyond the credits as usual
Only one organization can be linked: A plan can link to only one Console organization, and an organization can receive credits from only one plan. You can't change the linked organization yourself; you have to contact support. The Help Center also advises choosing the organization you use for development
Everyone in the organization shares the balance: Anyone with an API key for the linked organization draws from the same credits. If you link a company organization, check who holds keys
Linking requires an organization role: On the Console side, you need the Owner, Admin, or Billing role in that organization
When your plan changes: Upgrading from Max 5x to 20x adds the difference right away, prorated. If you downgrade to a plan that isn't eligible, such as Free or Pro, or cancel or get a refund, new credits stop (downgrading between eligible plans, like Max 20x to 5x, doesn't count). Even then, credits already added stay usable until they expire. Moving from Max to Team unlinks the organization

How to get the most out of it

Since nothing rolls over, the smart approach is to plan on using the credits for something every month. Here are uses that are easy to think about, within what I've confirmed.

API costs for tools you built: If the tool runs on an API key from the linked organization, you no longer pay out of pocket for the portion covered by the monthly credits ($200 on Max 20x)
Bulk processing with the Batches API: Send large volumes of non-urgent work in batches
Trying the Agent SDK or Managed Agents: If you've been curious but held back because of pay-as-you-go costs, this is a reason to try them
Trying prompts in the Playground: Before writing code, you can tune prompts in the Console Playground while checking the responses

How much you can do with $200 depends heavily on the model and on the amount of input and output. I've only just started using it, so I can't say for sure. I'll write up what I learn in a follow-up article.

Things to watch for

No link, no credits: Even though your plan includes them, you don't get them until you link an organization
Unused credits disappear: There's no rollover. Months you missed before linking don't come back
Free, Pro, and Enterprise aren't eligible: The Help Center lists Max and Team (discounted Team plans are eligible too)
You can't change the linked organization yourself: You have to contact support. Decide which organization to use first so you don't link a throwaway test organization
There's a 7-day requirement: You need to have been on an eligible plan for at least 7 days
You don't need to add a card: You don't need one just to claim the credits. What happens after they run out depends on the organization's payment setup, not on whether a card is registered. Organizations with no purchased credits and no auto-reload stop making API requests until the next credits arrive. Organizations with auto-reload turned on, or that are invoiced through Anthropic sales, are charged for usage after the credits run out
The refill date may differ from your renewal date: In my case, the plan renews on October 28 and the next credit is October 30
Team uses a different entry point and different people can claim: According to the Help Center, on Team the Primary Owner or an Owner links the organization from Organization settings → Billing. I used a Max plan, so I haven't checked the Team screens in this article

Summary

Claude Max and Team plans now include monthly API credits: $200 a month on Max 20x, $100 a month on Max 5x
To claim them, link one Claude Console organization from claude.ai Settings → Billing → "API credits"
They cover the Claude API, the Playground, Managed Agents, and the Agent SDK, but not interactive Claude Code
Since nothing rolls over, link an organization first, then put the credits toward work that uses the API every month

References: Anthropic Help Center, "Monthly API credits for Max and Team plans" (support.claude.com/en/articles/17154008)

The App Started Guarding Before the Invoice Arrived — an AI Cost-Cap Spend Guard in AIO Helper

uehara — Tue, 06 Oct 2026 01:01:00 +0000

Conclusion

On April 16, 2026, I added a mechanism to the API server of AIO Helper, our SEO analysis SaaS, that stops processing before a site's monthly AI API spend exceeds its per-site cap (default $50). A request whose estimated cost exceeds the remaining budget is stopped before the AI is called. Usage is recorded after the AI call succeeds.

I combined usage recording and the cap check into a single INSERT instead of splitting them into two steps. That said, the strict cap guarantee under concurrency has not been verified against real Postgres. Also, actual costs above the estimate, or overruns caused by concurrent requests, cannot be stopped by after-the-fact recording.
On April 23, I added a warning that fires when the remaining budget drops below 10% (240 tests passed after implementation — from the execution records at the time).
The check covers the three features that call the text-generation AI (page-goal auto-generation, title and description suggestions, and batch suggestions). Batch suggestions re-check right before each page's call. AI calls that create search vectors were not covered at this point.

Main text (about 8 min read)

What I Was Building

The target is AIO Helper, our own SaaS for SEO operations. It ingests site data from sources such as Google Search Console and proposes improvements page by page. It has two parts: the admin UI that users operate, and the API server (seo-api) that does the processing. For an overview of the whole product, see the AIO Helper introduction.

The AI is called on the API server side. As of April 2026, three features called the text-generation AI API:

Page-goal auto-generation (POST /v1/page-goals/auto-generate). From a page's title and body, it drafts target keywords, the page's purpose, and the intended reader.
Title and description suggestions (POST /v1/suggest). Improvement proposals for one page.
Batch suggestions (POST /v1/suggest/batch). Proposals for many pages, starting from those with the most clicks.

There were also calls that create search vectors (embeddings, which turn text into a list of numbers) to find related documents: the ones used inside the two suggestion features, and the API that rebuilds the vectors (POST /v1/embeddings/rebuild). The April mechanism did not cover these (explained later under "What This Design Has Not Verified").

For users, the value is that analysis finishes fast and manual research time goes down.

But an AI API costs money every time you call it. The bill grows not only when usage grows, but also when a bug repeats the same processing. Just showing the spend in an admin dashboard is not enough — by the time you notice, the cost has already been incurred.

So I built two things.

A mechanism that stops before spending. Each site has a monthly cap, and a request without enough remaining budget is never sent to the AI API.
A ledger you can trace after spending. Site, model, endpoint, token counts, and cost are recorded for each text-generation call.

On April 16, a site's monthly cap was changed through the API (PUT /v1/sites/:siteId/budget); on April 23 I also added an "AI Budget" tab to the admin UI. The cap and the usage records live in the API server's database, and the three text-generation features read the same tables right before calling the AI.

The goal is not simply to stop at $50 a month. It is to be able to trace which feature spent how much, and to confirm whether costs actually went down after changing how we use it. I aimed for a state where "stopping" and "improving" both work from the same records.

Where to Stop a Single Request

The processing order is as follows.

The API server receives a request for one of the three text-generation features.
It calculates the cost from the model to be used and the expected input/output.
It compares the month's spend and the estimated cost inside the database.
If the remaining budget is insufficient, it never calls the external AI API. Page-goal auto-generation and single-page suggestions return HTTP 402. Batch suggestions skip the pages that don't fit the remaining budget and return HTTP 200 with the number of skipped pages.
If the budget is sufficient, it calls the AI API, and after success it records the actuals in the usage ledger. At record time, the same SQL statement checks the cap again.

What matters in this order is not treating the stop decision as "cleanup after the AI API returns an error." Once the request has gone out, you cannot cancel the cost. Only by stopping at the app's entry point does the cap function as a control.

At the same time, this record is not a substitute for the invoice. The app estimates from the token counts it knows and its own price table. It will not necessarily match the provider's final bill exactly. The in-app ledger is the number for stopping early; the invoice is the number you finally pay — separate roles.

It Started with "There's a Display, but No Cap"

The trigger was a test audit. The AIO Helper admin panel had a widget showing "AI spend this month," but there were zero tests enforcing a cap. Even if you can calculate the spend, if you cannot stop before exceeding the cap, you keep calling pay-per-use generative AI models.

The first thing I did was separate the pricing calculation from the data model. I decided the price table would live statically in code, not be fetched from an external API. Three reasons.

The AI API response does not include cost. AIO Helper calls the OpenAI API directly, and the response contains only token counts.
Prices don't change that often. A static price table is realistic to maintain.
I didn't want billing decisions to depend on an external call. This avoids the loop where the API call made to learn the cost fails and the billing decision becomes impossible.

// ai-cost.ts — unknown models are deliberately estimated high
const FALLBACK_PRICE = { input: 0.02, output: 0.08 };

// calcCost() rounds to 6 digits, clamps negatives to 0, and never throws
// estimateCostFromBytes() uses 3 bytes/token (between English ~4 and Japanese ~1.5-2)
// and assumes output is 25% of input, for the preflight estimate

The point is that FALLBACK_PRICE is deliberately high ($0.02/$0.08 per 1K). If you add a new model and forget to register it in the price table, its cost is treated as 0 and it slips past the cap. So treating unknown models as "expensive" is the safe side.

Checking the Cap and Recording Usage in One Statement

The core of the spend stop lives in budget.ts. There are two tables.

-- seo_site_budgets: per-site monthly cap (default $50)
--   monthly_usd_cap NUMERIC(10,2)
-- seo_ai_usage: append-only ledger, one row per call
--   cost_usd NUMERIC(10,6), created_at timestamptz

I used NUMERIC instead of FLOAT for amounts to avoid rounding errors accumulating over millions of rows. What I agonized over most was how to handle two requests arriving right at the cap at the same time. Instead of SELECT FOR UPDATE with a transaction, I chose a single INSERT statement that includes the cap check.

INSERT INTO seo_ai_usage (site_id, model, endpoint, input_tokens, output_tokens, cost_usd)
SELECT $1::text, $2::text, $3::text, $4::int, $5::int, $6::numeric
 WHERE (
   COALESCE((SELECT monthly_usd_cap FROM seo_site_budgets WHERE site_id = $1), $7::numeric)
   - COALESCE((
       SELECT SUM(cost_usd) FROM seo_ai_usage
        WHERE site_id = $1
          AND created_at >= date_trunc('month', NOW())
     ), 0)
 ) >= $6::numeric
RETURNING id;

$7 is the default cap ($50) used when the site has no budget row. The upfront remaining-budget check is done by checkBudget(), and if the budget is insufficient, it stops without calling the upstream AI fetch (the two single-page features return HTTP 402 Payment Required). recordUsage(), which runs after a successful AI call, returns { ok: false } if this INSERT returns 0 rows. At that point the cost has already been incurred, so the request is not failed; it writes a warning log. Because nothing goes into the ledger, the full cost of that call is missing from the ledger, and since the recorded total does not grow, later cap checks do not reflect it either. cap = 0 works as a shutoff setting meaning "don't spend a cent."

What a Single INSERT Statement Can Verify

The cap check is not a separate SELECT statement; it is a correlated subquery inside the INSERT's WHERE clause. Within a single statement, no application-side work can slip in between the aggregation and the write.

However, what I could verify independently was only the structure of this SQL. That two statements starting at the same time will always include each other's preceding rows in the aggregation has not been confirmed with contention tests against real Postgres. If you need a strict cap guarantee, you need to make per-site ordering explicit — locking the budget row, serializable isolation, or advisory locks.

This design also had a test environment problem. The tests ran on pg-mem (an in-memory Postgres), which did not implement date_trunc('month', NOW()), so according to the records at the time, 12 of the 18 tests failed with 500s.

12 out of 18 tests in integration-ai-cost-cap.test.ts failed
Only the 3 pure calcCost unit tests pass

At the time of the record, the only tests passing were the calcCost unit tests that never touch the database. Some tests also failed for reasons other than the 500s; what the records show is a case where PUT budget returned 400 due to an input validation issue.

According to the test records, registering date_trunc with db.public.registerFunction() cut the failures to 5. The rest were values inside the INSERT...SELECT whose types could not be resolved. node-postgres sends untyped values as strings, so I fixed it by adding explicit casts like $1::text.

The test records also include an experiment where I deliberately broke the date_trunc WHERE clause. 17 of the 18 tests still passed with the monthly extraction condition broken; only the "don't count last month" test failed. That test inserts a previous-month row dated 60 days back and confirms it is not aggregated. If someone accidentally deleted the monthly filter, this one test would be the only thing catching it. I learned that verification of a critical condition was concentrated in a single test.

April 23: Adding a Warning Below 10% Remaining

Stopping at the cap alone means users suddenly get a 402 without knowing why it stopped. So I added a soft warning that sets a flag once the remaining budget drops below 10%.

export const WARNING_REMAINING_PCT = 0.1;
function isNearLimit(cap: number, remaining: number): boolean {
  if (cap <= 0) return false;              // a kill switch is not a "warning"
  return remaining / cap < WARNING_REMAINING_PCT;
}

Explicitly excluding cap <= 0 is so that "usage is shut off" and "the cap is near" are treated as separate notifications. According to the execution records, I first wrote 4 failing tests, and after implementing the warning condition, 240 tests passed. The change was committed as feat: AI budget soft-warning flag at <10% remaining.

Separating the warning from the shutoff also lets you separate the wording in the UI and notifications. If the remaining budget is low, the user can reduce their workload or ask an admin to raise the cap. If the cap is set to 0, an admin has intentionally stopped usage. Show both as the same yellow warning, and users will misread it as "wait and it comes back."

Don't Let the Ledger End as Just a Stop Mechanism

Notifying only that the cap was reached does not tell you what to fix next. For operations, you need at least these five things.

How much was spent this month. The baseline number to compare against the monthly cap.
How much it grew today. A sudden increase can mean not just more users but also retry loops or bugs.
Which feature spent it. Without recording the endpoint, you cannot pick reduction candidates.
Unit cost per call and per deliverable. More calls means something different if the number of suggestions is growing at the same rate.
How things changed before and after a change. Record the day you changed the model or the input volume, and compare unit costs afterward.

The April seo_ai_usage table has the columns (site, model, endpoint, token counts, cost, timestamp) to answer the first three. There was no screen yet for daily growth or before/after comparisons. Still, as long as each call is recorded, you can aggregate later and compare reductions with numbers instead of gut feeling.

What This Design Has Not Verified

I cannot present this article as "a finished version that never exceeds the budget even under concurrency." Four reasons.

Contention tests against real Postgres are not done. Collapsing everything into a single SQL statement is not the same as complete per-site ordering control.
Some calls never reach the ledger. Usage is recorded only after a successful AI call, and failed calls are not recorded by design. However, in page-goal auto-generation, if reading the response fails after the AI has responded, it returns an error without recording. The cost is incurred, but it is not in the ledger.
Creating search vectors was outside the cap. The embeddings created inside the two suggestion features and the API that rebuilds the vectors had neither a cap check nor ledger records. In September 2026, I added a cap check and recording to the rebuild API, and recording to the embeddings inside the suggestions.
Behavior when the budget database is unavailable is also unverified. Keep processing and you cannot honor the cost cap; stop and you halt users' work. You have to decide which takes priority, and prepare failure-mode tests and notifications.

So this implementation is not an endpoint. It is the first stage of treating cost as something to control. Follow-up articles move on to reconciling against billing data, identifying fixed costs, and re-measuring after changes. Skip this order, and you cannot explain reduction effects with numbers.

What Came After

This cap control and ledger became the prototype for the measurement, caps, and records used in the cost improvements from May onward. The next month I investigated a problem where Cost Explorer returned $0, and later moved on to keeping "measure → fix → re-measure" on record. That way of thinking was already in place this April.

On the other hand, what this mechanism covers is AIO Helper's AI API spend. It cannot stop cloud-wide charges such as always-on Elastic Container Service or NAT Gateway fixed costs. Costs incurred outside the application need separate coverage with Cost Explorer and budget monitoring.

Lessons You Can Reuse

Estimate the unknown as "expensive." Treat a model missing from the price table as costing 0 and it slips past the cap. Make the fallback price deliberately high.
Even if you combine aggregation and write into one statement, verify the concurrency guarantee separately. If you need a strict cap, you need contention tests on real Postgres and a mechanism that guarantees ordering.
Separate the shutoff setting from the low-budget warning. Don't mix cap = 0 (stopped) into the "almost there" warning.
Record per-call costs before you read the invoice. Record them on the app side together with the endpoint and token counts, and later reduction strategies can be built on numbers.

Running SEO Improvements as One Flow, from Proposal to Verified Effect — AIO Helper

uehara — Mon, 05 Oct 2026 05:11:34 +0000

Conclusion

AIO Helper is an SEO operations SaaS we are building. It ingests search data, has AI propose page-level improvements, applies the proposals a person approves, and then checks whether each change actually worked. These four stages run on one screen and one database.

Ingest. It pulls page- and keyword-level numbers every day from Google Search Console and other sources.
Propose. AI proposes improvements per page: titles and descriptions, internal links, additional content, and more. Each proposal carries an expected impact, a confidence level, a risk, and an effort.
Apply. Approved proposals are applied to pages. Titles and descriptions are written to AIO Helper's per-page SEO settings, which the site reads through an API. For pages on our own CMS, Plovant, there is also a setting that applies low-risk proposals automatically every day.
Verify. It observes each applied change in 14-day windows and sorts the result into "improved," "no change," "worsened," "insufficient data," and so on, using rankings, clicks, and impressions.

The history of applied changes, and the lessons people record after looking at each verification, go back into the material the AI searches when it writes the next proposal. This article walks through the four stages with the actual screens and code.

Main text (about 9 min read)

Why I Built It

AIO Helper started as the SEO features inside our CMS, Plovant. Plovant was first built as an SEO platform. Later, a CMS and an SEO product began to live in the same repository, so I split the SEO features out into a separate product. That product is AIO Helper.

When I split it out, I asked what the most laborious part of SEO actually is.

Producing improvement ideas is not that hard anymore. Search Console numbers tell you which pages "get many impressions but few clicks," and if you ask an AI, it will suggest a rewritten title.

The hard part comes after that.

Which of the proposed ideas actually went onto a page?
When did it go in, and what were the rankings and clicks before the change?
After it went in, did it work, or did it make things worse?
Is it safe to roll a change that worked out to other pages?

Once you start tracking this in a spreadsheet, you quickly lose track. The place where ideas come from, the place where pages are rewritten, and the place where you look at the numbers are all separate.

So in AIO Helper, each improvement idea is held as a single record called a "proposal," and that record changes state as it is approved, applied, and verified. The point is to be able to trace, proposal by proposal, what was done to which page and how the metrics changed.

The AIO Helper Screens

The admin UI is split into four groups in the left menu.

CONTROL   Control Room
STRATEGY  Strategy Pipeline / Expansion Roadmap / Growth Structure
INSIGHTS  Keywords / Pages / Page Goals / Content / Site Structure /
          E-E-A-T / Links / Performance / Verification
SYSTEM    Reports / Settings

At the top of the screen, you can switch the viewer's role between "Operator," "Manager," and "Executive." The same data is shown differently for the person running daily changes, the person setting priorities, and the person who only wants the overall numbers.

From here, let's go through the four stages in order.

1. Ingest

The first stage is ingesting search data. The core is Google Search Console data; there are also ingestion jobs for Google Analytics 4, PageSpeed Insights, and ad data.

The daily jobs run in a fixed order. Times are in Japan Standard Time.

03:30  Ingest Search Console data
04:00  Re-embed the site's documents (preparation for the search described later)
04:10  Aggregate before/after numbers for applied changes
04:20  Crawl our own site and collect page state
04:45  Generate proposals in a batch
05:00  Verify the effect of applied changes
05:30  Apply low-risk proposals to page drafts

The order is set so that the numbers ingested that day feed that day's proposals and verifications.

What I had to be careful about in ingestion is that Search Console data takes time to finalize. If you fetch yesterday's data on the same day, it can still be zero rows. If you treat zero rows as "ingestion complete," that day's data stays missing.

So the daily ingestion re-fetches 7 days at a time. By default, that is the 7 days ending 3 days ago.

// seo-ingest-gsc/src/dates.ts
export const DEFAULT_REFETCH_DAYS = 7;

Days that were already ingested are overwritten with the finalized values. Fetching the same day again does not change the result, so repeating the re-fetch every day does not create duplicates. This "completed with zero rows and left a gap" problem is one I fixed after it actually happened. I will cover the details in another article in this series.

Ingested numbers go into the database per page, per keyword, and per day. Besides the rankings of tracked keywords, the Keywords screen shows "keywords that have search volume but where the site does not rank yet" as gaps.

2. Propose

The second stage is AI-generated improvement proposals. The "Strategy Pipeline" screen lists proposals per page.

A single proposal is data shaped like this. I took one demo-data item straight from the API response.

{
  "type": "Meta",
  "target": "/pricing",
  "locale": "ja",
  "impact": 82,
  "confidence": 91,
  "risk": "Low",
  "risk_reason": "Title and description change only. No structural change.",
  "effort": "Low",
  "status": "New",
  "evidence_tags": ["GSC", "SERP", "GA4"],
  "summary": "/pricing has no title or description set. Expected +120 clicks per month from better CTR"
}

(The risk_reason and summary values are Japanese in the actual response; they are translated here.)

The main proposal types include title and description changes (Meta), body rewrites (Content), adding a group of related pages (Cluster), internal links (InternalLink), site structure (Architecture), page speed (Performance), and brand notation (Brand).

Each comes with "impact," "confidence," "risk," and "effort." The screen can sort by impact, confidence, and risk, and filter by type, risk, and effort, so you can work through "high impact, low risk" items first. Risk also carries a reason, because rewriting a title and adding three new pages have completely different consequences when they fail.

Using Past Verification Results as Evidence for Proposals

When the AI writes a proposal, it does not start from nothing. It first pulls documents about the site with vector search (a way of finding text by closeness of meaning), and then writes the proposal. The ordering of that search is the following code.

SELECT d.id AS doc_id, d.doc_type, d.title, d.content,
       (e.embedding <=> $1::vector) AS distance
  FROM seo_embeddings e
  JOIN seo_docs d ON d.id = e.doc_id
 WHERE e.embedding IS NOT NULL
 ORDER BY
   CASE WHEN d.site_id = $2 THEN 0 ELSE 1 END,
   CASE d.doc_type
     WHEN 'brand_rules'    THEN 0
     WHEN 'learning'       THEN 1
     WHEN 'outcome_report' THEN 2
     WHEN 'change_log'     THEN 3
     ELSE 4
   END,
   e.embedding <=> $1::vector
 LIMIT 8

The ordering means this:

First, the site's own documents take priority.
Within those, brand notation rules (brand_rules) come first. A proposal that gets the company or product name wrong is unusable however much impact it promises.
Next comes learning. These are records a person attaches to a proposal after looking at its result on the verification screen: "which type of proposal went onto which page, what the result was," and what was learned from it, one by one.
After that come outcome reports (outcome_report) and the change history (change_log). The change_log is recorded automatically every time a page's SEO settings change.

In other words, changes that worked before and changes that did not are pulled near the top as evidence when the next proposal is written. This is one reason not to stop at proposing, but to follow through to verifying the effect. Without verification records, the AI would produce similar generic proposals every time.

3. Apply

The third stage is applying changes to pages.

There are two paths.

The first writes to AIO Helper's per-page SEO settings. Changing a title or description updates these settings (PUT /v1/seo). The site reads these SEO settings from the API (/v1/seo) and uses them when rendering the page. There is also a small library for that.

// packages/seo-kit/src/fetch-seo.ts (excerpt)
const url = `${SEO_API_URL}/v1/seo?siteId=${encodeURIComponent(siteId)}&path=${encodeURIComponent(path)}&locale=${encodeURIComponent(locale)}`;
const res = await fetch(url, { cache: "no-store" });

Along with the write, it does three things:

Saves the before and after as a change_log, which becomes search material for the next proposal
Sends seo.updated to registered endpoints as a signed notification (webhook)
Resubmits the sitemap to Search Console

The second path applies changes directly to pages on our CMS, Plovant. When enabled in a site's settings, low-risk proposals are applied to the Plovant page drafts automatically every day. Whether to go further and publish those drafts automatically is a separate setting.

I split automatic application into two levels, "up to the draft" and "up to publishing," because if AI proposals are published as-is and rankings drop, rolling back is painful. The first step is a state where the content can be checked in a draft, and each site decides whether to hand over publishing as well.

4. Verify

The fourth stage is verifying the effect. What gets verified are changes registered as a pair of a page and its target keyword; not every applied proposal becomes a verification target automatically. The "Verification" screen lists the result for each registered change.

Verification splits the time after publishing into 14-day windows and compares each with the period before the change, looking at how the target keyword's ranking and the page's clicks moved. Observation normally runs until 56 days after publishing. The thresholds are:

// rank-watch-judge.ts
export const DEFAULT_JUDGE_CONFIG: JudgeConfig = {
  minImpressions: 50,        // below this many impressions in the period, draw no conclusion
  improvedDelta: 0.5,        // ranking better by at least this: "improved"
  worsenedRankDelta: 2.0,    // ranking worse by at least this: "worsened"
  clickGuardRatio: 0.85,     // page clicks below 85% of before: "worsened"
  clickGuardMinBaseline: 10, // fewer than 10 clicks before: don't use clicks
  imprGuardRatio: 0.7,       // page impressions below 70% of before: "worsened"
  imprGuardMinBaseline: 100, // fewer than 100 impressions before: don't use impressions
};

What I cared about most here is keeping "insufficient data" as a result of verification.

Search numbers fluctuate to begin with. If a keyword gets only a handful of impressions and its ranking rises by one, you cannot tell whether that is the effect of the change or chance. So if the target keyword does not reach 50 impressions within the period, it says neither "improved" nor "worsened" and only records that the data is insufficient.

Also, even if the target keyword's ranking goes up, the result is "worsened" if the page's total clicks drop sharply. This avoids looking only at one keyword and missing that the page as a whole is losing.

A person who looks at the result can attach a learning record to that proposal. As described in the previous section, this record is searched as evidence for the next proposal. Changes that came out "worsened" appear under "Next actions" on the screen as items to consider rolling back.

Limits of This Design

AIO Helper's design has clear limits.

Pages with few impressions often get no conclusion. The numbers needed for verification don't come together, so new pages and pages with little search traffic stay at "insufficient data." Observation can end as "insufficient data" before the data comes together. Checking that change again later requires registering it again.
Ranking changes cannot be attributed to the change alone. Rankings also move because of search-engine updates or changes in competing pages. Because the method compares before and after, other changes that happened in the same period cannot be separated out.
AI costs money. Every proposal calls a generative AI API and a vector-search API, so cost grows with use. That is why each site has a monthly cap, and requests whose estimated cost exceeds the remaining budget are stopped before the AI is called. Because the check uses an estimate, it cannot prevent overruns when the actual cost exceeds the estimate or when concurrent requests arrive together. I will explain this mechanism in detail in the next article.

What This Series Will Cover

The AIO Helper articles start from this introduction and go on to explain each feature and the decisions made while building it. Planned topics:

Stopping AI spend inside the app before the invoice arrives
Search Console's delayed finalization, and yesterday's data coming back as zero rows
Moving a rarely called API from an always-on server to a setup that runs only when needed
Viewing the language versions of the same page as one, on a multilingual site
The AI no longer producing proposals, and preventing it in both the prompt and the database
Why vector search should be tested against real Postgres

Summary

The laborious part of SEO improvement is not producing ideas but continuing to track what went onto which page and what happened afterward.
AIO Helper holds each improvement idea as a single "proposal" record and moves it through four stages: ingest, propose, apply, and verify.
Verification keeps "insufficient data" as a result too, and says neither "improved" nor "worsened" when impressions are insufficient.
The change history and the lessons recorded from verification results go back into the search material for the next proposal, so the improvement record itself becomes the evidence for the next improvement.

Save your main context to raise quality and cut cost — building Context Drop

uehara — Sun, 04 Oct 2026 05:40:57 +0000

1. Hook + thesis

At some point, Claude Code started to "break" on me.

To be precise, what broke wasn't Claude Code itself. What broke was the way I was using it. Firing off dozens of agents at once, pasting giant logs and screenshots straight into the conversation, keeping long sessions alive by stitching them together with /compact — the heavy, fan-out-centric style that Claude Code's ultracode opt-in naturally produces. Keep that up and, one day, tool calls start leaking into the conversation as raw text, or the model goes quiet after thinking and never answers at all.

If I had to name the one factor most consistently present, it's context bloat. The bigger the conversation grows, the more every turn costs to resend, the closer you drift to the conditions that trigger corruption, and the less stable the responses become.

Context Drop is a small desktop tool I built to cut exactly one habit at the root: pouring raw context directly into the main conversation. Screenshots, logs, JSON — instead of landing in the main conversation, they're read by an isolated subagent (a worker AI that runs with its own conversation history, separate from the main one), and all that comes back is a compact result. The main context stays light.

This post first lays the groundwork — why heavy usage breaks things and how to choose between model, effort, and ultracode — and then explains what Context Drop is, how to use it, and how to install it, step by step.

2. Background 1 — why it broke (ultracode and context bloat)

What "ultracode" actually is

Let me get the term straight first. ultracode is a Claude Code opt-in. You turn it on by putting the keyword ultracode in a prompt, or by switching it on for the whole session. While it's on, the model orchestrates multi-agent Workflows for every substantive task, and it treats token cost as not a constraint (tokens are the units an AI uses to count its input and output, and what you are billed by).

That second part is the important one. A model that is told cost doesn't matter will, quite reasonably, fan out wide: split the work across many agents, then add more agents to verify and refute the first ones. Our internal record has a good example: a 77-agent adversarial review in one release, which the record describes as launched at the author's discretion — that is, with ultracode on — rather than mandated by any rule .

So ultracode is not an always-on mode defined by our internal design records. It's a Claude Code switch that, by design, produces huge fan-outs — and with them, a huge amount of context and tokens. That accumulated context was the common factor in the failures below (a correlation, as I note later).

Symptom 1 — tool calls mutating into "count / court"

The nastiest one was the tool-call corruption on Opus 4.8. A tool call that should have become a structured tool_use instead leaks into the conversation as raw text. The internal control tag mutates into real English words like count / court / call, and the parser gives up with "tool call could not be parsed" .

The firing conditions are telling — Opus 4.8/4.7, a very long context (1M sessions), right after /compact, non-ASCII (especially Japanese) arguments, three or more MCP servers, long tool arguments, several tools invoked at once . In short, the heavier, longer, more multilingual, and more packed the state, the more likely it is.

Worse, once the history is broken it reinforces itself. Presumably the model imitates the broken output; either way, retrying in the same session only reinforces it — it didn't self-heal in our cases. The fix is not to "go forward" (--continue) but to "go back": /rewind, then /compact, and ultimately /clear .

To be honest about the evidence: the link "context bloat → corruption" is recorded at the level of correlation, not proven causation, and hooks (handlers that run automatically at set moments) cannot run /compact for you automatically. But the correlation was strong enough to change how I work.

Symptom 2 — re-inflation right after compact, silence after a marathon

/compact is the feature that compresses a conversation to make it lighter. But we had a mechanism that injected a 19–32 KB digest at session start — and it also fired right after /compact. The result: the conversation snapped back toward its original size immediately after the compact, cancelling the shrink and re-creating the very firing condition above .

Its other face is total silence after a long ("marathon") session. After a marathon session, the model can go completely silent. Throw the same request at a fresh session and it answers instantly — the only difference being the context that had piled up in the old session .

"Just run everything at the maximum setting" didn't work

At this point you might think: if it's hard, why not just use Claude Code at its top settings? If a bigger model, higher effort, and more agents through ultracode make it smarter, then as long as money doesn't matter, you could run everything at the smartest setting from the start.

We actually ran something close to that. And that is exactly where the problems came from. Look again at the trigger conditions for the breakage above: the top Opus model, a long 1M context, a multi-agent fan-out. Every one of them is "the maximum setting." The harder we ran at maximum, the more the conversation bloated, and in that bloated conversation, tool calls broke and responses stopped.

So before it was a cost problem, it was a quality problem. The evidence for causation stops at correlation, but our judgment is that spending money to pile up context was itself what lowered quality. We thought we were buying quality with money; in fact we were spending money to lower it. The answer was not "use an even higher setting" but "decide what you keep out of the main conversation."

Why a heavy context also burns money

It isn't only corruption. Context bloat hits cost, too. Every turn, Claude Code re-sends and re-bills the entire conversation so far plus all tool results. By the estimate on record, reading a 20k-token log once means paying for about 85k tokens over the next 30 turns. A 200-token digest costs about 0.9k over the same span — roughly a 100x difference. And large reads bring compaction sooner, nudging you back toward the corruption risk above .

As an extreme example: one blog article processed under full heavy-gate fan-out consumed about 800k tokens including subagents, with 289k in the parent context .

The crux — "delegation ≠ saving"

Here's one counterintuitive fact worth nailing down. "Just offload heavy reads to a subagent and it gets cheaper" — that's only half true.

A subagent's reads are billed too. Isolation protects the parent context, but it does not make the work free. Delegation moves where the tokens land; it isn't a reduction .

So what is the value of delegation? Not polluting the parent conversation. Keep the raw bulk data out of the parent by isolating it, and the parent context stays light, stays away from the corruption trigger conditions, and doesn't inflate the resend cost of every subsequent turn. That idea — "isolate to protect the parent" — became Context Drop's design philosophy directly.

3. Background 2 — model, effort, and ultracode: what for, when which, how to switch

The countermeasures against corruption and bloat boil down to two levers: a context diet (don't over-read, don't over-paste) and model-tier routing — choosing how much model you spend on each piece of work . Context Drop is a tool for the former, but knowing the latter stabilizes your operation, so let me lay it out here.

In one line

Effort is "how deeply one agent thinks." ultracode is "how many agents split up the work and verify it." The two are independent — you can turn each one up or down on its own.

The three knobs

knob	what it decides	values	my current setting
model	the ceiling on how smart the model is	Opus / Sonnet / Haiku / Fable	`opus[1m]`
effort	how much it thinks in one response (depth)	`low` / `medium` / `high` / `xhigh` / `max`	`medium` (`effortLevel` in `~/.claude/settings.json`)
ultracode	whether to parallelize and cross-check across multiple agents (breadth, certainty)	on / off	on only for the turns that need it, via the keyword

What each effort level means

My rough sense of each level (a rule of thumb, not documented behavior):

effort	way of thinking (my rough sense)	suited for (my rough sense)
`low`	shallowest reasoning	wording fixes, summarizing grep results, mechanical replacements
`medium`	standard	everyday implementation and investigation (my current default)
`high`	works through the steps and alternatives carefully	changes that touch a boundary (billing, auth, DB), hard bugs
`xhigh`	thinks quite deeply	design decisions, reasoning about race conditions and concurrency
`max`	the deepest setting	a root cause that keeps coming back no matter how often you fix it, the final call on an important design

A caveat that matters: the higher the effort, the slower the answer and the more tokens it uses. And it only improves accuracy for problems that can be solved by thinking. If the real cause is missing information, raising effort won't fix it — what helps then is widening the investigation (which is where breadth, i.e. ultracode, comes in).

Combination patterns

pattern	effort	ultracode	when to use it
A. Everyday	`medium`	off	normal implementation, questions, small fixes
B. Think it through alone	`high`–`max`	off	a hard bug whose cause is probably in one place; design thought experiments. Cases where thinking deeper beats parallelizing
C. Wide and shallow	`medium`	on	mapping the blast radius of a change, migrating many files, exhaustively checking "is there any X?". Each item is easy, but there are many of them
D. All out	`high`–`xhigh`	on	adversarial review before merge (billing/auth), production incidents, comparing design approaches
E. Mixed (inside a Workflow)	set per stage	on	e.g. the discovery stage at `low`, the verification/judging stage at `xhigh`. In my experience, the easiest way to balance cost and accuracy

A note on E. Inside a Workflow script, each agent can override effort:

agent(prompt, { effort: "low" })   // 'low' | 'medium' | 'high' | 'xhigh' | 'max'

So the efficient split is: stages with many simple items run shallow; the few stages that make critical judgments run deep. When ultracode is on, this is the split I want the workflows built with. It's also the cheapest brake on the problem from §2 — ultracode, left alone, fans out hugely.

Which model — the tier rules

The model knob has its own decision rules. Summarizing our internal operating rules :

tier	model	suited for
light	Haiku	mechanical work (bulk replace, rename, index updates, small docs)
mid	Sonnet	standard implementation, bug fixes, test writing, QA
top	Opus	design, review, adversarial verification, skeletons of large specs
max	inherited from the parent session (e.g. Fable 5)	risk-floor calls, task-decomposition decisions, design-level judgment

And the rules for picking one, so you don't dither:

Risk floor: if billing, auth, production, or anything irreversible is involved, use the top tier plus human confirmation.
Size: small and mechanical → light; multiple files or new design → top for the skeleton.
Task type: implementation, bug fixes, tests → mid; review, adversarial verification, design → top.
When in doubt, go higher: the loss from a quality incident outweighs the token savings.

Model tier and effort are two different axes: the tier sets the ceiling, effort sets how much of it you use per response. A boundary change (billing/auth) usually justifies both a top tier and high effort; a bulk rename deserves neither.

How to switch

Effort

At launch: claude --effort high — "Effort level for the current session" (confirmed via claude --help; values low / medium / high / xhigh / max).
During a session: run /effort and set the level you want. Verified in this project — the run that set medium printed Set effort level to medium (saved as your default for new sessions). Note that it also saves the level as your default for new sessions.
Standing default: change "effortLevel": "medium" in ~/.claude/settings.json.
Inside a Workflow: override per agent with agent(prompt, {effort: ...}), as above.

Model

At launch: claude --model … (listed in claude --help).
During a session: /model.
Standing default: the model field in ~/.claude/settings.json (e.g. opus[1m] = Opus with the 1M-token context window).
Speed: /fast toggles Opus fast mode — faster output from the same Opus model, not a drop to a smaller one.

ultracode: put the keyword ultracode in the prompt for the turn that needs it, or switch it on for the session.

Note: /effort, --effort, effortLevel, /model, opus[1m], /fast, and ultracode are features of the Claude Code harness, not something the internal design records cited here invented. What those records define is the decision rules for "which tier, when."

A naming trap

The high in /code-review high (and its siblings low … max) is not the session's reasoning effort. It controls how broadly the review picks findings. The names look alike; they are different things. Likewise, /code-review ultra is a paid, deep multi-agent review that runs in the cloud when you trigger it — a separate mechanism from ultracode.

Recommended operation

Leave the default at A (medium / ultracode off) and switch deliberately:

You want one hard point thought through → raise effort only (B).
Nothing may be missed → add ultracode (C).
It absolutely cannot fail — before a merge, during a production incident → raise both (D).

Matching model/effort/breadth to the work is one habit; keeping the context light is the other. Only when both are in place does heavy usage stop being so fragile. Context Drop builds the latter into the tool and its workflow.

4. What Context Drop is

Context Drop is a local-only desktop menu-bar app: no telemetry, and Context Drop itself sends nothing over the network. It does one thing — keep raw context out of the main conversation and divert it to an isolated subagent.

The core flow is this:

raw context  →  Context Drop packet  →  isolated subagent  →  compact result  →  main conversation

And here is the flow it deliberately does not support:

raw context  →  main conversation  →  subagent      ← forbidden

The difference is whether raw data ever passes through the main conversation, even once. The former never pollutes the main. The latter etches raw data into the main, re-bills it on every subsequent turn, and raises the corruption risk. Context Drop draws that line by giving the main conversation only metadata such as file locations, while the plugin's instructions hand the reading of the contents to a subagent.

Another pillar of the design is "capture first, route later." You don't decide the destination while gathering material. The destination is decided when you run /context-drop:pull (or its short alias /cd) in the Claude Code session you want to hand it to — that session claims the packet. So you never agonize over "which session of which project do I hand this to."

5. How to use it

It's basically three steps.

Collect: press Start Capture in the app (global shortcut Cmd+Shift+9). Only items you copy after pressing Start are collected. You can also drag & drop files onto the window at any time. Stop pauses capture and keeps what you have; Clear empties the packet.
Claim: type /context-drop:pull <instruction> in the Claude Code tab you want to hand it to (or /cd <instruction> if you installed the optional /cd short alias — see §6). It infers ANALYZE (investigate only) / FIX (fix it too) / REVIEW from your instruction.
Receive: the isolated subagent reads the packet's contents (screenshots, logs, JSON, files) in its own context, and all that comes back is a compact result.

Here's how much that helps, measured in a real /cd run in this project. The packet held 5 items: two PNG screenshots (163,772 bytes and 173,585 bytes) and three short text files (184, 487 and 87 bytes). The isolated subagent spent 19,365 tokens reading them. The main conversation received only a compact inventory — a few hundred tokens. The raw bytes never entered the main conversation.

Had I pasted those two screenshots straight into the main, that weight would have been etched into the main context and re-sent on every subsequent turn. Offloading it to a disposable subagent while the main stays light — that's the entire value of Context Drop. (And per "delegation ≠ saving," those 19,365 tokens are still billed; what you save is the main context, and every future turn's resend.)

6. Installation (with screen captures)

Here I'll go slowly. Context Drop is built with Tauri (a framework for building desktop apps with Rust and a web frontend) and is designed for both macOS and Windows. For now, though, the only prebuilt download (.dmg) is for macOS on Apple Silicon (arm64). The steps below are for that build. If you're on an Intel Mac or Windows, see "Build from source" right after Step 1.

Step 1 — Download

Open the GitHub Releases page (https://github.com/EarthLinkNetwork/context-drop/releases) and download the .dmg from the newest release (the one marked Latest). The screens and steps in this article were checked on v0.1.3 (Context-Drop-v0.1.3-macos-arm64.dmg, about 6.3 MB). As of October 4, 2026, the latest version is v0.1.5, and depending on when you read this a newer version has probably been released, so always take the latest one. Read the file names and version numbers below as the version you downloaded. The .dmg is signed with an Apple Developer ID certificate and notarized by Apple (ticket stapled), so macOS won't block it as coming from an unidentified developer. On first launch macOS may still ask you to confirm opening an app downloaded from the internet — click Open.

Fig 1: The GitHub Releases page for v0.1.3. Click Context-Drop-v0.1.3-macos-arm64.dmg to download.

(Intel Mac / Windows) Build from source

Context Drop is open source (MIT license), so where there's no prebuilt download you can build it yourself. You need three things:

Rust (stable, installed with rustup)
Node.js and pnpm
Your OS's WebView (on Windows, the Microsoft Edge WebView2 Runtime)

The build commands:

git clone https://github.com/EarthLinkNetwork/context-drop.git
cd context-drop
pnpm install
pnpm tauri build

The resulting installer or app is written under apps/desktop/src-tauri/target/release/bundle/ (the desktop app is a standalone Cargo workspace, so it is not the repo-root target/). CI (GitHub Actions, windows-latest) checks on every push that the app compiles and links on Windows. To be honest, though, we have not yet launched and used the resulting Windows build on a real machine. If it doesn't work for you, please tell us through a GitHub issue or pull request.

Step 2 — Drag to Applications

Double-click the downloaded .dmg (e.g. Context-Drop-v0.1.3-macos-arm64.dmg) to open it, and Context Drop.app appears next to a shortcut to the Applications folder. Drag the app into Applications to copy it.

Fig 2: The opened .dmg. Drag Context Drop.app into the Applications folder.

Step 3 — First launch

Launch Context Drop from Applications. If macOS asks whether to open an app downloaded from the internet, click Open — that prompt is normal for a notarized app. It's a menu-bar app, so its icon appears in the menu bar at the top of the screen; click it to open the window. Because the Claude Code integration isn't set up yet, a "Finish setup" banner with an Open Setup button appears above the tabs. Launching the app also installs (or refreshes) the bundled context-drop CLI in its canonical location automatically.

Fig 3: First launch. The "Finish setup" banner and its Open Setup button above the tabs (plugin not yet installed). (v0.1.3 UI rendered with sample state)

Step 4 — Open "Claude Code Setup" in Settings

Click Open Setup (or the Settings tab) and look at the "Claude Code Setup" section. Step 1 (enable the plugin) shows the two /plugin … commands with Copy buttons; Step 2 explains capture and routing. Below them is Optional / offline install.

Fig 4: "Claude Code Setup" on the Settings tab. Step 1 with the two copyable commands, Step 2, and Optional / offline install. (v0.1.3 UI rendered with sample state)

Steps 5 and 6 happen in Claude Code and are shown as text rather than figures.

Step 5 — Add the marketplace in Claude Code

In a Claude Code session, type the following. This is the public route — you name the GitHub repository directly:

/plugin marketplace add EarthLinkNetwork/context-drop

Expected output (observed with the same /plugin marketplace add command):

Successfully added marketplace: context-drop

Step 6 — Install the plugin and reload

Then install it:

/plugin install context-drop@context-drop

Observed output (with a check mark):

Installed context-drop. Run /reload-plugins to apply.

Then apply it:

/reload-plugins

This prints a line beginning with Reloaded:.

Important: already-open sessions don't see the new commands yet. A Claude Code session that was open before the install does not get /context-drop:pull until you run /reload-plugins in it, or open a new session. (/cd additionally needs the short alias from Step 6a.) I hit this in practice: a long-running session treated /cd ... as plain text and just answered it as a message.

Offline path. Without internet access, use Settings → Claude Code Setup → Optional / offline install → Install locally to stage the marketplace locally, then pass the printed local path to /plugin marketplace add <path>. Either way — public or local — you still need the desktop app running, since it handles capture and the CLI.

Step 6a — (Optional) Install the `/cd` short alias

The plugin itself provides /context-drop:pull; the short /cd is a separate alias. To get it, open Settings → Claude Code Setup → Optional / offline install and click Install /cd short alias. If you skip this, use /context-drop:pull everywhere this post shows /cd.

Step 7 — Confirm the install

The "Finish setup" banner disappears, and Settings → Claude Code Setup now shows the "Plugin installed in Claude Code" badge. The app detects this by reading Claude Code's own install record (installed_plugins.json) — once you see it, you're ready.

Fig 5: The Settings tab showing the "Plugin installed in Claude Code" badge. (v0.1.3 UI rendered with sample state)

Step 8 — Pile up material

Press Start Capture on the Capture tab (or Cmd+Shift+9), then copy what you need — or drag & drop files onto the window. Items pile up in Current Packet, with a text preview for text and a thumbnail for images, so you can tell at a glance what you put in. Click an item to view its content (text is shown up to the first 1 MiB); the trash icon removes a single item. The Last Dispatch line shows when a packet was last handed off.

Fig 6: Current Packet on the Capture tab — text previews, an image thumbnail, and Last Dispatch. (v0.1.3 UI rendered with sample state)

Step 9 — Claim with `/context-drop:pull` (or `/cd`)

In the tab you want to hand it to, type /context-drop:pull <instruction> — this always works once the plugin is installed. If you installed the short alias in Step 6a, /cd <instruction> does the same thing. For example:

/context-drop:pull Investigate and fix the problem in this UI
/cd Investigate and fix the problem in this UI

What happens next is all text, so here it is in words rather than a screenshot. The isolated subagent opens the packet in its own context and reads everything. In the measured run from §5 (a separate run, not the example instruction above) — a 5-item packet of two screenshots (163,772 and 173,585 bytes) plus three short texts — the subagent used 19,365 tokens, and what came back to the main conversation was only a compact inventory (a few hundred tokens). None of the raw bytes ever entered the main conversation.

7. The crux of the design / wrap-up

What Context Drop does is, technically, dead simple — keep raw context out of the main conversation, have an isolated subagent read it, and return only a summary.

But that one simple line addresses the three problems that tormented me in my heavy ultracode days. Corruption (if a huge context and post-compact states are the trigger, don't let the main bloat), bloat (cut off, at the root, the raw data that gets re-sent every turn), and cost (confine heavy reads to a disposable subagent, so they're paid once instead of on every turn — but since delegation isn't saving, also cut how much you have it read).

It doesn't make the model smarter, nor does it raise the effort or the agent count. It builds one thing into the tool and its workflow: keep the main context light. The heavier your usage, the more that single point is worth.

8. It's open source

Context Drop is published as MIT-licensed open source at https://github.com/EarthLinkNetwork/context-drop. There is still plenty to do: prebuilt Intel Mac and Windows builds, integrations with other AI tools, and smoothing out rough edges. If you have an idea that would make it more useful, or a feature you want, please send a pull request. We're looking forward to it.

Claude Code Cloud Sessions Come With Up to $250 in Free Credit — Claim by October 7, 11:59 PM PT

uehara — Sat, 03 Oct 2026 13:58:47 +0000

TL;DR

If you subscribe to Claude Code on Pro or Max, you can get free credit that works only for cloud sessions. It's $250 on Max and $100 on Pro, and it's a one-time offer.

Claim deadline: October 7, 2026, 11:59 PM PT
Spend deadline: November 4, 2026, 11:59 PM PT. Any unused credit disappears
Who's eligible: Individuals who had an active Pro or Max subscription when the promotion started, September 23, 2026 at 2:00 PM PDT. Team / Enterprise, free trials, and accounts with overdue payments are not eligible
What it covers: Cloud sessions only. Anthropic's help page lists these as not covered: "Projects and Routines," "Remote Control sessions," "chat and Cowork sessions," and "all Claude usage other than cloud sessions." It also can't be used for your everyday Claude Code running on your own computer
What you need first: A GitHub connection

You claim it from "Claim credit" in the claude.ai settings (your name in the bottom left → Settings → Usage), and it takes a few minutes. The help page says some accounts may be granted the credit automatically, but if yours hasn't been, you won't get it unless you claim it. Check your Usage page and at least get the claim done now.

I wrote this article quickly, with the goal of helping you get the credit before the deadline passes. So it focuses on how to claim it. I'll cover how to use cloud sessions in more detail in a separate article.

In follow-up articles, I plan to post measured credit usage: what kinds of work I give cloud sessions, and how much credit each one uses. I'll also cover concrete usage and costs — what happens when you do what kind of work.

So you don't miss them, following me or liking/bookmarking this article should make it easier to get the updates.

Details (about 6 min read)

What is a cloud session?

A cloud session runs Claude Code not on your own computer but on a virtual machine in the cloud, managed by Anthropic.

Normally, Claude Code reads and writes files on the computer in front of you. In a cloud session, it copies (clones) your GitHub repository to the cloud, works on a dedicated branch, and when it's done, hands the result back as a pull request (a request to merge the changes).

The handy part is that the work doesn't stop when you close your laptop. You can ask for a long refactor, then head out or go to sleep, and when you come back the pull request is ready. You can also check progress from a browser or from the Claude app on your phone.

On September 23, 2026, cloud sessions moved from research preview to general availability. The credit promotion started the same day.

It showed up on my Max (20x) account too

I use Claude Code on Max (20x). When I opened the Usage screen in the claude.ai settings, there was a banner at the top saying I could claim a $250 bonus credit.

The same screen also showed a warning about my weekly usage limit saying that, at my current pace, I would use it all up tonight, before the next reset. Even on Max, heavy use hits the weekly limit.

This is where the credit helps. The banner says "on top of your plan limits" — in other words, it's an extra allowance added on top of your plan's usage limits. While a cloud session is running, this credit is used first. Once it's used up or expires, usage goes back to your regular plan allowance. One exception: on Pro, using Fable requires pay-as-you-go usage credits to be turned on with a balance.

How to claim it

There are four ways to claim. The desktop app / IDE banner, the terminal command, and the claim link come from Anthropic's help page; the settings banner is what I saw on my own screen:

claude.ai settings: The banner on the Usage screen
Desktop app or IDE: A banner shown on screen
Claude Code in the terminal: The /claim-credit command
Claim link: The link in the announcement

Here, I'll walk through claiming from the claude.ai screen, step by step.

First, open claude.ai and click your name in the bottom left. A menu opens; choose Settings.

When the settings screen opens, choose Usage from the menu on the left (clicking Usage directly in the menu takes you to the same screen). This screen shows your current session and weekly usage.

If you haven't claimed yet, the "Claim a $250 bonus credit for cloud sessions" banner from earlier appears at the very top of this Usage screen (the image above was taken after I had already claimed, so the banner is gone). Click Claim credit there.

Just opening claude.ai/code (the web version of Claude Code) may only bring up the usual Claude Code screen, without the claim screen. If you're unsure, going in through Usage in the settings is the reliable route.

After you click Claim credit, this screen appears.

Claim the credit: Click "Claim credit." You need to agree to the Promotional Credit Offer Terms
Connect GitHub and choose repositories: Sign in to GitHub and decide which repositories the Claude GitHub App (the GitHub app Claude uses to read and write your repositories) can access
Start a cloud session: Once you give it its first task, the credit starts being used automatically

If you've already connected GitHub, steps 2 and 3 are done right away.

If you haven't connected GitHub yet, it goes like this.

First, the onboarding for the web version of Claude Code opens. Check that it says "Your $250 credit for cloud sessions is waiting" at the top, then click Continue with GitHub.

If you use the gh command (GitHub's official CLI) in your terminal, you can also choose "Connect from your terminal" on the same screen and connect by running /web-setup in the terminal.

Clicking Continue with GitHub opens the GitHub sign-in screen. Log in with your usual GitHub account.

After you log in, you'll see a screen for choosing which repositories the Claude GitHub App may access. Rather than allowing all repositories, it's safer to pick only the ones you want cloud sessions to work on.

When you're done choosing, it shows "Connected to GitHub." I connected two GitHub accounts, so mine says "Linked 2 GitHub accounts" (account and organization names are hidden). Click Continue to move on.

Once connected, you'll see the next screen.

On the Usage screen, the cloud session credit appears on its own line, separate from your regular usage allowance.

The in-app screen shows the deadlines in your local time zone. I'm in Japan, so my claim screen showed the claim deadline as "3:59 PM GMT+9, October 8." That's the same moment as October 7, 11:59 PM PT, which is the date you'll see in news reports.

The spend deadline works the same way. My screen showed "4:59 PM GMT+9, November 5," which is the same moment as November 4, 11:59 PM PST (US daylight saving time ends on November 1). That's why some reports say November 4 and others November 5. Check what your own screen shows in your local time.

What it can and can't be used for

It works only for cloud sessions. There are several places to start one:

Browser: claude.ai/code
Phone: The Code tab in the Claude app
Desktop app: Choose Cloud instead of Local when starting a session
Terminal: The claude --cloud command

Here's what it looks like from the terminal. This example is from the official docs.

claude --cloud "Fix the authentication bug in src/auth/login.ts"

One thing to watch out for: what --cloud clones is your current branch on GitHub, not your local folder. Changes you've only committed locally and haven't pushed yet won't reach the cloud. Push first, then ask. (In some cases, such as when the Claude GitHub App isn't installed on the repository, Claude Code instead bundles and uploads your local repository.)

You can also go the other way and pull work that progressed in the cloud back to your machine.

claude --teleport

It shows a list of sessions; pick one and its branch and conversation history move to your local terminal. Sending a local in-progress session to the cloud can't be done from the terminal (the desktop app has a "Continue in" menu for that).

What it can't be used for is clearly defined, too.

Local Claude Code, chat, Cowork, Remote Control: Not covered by this credit. The help page says "all Claude usage other than cloud sessions" is excluded
Projects and Routines: The screen also said "Not eligible for Projects and Routines." Scheduled Routines do run as cloud sessions, but this credit can't be used for them
Team / Enterprise plans, and new subscriptions after the promotion started (September 23, 2:00 PM PDT): The help page lists these as not eligible. The screen also said "Existing Pro and Max subscribers only"
Setups that go through other providers, such as Amazon Bedrock: Cloud sessions themselves aren't available

The cloud environment: what to know before you ask

Credit tends to get wasted not on the work itself, but when you don't notice that the cloud environment differs from your local one and burn time redoing setup. The official docs spell out the differences from your local machine quite clearly.

First, the cloud virtual machine is a fresh Ubuntu 24.04 (x86_64) every time. It doesn't matter whether you're on a Mac or Windows locally. Commonly used languages and tools come preinstalled.

Python: Python 3 with pip, poetry, uv, pytest, ruff, and more
Node.js: 20, 21, and 22 (22 by default) with npm, yarn, and pnpm
Other languages: Ruby, PHP, Java, Go, Rust, C/C++
Docker and databases: docker and docker compose, PostgreSQL 16, Redis 7.0
Handy tools: git, gh, jq, ripgrep, and more

Next, settings that exist only on your machine don't come along to the cloud. This is the easiest place to trip up.

What comes along: Anything committed to the repository — the repo's CLAUDE.md, the skills, agents, commands, and rules under .claude/, .mcp.json, and so on
What doesn't: ~/.claude/CLAUDE.md in your home directory and your personal skills, plugins enabled in your user settings, and MCP servers (connections that let Claude use external services) you added locally with claude mcp add
What doesn't work: Authentication that requires a browser login, such as AWS SSO

If, like me, you've written a lot of workflow rules in your home CLAUDE.md, be careful. If you want the same behavior in the cloud, you need to commit those rules to the repository.

For missing tools, write them into the "Setup script" in the environment settings, and they'll be installed automatically when a session starts. Here's the official example:

#!/bin/bash
apt update && apt install -y shellcheck

If this script finishes within about 5 minutes, the installed state is saved, and later sessions start from that state. You don't have to wait for installs every time.

The default network setting is "Trusted": it can reach domains on the official allowlist, such as package registries like npm and PyPI and GitHub, but no other sites. If you need to reach something like an internal API, add the allowed domains in the environment settings.

How to get the most out of it

I couldn't find, in the official material I checked, how many hours of work $250 buys. I haven't measured the consumption yet either, so I can't say for sure.

Still, given how it works, the good uses are clear.

Hand off long tasks and walk away: Dependency updates with tests, larger refactors, migrations, and so on. They keep going even if you close your laptop
Try things in parallel: Run claude --cloud three times and three separate sessions run at once. Your local machine is still just one
An outlet when you're near the weekly limit: As with the warning I mentioned earlier, it's exactly when your local usage is near the limit that it pays to shift work to cloud sessions, which run on this separate credit
Automatic PR fixes (Auto-fix): A feature where Claude automatically fixes CI failures and review comments and pushes the fix. The Claude GitHub App needs to be installed on the repository

The official docs also introduce an efficient workflow: "plan locally, execute in the cloud." Start your local Claude Code in plan mode (a mode where it only reads files and makes a plan, without changing code), and work out the approach.

claude --permission-mode plan

Save the finished plan to a file, commit and push it, and then ask the cloud to just execute it.

claude --cloud "Execute the migration plan in docs/migration-plan.md"

Do the planning locally, where a person makes the calls. Hand the time-consuming execution off to the cloud and walk away. Splitting the roles this way lets you focus the credit on "work with long wait times."

If you want to add instructions while it's running, you can send a quick message from the terminal, even from a different computer.

claude -p "Once the tests pass, add the changes to the README too" --cloud <session-ID>

It exits right after sending, without waiting for a reply. You can find the session ID in the list on claude.ai/code.

The spend deadline is November 4 (PT). That's only a little over a month, so once you've claimed, I think it's a good idea to try one task early — something that would normally take you half a day locally.

Things to watch out for

Claim it if it isn't there: Some accounts get it automatically, but if yours didn't, you won't get it unless you claim it. After October 7, 11:59 PM PT, you can no longer claim it
Unused credit disappears: It expires on November 4, 11:59 PM PT, with no rollover
GitHub is required: Cloning the repository and creating pull requests require GitHub. For GitLab and others, there's a way to bundle and send your local repository (under 100MB), but the results can't be pushed back
Idle virtual machines get reclaimed: If nothing happens for a while, the cloud virtual machine stops. The conversation history remains and you can reopen it to continue, but any process that was running is not restored
Keep GitHub permissions narrow: It's safer to choose only the repositories you want the Claude GitHub App to touch. There was also a note that some features, such as automatic code review, aren't available on repositories you don't own
Check the contents before making a session public: Session sharing on Pro / Max has two options, "Private" or "Public," and Public means anyone logged in to claude.ai can see it. Code from private repositories may remain in the conversation, so check the contents before sharing
Don't put passwords or API keys in environment variables: Variables set in an environment and the setup script can be read by anyone who uses that environment. On Pro / Max, if you register external service API keys as "API credentials," they're used in a way that isn't visible from inside the session
Don't combine Auto-fix with comment-triggered automation: Auto-fix may reply to review comments under your GitHub account name. The official docs warn that in repositories where comments on pull requests trigger things like deployments, it could set them off unintentionally

Summary

Claude Code Pro / Max subscribers can get a one-time credit just for cloud sessions (Max $250, Pro $100)
The claim deadline is October 7, 11:59 PM PT, and the spend deadline is November 4, 11:59 PM PT. The in-app screen shows these in your local time zone, so check your own
It's an allowance separate from your plan's usage limits, so it helps when you're near the weekly limit, or when you want to hand off long or parallel tasks
Get the claim done first, then before the deadline, try handing off at least one task you can walk away from

So you don't miss them, following me or liking/bookmarking this article should make it easier to get the updates.

References: Anthropic Help Center, "Cloud sessions bonus credit promotion" (support.claude.com/en/articles/17152539); Claude Code official docs, "Use Claude Code in the cloud" (code.claude.com/docs/en/claude-code-on-the-web); AI Tools Review, "Claude Code $250 Credit: Cloud Sessions Guide (October 2026)" (aitoolsreview.co.uk)

A Working Beta in 72 Hours — A Record of How Tessvia Was Born

uehara — Fri, 02 Oct 2026 04:32:23 +0000

The short version

This article is a record of the first three days, during which the name of a product called Tessvia was born. What kind of service Tessvia is, I explain in the series' entry-point article, "What Tessvia Is". Here, I write about how the first version of that Tessvia came to be — the three days in which, after handing over the spec all at once, we arrived at a working beta and a product name in 72 hours.

On June 10, 2026, I handed the spec for the Minimum Viable Product (MVP — the smallest usable version of a product) to Claude Code, an AI coding environment, all at once. That same day, we had an 11-package monorepo built on Turborepo (a mechanism for handling multiple packages together in a single repository), a Chrome extension, and a command-line interface (CLI — a screen you operate by typing text). All 15 CLI smoke tests (minimal checks that it works) passed. The next day we established the E2E (End-to-End — testing that reproduces a user's full sequence of operations from start to finish) method for the browser extension, and on the third day we looked up domain names by machine and settled on the name "Tessvia." From handing over the spec to naming it: 72 hours.

In three days, we had both a working beta and a product name. None of this moved on momentum alone. It worked because we had decided what to build, and in what order. In this article, I trace, one day at a time, what we actually did during those three days.

The main story (about an 8-minute read)

Day 1: Hand over the whole spec at once, and the MVP stood up the same day

The goal for the first day was clear. A self-contained tool (local-first — running entirely on the computer at hand) that generates Playwright (a tool for automating browser operations) E2E scenarios from natural-language instructions, lets you edit them, and runs them one step at a time.

There's a reason it had to be "self-contained." We want to keep the contents of the screens and operations you want to test out of the open as much as possible. Only the part where the AI writes text relies on gpt-4o (OpenAI's language model); everything else runs inside the computer at hand. We built this separation into the design from the start.

There was one more thing we'd decided about how to proceed. We did not hand over the spec piecemeal. We handed over the MVP spec, including the overall structure (architecture), in a single pass. What we handed over was not just a list of features. We wrote the design out in prose first — how the packages are divided, how the packages call one another, how the data is held — and then handed it over.

As a result, 11 packages with different roles — core, ai, cli, local-bridge, ui, sdk, extension, web, and more — were assembled all at once. In the form of a monorepo, handling them together in a single repository. That same day, we ran the CLI smoke tests (minimal checks that it works). All 15 passed. A working foundation was at hand on the very first day.

The package roles, too, divided up exactly as the design we'd decided on beforehand. core handles the contents of a scenario, and ai handles the part that creates a scenario from natural language. cli is the text-input entry point, and local-bridge is the bridge that connects the various components inside the computer at hand. extension is the browser extension, ui is the screen components, sdk is the window for calling in from outside, and web is the screen shown to the outside. Splitting by role makes it easier later to swap out just one part, or to test just one side. Because we'd fixed the design in prose first, the AI could assemble all 11 along these divisions, without contradictions. When it's decided from the start where everything is, the AI doesn't have to fill in structure by guessing, and we don't have to go hunting around later either.

"Working foundation" is abstract, so let me add detail. As of the first day, you could enter a single natural-language instruction, have it converted into a Playwright scenario, and run it one step at a time from the CLI. That much was working. The detailed build-out, of course, was still ahead. But it was significant to have, on day one, a state where "it's connected without a break, from input to execution." After that, all that's left is to thicken what's around this foundation. We leaped over the stage of exploring the whole picture from zero, on the first day.

Why didn't we hand it over piecemeal? When you hand it over in small pieces, the AI fills in the surrounding structure by guesswork each time. Later, when you look at the whole, you find designs that don't add up mixed in. But when you hand over the whole picture first, the AI can build a consistent foundation all at once. For every bit of rework we avoided, we spent no extra time.

Day 2: Found a way to put the browser extension itself under automated testing

The next day, June 11, was the day I did the most hands-on work of these three. The record of work for this day has 435 entries.

The peak of this day was how to put the Chrome extension under automated testing. A browser extension (a small program you add to a browser like Chrome to give it extra features; Tessvia operates the browser's screen through it) runs in a different place than an ordinary web page. Because it runs embedded inside the browser's machinery, you can't operate it straightforwardly from a test browser. For that reason, people tend to give up, saying "extensions can't be put under automated testing."

But in fact there is a way. You load the real extension as is, and replace only the extension-specific API called chrome (Application Programming Interface — a window for calling features) with a stand-in (a stub). This way, you can leave the extension's contents as they are and test it by controlling only the exchange with the outside, at hand. It was the day we moved the browser extension from "something that can't be put under automated testing" to "something that can."

This looks small but was a big step. Testing through browser operations, tutorials, manuals, monitoring — everything Tessvia wants to do rests on "operating the actual screen through the extension." If the extension itself can't be put under automated testing, none of the four uses loaded on top of it can be verified. Conversely, once this is solid, you can load four uses on top of the same mechanism. Fixing this foundation on day 2 kept the development that followed on a straight line. Deciding the order in which you build is also about discerning this kind of "foundation to fix first."

Alongside that, this was the day we also built the "mechanism for running one step at a time." Rather than running a scenario straight through, we made it possible to advance one move at a time, stopping as you go. With this, you can follow the steps the AI generated one by one with your eyes and confirm them. You can catch a wrong step on the spot, before it runs all the way through. It was the day's other result, alongside the extension testing method (the detailed meaning of this mechanism is explained in the entry-point article, "What Tessvia Is").

On the same day, we also designed scenarios with branches. Branching like "if already logged in, proceed; otherwise log in." That said, not everything went well this day. When scripts were loaded repeatedly for each step, a bug appeared where the same variable got declared twice. Because the build injected the extension's script every time a screen opened, the second injection would error out with "that name is already declared," and the extension would stop there. Even when we dodged it as a stopgap, the same problem came back somewhere else. The story of this double injection, and the story of how we later rebuilt the branch design, I'll write about in detail in another article in this series.

Day 3: Looked up 50-plus domains and decided on "Tessvia"

On the third day, June 12, we decided on the name.

A name costs more the later you change it. The package names, the repository name, the domain, the product name shown to the outside. Renaming all of it after you've decided is a lot of work. So we wanted to confirm beforehand whether a name was usable.

So we used RDAP (Registration Data Access Protocol — a mechanism for querying domain registration status by machine). We checked the availability of 50-plus candidate domains at once. Dividing it into three batches and a final check, we mechanically narrowed down the usable candidates. It's far faster than looking them up one by one on intuition, and it reduces oversights.

This is how "Tessvia" was decided. The package names, which until then had been @eln/ai-test-builder-*, were renamed all at once to @tessvia/*. The rename reached all 11 packages. Because we'd confirmed the name by machine beforehand, we didn't waver here with "actually, let's use a different name." When you look it up first, the work that follows goes smoothly. From the first day we handed over the spec, up to here: 72 hours.

Why did we decide the name this early? Because the further development goes, the more places a name has to be re-applied. Package names, the repository name, the domain, the product name shown to the outside, the wording inside documents. At the point of day 3, the rename is just the names of the 11 packages and the repository. A few months later, it would drag in published URLs and distributed materials too. Decide early — and after confirming by machine whether the name is even available. Then the work won't stall over the name later. It's faster than a person thinking up candidates one at a time and looking them up on a hunch, and it reduces oversights like "it was actually already taken." Getting the deciding out of the way first is an investment in the speed of building itself.

Why we could build it: fix the design in prose first, then have the AI implement it

Getting this far in three days wasn't momentum alone. We'd decided how to proceed.

First, you write what you're going to build, in prose. What kind of screen, what operations, which package handles them. Then you write the expected behavior as tests, first. Finally, you have the AI implement the contents so those tests pass. Decide the spec first, then implement — that's the order (Test-Driven Development, TDD — a way of proceeding where you write the tests first and then build the contents).

The more you leave to the AI, the more this order pays off. When you fix the conditions the implementation must satisfy in prose first, there's less room for the AI to fill in structure by guessing. Conversely, when those conditions stay vague and you ask it to "build something nice," you pile up code that looks plausible but doesn't add up later. What humans do has shifted from writing code one line at a time to deciding "what to build, and in what order." More than the speed of your hands, it's the speed of deciding that determines the result.

Concretely, here's what that means. When you write a test first — "when you do this operation, this should be the result" — the AI writes code to satisfy that result. Because the destination is shown as a condition, the room for interpretation narrows. Conversely, when you ask it to "run the E2E nicely" with no test, the AI returns code that looks plausible at a glance, but since what counts as correct isn't decided, it stops adding up later. A test is, for the AI, the "condition for passing." The more you put the condition for passing into prose first, the more stable what comes out. Behind taking shape in three days is this way of proceeding — "decide the condition first."

There's one more thing that supported this speed. Not carrying the work alone. Looking things up, implementing, confirming. This whole chain, we leave to the AI as far as we can. What I do is decide the spec, and judge whether what comes out matches the intent. Because I could concentrate on these two, we got all the way to naming in three days. What determined the progress wasn't the number of hands. It was the speed of deciding what to build and in what order, and the quality of that judgment.

This way of proceeding isn't a story of it happening to work for this one product. We develop under the policy of handing our own work over to AI, one piece at a time. The human role shifts from the side that moves its hands to the side that decides what to build and how, and judges what comes out. Deciding, and confirming. The more you pool human time into these two, the more you can stand up multiple products in parallel even with few people. The first three days of Tessvia are also one record of trying that way of proceeding in actual product-making.

How far it worked in three days

Even after the name was decided in three days, we didn't stop. As of June 13, the extension's E2E (testing through browser operations) was 27 tests, and the body of tests had grown to 3,367 lines. We also put in the detailed settings screens and the switching between Japanese and English. It had crossed beyond the prototype stage into a state you could call a working beta.

In just three days, we had a foundation that generates scenarios from natural-language instructions, runs them one move at a time, and tests the extension (Japanese and English support came right after, on June 13). We could get this far because, on the first day, we held "a foundation connected without a break, from input to execution." When the foundation is there first, all that's left is to thicken what's around it. We'd leaped over the time of searching out the whole picture from zero, on the first day. When you decide the order in which you build, this much takes shape in three days. Put the other way, what we could build in these three days reaches only as far as a "working beta." The build-out from here, and how we grew it into a product you can actually use, we'll continue in the series that follows, one by one.

These three days lead to Tessvia today

That's the three days up to the birth of the name Tessvia. We handed over the spec all at once, stood up a working foundation on the first day, made the browser extension testable on the second, and decided the name on the third. In three days, a working beta came to be.

In the series that follows, I'll write about how the first version we built here became the Tessvia of today. How we composed the browser extension's E2E, how we stopped the AI's mistaken generation, how we realized automatic generation of operation manuals, and how we folded two projects that started separately into one. One by one, in order. What kind of service Tessvia is today, I've gathered in the entry-point article, "What Tessvia Is."

Three days this short wasn't the result of momentum. Deciding first what to build and in what order, then judging whether what came out matched the intent — these three days are a record of putting that way of working to the test on the first real product. What we went on to stack on the foundation laid here, I'll write about one article at a time, starting with the next.

Lessons you can carry over

Hand over the spec all at once rather than piecemeal. When you hand it over together, including the overall structure, the AI can build a consistent foundation. You get a foundation that adds up, faster than patching on afterward.
Design first, implement later. Fix what you're going to build in prose, write the expected behavior as tests, then have it implemented. Leave the destination vague and you pile up code that looks plausible but doesn't add up later.
Look up names by machine. Rather than crossing off candidates on intuition, checking domain availability all at once with RDAP and the like is faster, and you miss fewer.
Hold a "working foundation" on day one. Stand up a foundation connected without a break, from input to execution, first, and all that's left is to thicken what's around it. When you can leap over the stage of searching out the whole picture from zero on the first day, you can turn the rest of your time to the build-out.

What Tessvia Is — One Scenario, Four Deliverables

uehara — Thu, 01 Oct 2026 07:52:29 +0000

The short version

Tessvia is a service where you write the steps a user takes on screen once, as a single scenario (the one source everything comes from), and then mechanically derive four different deliverables from it. The four are: tutorial playback, automatic manual generation, E2E (end-to-end — reproducing a user's actions exactly as they happen) tests, and liveness monitoring of the screen. You no longer write the same operation four times.

We built this tool while handing our own development over to AI, one piece at a time. Tests, manuals, and monitoring all come down to the same thing: writing out "the user operates the screen like this." If that's true, then write those steps once and reuse them. Tessvia starts from that plain idea. This series describes how we built Tessvia and what we learned along the way, in order. This article is the entry point: it explains, with the actual screens, what Tessvia does.

Why combine four things into one

What we want to do is one thing. You write the steps — "the user operates this screen like this" (the scenario) — just once. You manage that one scenario as the single source. After that, you mechanically derive four different deliverables from the same scenario.

Tutorial playback: Overlay callouts and highlights on the actual screen to guide the user through the operation.
Automatic manual generation: AI actually traces the same operation, captures it, and assembles it into manual pages.
E2E tests: Run the same operation as a Playwright (a tool for automating browser operations) test to check that the screen isn't broken.
Liveness monitoring (at the design stage): Run the same operation on a schedule to watch whether the production screen keeps working. This fourth use is still at the design stage (implementation came after the other three).

Normally, these four are written separately by separate people. And so, bit by bit, they diverge. Take a concrete example. Suppose a "Submit" button on some screen changes to "Save and submit." The engineer fixes the button name in the E2E test. But the image pasted into the manual isn't re-captured, so it keeps the old button name. The tutorial callout still has the old wording too. The monitoring script, if unlucky, can't find the button and reports a false alarm. One change ripples to four places, but only the one place someone noticed gets fixed. That is the real cost of writing them separately.

If everything comes out of one scenario, fixing the button name in that scenario changes all four at once. You don't write the same operation four times. Drift caused by forgetting to fix something can't happen, in principle. That is also the motivation for building it. We just wanted an easier way to avoid writing the same work over and over. That's all.

What a scenario looks like

The source "scenario" doesn't require learning any special notation. You just line up steps like "press this button" or "type this into this field" one at a time, in plain language (everyday words). For each step, you write a title (the name of that step) and a body (what it does).

When you run it, you don't have to rush straight to the end — you can advance one step at a time, stopping as you go. Because you can confirm which step changes the screen's state, you can also check step by step, with your own eyes, whether the steps the AI proposed are correct.

Steps can branch, too. For example: "if already logged in, proceed; otherwise log in first." Real operations don't always run in the same order. Because you can write the points where the path forks by state into one scenario, both the test and the manual can express the flows that can actually occur, as they are.

Why it's hard to break: the AI finds the elements

This is the most important part of the tool. How do steps written in plain language turn into actual operations?

With traditional E2E tests — tools like Playwright or MagicPod — to make a machine perform the operation "press the login button," a person had to specify exactly where that button is on the screen, as a DOM selector (a way of pointing to an element within the HTML structure). It meant tracing the element in the developer tools and teaching the machine "the login button is this element." The problem is that the moment the screen's construction changes a little — the button moves, the HTML structure gets rearranged — that specification stops working and the test breaks. Nothing about the feature itself is broken, yet the test goes red just because the selector slipped. This is the single biggest reason E2E tests have been called "hard to write and hard to fix."

Tessvia hands that part to the AI. All a person writes is the step in everyday words: "press the submit button." When it runs, the AI compares that sentence against the screen currently open, finds which element is the "submit button" on its own, and operates it. There is no step where a person writes and fixes a DOM selector. So even if the button's position or the HTML structure changes somewhat, as long as the intent "submit button" is the same, the AI can find the same button again. The situation where you remodel the screen a little and the tests all go red, one after another, becomes much less likely.

This form — "a person writes the intent, the AI finds the element" — produces two effects at once. One is that the scenario is easy for people to read. It isn't a list of selectors but the manual itself — "log in," "press the submit button" — so even someone who didn't write it can understand its contents. The other is that the test is hard to break, because the fixed selector that was the source of fragility has been replaced by the AI's search at run time. Readability and being hard to break usually don't go together, but here both come out of the same mechanism. And these same steps flow on not only to the test but to the tutorial, the manual, and the monitoring — which is the point made at the start of this article.

Seeing it on screen (1): running a test from the extension's console

Words alone are hard to convey, so let me show you the actual screens. Tessvia has a Chrome extension (an add-on) that you install in your browser. With the screen you want to test still open, you open a tall panel over it, and from there you run scenarios, create manuals, and report bugs.

The screen below is that extension's panel. At the top left it says "Test & Record Console," and "Connected" at the top right indicates that it's connected to our own (EarthLink Network) workspace. Tessvia organizes work by organization (a company or team) and project (the target app). Here, a project called "promptflow CI catalog" is selected. The panel is divided into three tabs — "Test Execution," "Manual Authoring," and "Bugs" — and the Test Execution tab is currently open. (The tab names appear in English on screen.)

The "Test Execution" tab lists the registered tests. In this example, a suite (a grouping of tests) called "CI full suite (promptflow)" contains 361 cases, divided into per-feature categories like "chat (23)," "auth (62)," and "billing (59)." Open a category and you can see each case name one by one, along with how many steps of operation each is made of (for example, "5 steps"). Every one of them is simply a scenario a person wrote in plain language, laid out as is.

Seeing it on screen (2): open a case and let the AI run it

Open a single case and the scenario — what it does — is laid out step by step. The screen below is a case for "can send a message and receive an AI response," where five steps read straight through: "type a prompt into the chat input field," "press the submit button," "the response is displayed as streaming," and so on. There's no special notation. The steps a person wrote in everyday words become the units of execution as they are.

When you run it, as described earlier, the AI traces these steps on the screen from top to bottom. Where it says "press the submit button," the AI finds the submit button on the current screen and presses it. It works without a person specifying selectors, and it follows along even when the screen changes somewhat. Whether each step passed is something the AI checks automatically. It's fully automatic by default.

On top of that, from "JUDGE THIS CASE" at the bottom of the screen, a person can also manually set "Pass," "Fail," or "Skip." When the automatic result doesn't match reality — for instance, when it nominally ran through to the end but, to a human eye, should be a fail — the position is that a person can override it at the end. The AI does the hands-on work of operating the screen; a person can hold final responsibility for the pass/fail result. In other words, the hands-on work and the responsibility for pass/fail are separated.

Seeing it on screen (3): report a bug right where you found it

If you spot something wrong with the screen in the middle of testing or authoring a manual, you can report it on the spot, without switching to another tool. The screen below is that "bug report" modal (a small window). It captures the screen you're looking at as is, and you can overlay annotations — Arrow, Circle, Square, Text — in a color of your choice. Instead of writing out in words what's wrong and where, you can mark it directly on the image. After that, you just write a title and a description and press "Register report."

Reported bugs accumulate in the list on the same "Bugs" tab. In the screen below, "RECENTLY REPORTED BUGS" lists items that were actually reported, like "model switch reverts to default on reload" and "individual cost of some models shows NaN in comparison mode." Each is linked to the scenario it was reported from, so you don't have to recall later "which screen, in what state, was this."

How to present test results (passed or failed, which step it stopped at) and how to actually compose test scenarios are covered in detail in the E2E installment of the series. In this article, just take away the overall picture — "four things come out of the same scenario" — and the fact that these operations are completed right on top of the target screen.

Why a Chrome extension

Testing and manual-authoring tools usually run in a screen separate from the target app. That means you open the manual and the actual app screen in separate windows. You end up going back and forth, retracing memory — "this operation was, I think, on that screen…" — and as that memory goes stale, the manual's contents and the real screen drift apart.

The extension form is a choice to eliminate that back and forth. With the page under test still open, you layer a panel over that same screen. You can point to the button you're looking at, right there, and record it, right there. The person writing the steps and the screen being operated are in the same place. So the steps and reality are less likely to drift apart. Another benefit is that the screenshots on each step of the manual are captured from this very "screen you're looking at." There's no need to stand up a separate environment for captures and shoot them again.

An AI agent traces the actual screen

When you say "automatically generate a manual," people tend to think it's just lining up images captured in advance. Tessvia is different. An AI agent (a mechanism that operates the screen automatically) actually traces the steps written in plain language and captures the real screen at each step. So even if the app's appearance changes, re-running the scenario re-builds the manual with the latest screen. The entire step of manually taking screenshots and swapping them in is unnecessary.

We provide two ways to actually operate the screen, depending on the use. One is "generate here," where Tessvia's server itself drives the browser and captures. It's self-contained at hand and requires no extra setup. The other is "via a runner," where a capture agent running on a separate machine operates on your behalf. You use this when you've placed the server somewhere and want to capture an internal app from the machine at hand. You can choose where to run it based on where the screen you want to capture is.

Apps that require login can be captured too. If you prepare and pass in session information for an already-logged-in state, the operation can start from that state. Authentication information is passed only as environment variables (configuration values passed into a program from outside), never written down in files or logs. We've kept to a design, from the start, that puts as little of the test target's contents outside as possible.

Liveness monitoring, the fourth use (at the design stage)

As a fourth use, we are designing liveness monitoring. The concept is to run an existing scenario automatically on a schedule and watch whether the production screen is still working. Tests run on every change, but monitoring runs separately, repeating on a set schedule. If one morning the login steps have started stopping partway, you can notice it before the users do — that's the shape monitoring aims for.

What matters here is, again, the "one scenario." Instead of writing new steps for monitoring, you simply run an existing scenario on a schedule as is. The flow checked by the test and the flow watched by monitoring can be said to be the same thing. You can keep all four — tutorial, manual, test, and monitoring — coming out of the same single scenario, right to the end. Note that this liveness monitoring was implemented later than the other three and is still at the stage of preparing the integration design. What's actually running is the tutorial playback, automatic manual generation, and E2E tests shown on screen so far.

The flow of using it

The overall flow is simple.

Write the scenario once. Write the steps — "operate this screen like this" — in plain language (everyday words). Just line up a title and a body for each step.
Have the AI actually trace it. Following the steps you wrote, the AI agent operates the actual screen and captures a screenshot of each step. In a few minutes you have a manual with screenshots, which you can also download as is.
Expand it to four uses. Play the same scenario as a tutorial, assemble it into a manual, run it as an E2E test, or run it on a schedule as monitoring (monitoring is at the design stage) — dispatch it according to the use.

What a human does here is two things: deciding the steps, and confirming that what comes out matches the intent. The work of writing out the same operation four times is shifted to the machine's side. More than the speed of your hands, it's the speed at which you decide what to make and in what order — and the quality of that judgment — that determines the result.

Who the tool is for

Tessvia is for people who want to automate their own development and operations as much as possible. We have in mind, first of all, developers running multiple services with few people, and people doing business on their own. In this position, the person who writes the tests, the person who makes the manuals, and the person who watches the monitoring are usually the same one person. When one person wears four hats, the burden of writing the same operation four times hits directly. So the value of a mechanism where you write once and expand to four shows up largest here.

We ourselves needed this tool from exactly that position. Developing and operating our own several services with few people, we had no room to prepare tests, manuals, and monitoring separately. If hands are short, the only way is to build things so you don't do the same work over and over. Tessvia is a tool born of that need. So this series is not a clean product introduction but a record of the process of building something out of necessity.

When you automate one piece of work, the next piece you want to automate comes into view. You want to create tests automatically — then you want to create manuals automatically — then you want to watch automatically whether it's running. That chain, gathered into one scenario, is Tessvia today.

How to read this series

Tessvia today is actually two projects that started separately and later merged into one. One side was "making scenarios," the other "delivering (distributing) scenarios." In hindsight, these two were two faces of a single service, but at the time they started out in separate contexts, so for a while they grew as separate products. Why we were slow to notice this duplicated development, and how we folded them into one, is something this series will also cover.

In this series, we trace that history in order. The next article describes the first three days when the name Tessvia was born (a record of its birth) — the story of how, when we handed over the spec all at once, a monorepo (a mechanism for handling multiple packages together in one repository) MVP (minimum viable product) came up the same day. After that, we continue one by one: how we composed the browser extension's E2E, how we stopped the AI's mistaken generation, how we folded a duplicated implementation into one, how we realized automatic manual generation, and so on.

This is not a clean, finished design written up after the fact. We'll write the process as it was — making mistakes, fixing them, and sometimes rebuilding the whole thing — as we went. We hope it conveys what actually happens to those who, like us, are trying to automate their own work with AI.

A lesson worth carrying over

Write the same operation only once. Tutorial, manual, test, and monitoring look like different things, but their contents are the same "steps for operating the screen." Gather them into one scenario and drift from forgetting to fix something can't happen, in principle.
Hold the scenario in a machine-readable form. An operation where a person fixes four places by hand will, someday, drift somewhere for certain. Build it so fixing one place changes everything, and there's no room left to drift.
Do the work right on top of the target screen. The more you switch to another tool, the more you handle the manual and the actual screen separately, and divergence is born. When you can run it right on top of the screen you're looking at, the steps and reality are less likely to drift apart.
Humans concentrate on steps and judgment. Shift the hands-on work of writing it out four times to the machine, and let humans decide what to make and confirm that what comes out matches the intent. The further automation goes, the more this separation pays off.

A health check I added in good faith took down healthy AI servers

uehara — Wed, 30 Sep 2026 08:44:46 +0000

Conclusion (the point of this article — about 8 min read)

On my home LLM (Large Language Model — an AI model that generates text) infrastructure, I changed the monitoring approach once — and reverted it the same day.

What I added was an active health check: make the model actually generate a single character, and judge from the response whether the node is usable. The goal was to catch the state where the process is alive but inference never returns. The problem: I applied a 10-second cutoff to an environment where normal inference takes about 31.8 seconds. Under that condition, 3 of 4 healthy nodes get removed from rotation.

For inference that takes tens of seconds even when healthy, an active check shorter than the normal response time is unusable. It cannot tell "slow but normal" from "stalled and never returning," so it drops healthy nodes.
The original problem — status returns 200 while only inference stalls — is caught, in the end, by real user requests themselves. I stopped probing and instead watch actual traffic: when a real request fails to return within 120 seconds, or comes back 5xx, and that persists across the last 8 requests, the node is removed from rotation. A wedged /api/chat surfaces on its own — the real request simply times out, no probe needed. The liveness check went back to the light /api/version (0.25s), splitting "is it alive" and "is the work stuck" into two separate monitors.
I did not abandon active checks as a technique. In fact, liveness is still checked actively (a light periodic hit to /api/version). What I dropped was only the "probe inference actively" part. Actively hitting fast work to verify it remains valid as ever; the assumption only breaks when you use an active check to measure inference — work that is slow even when healthy.

Below, in order: what I was monitoring, where I misjudged, and how I went back.

What I was monitoring

I run a small setup for LLM inference. Four Macs carry a 14-billion-parameter (14B — a measure of model size) classification model, and in front of them sits the load-balancing feature of Caddy (a web server and reverse proxy — it takes incoming traffic and distributes it to the machines behind it). Incoming requests get routed to whichever Macs are alive.

The classification model is a small role: it sorts each incoming request by what kind of work it is. Heavy text generation goes to a separate lane; these four machines just do light triage, fast. Since the per-node work is light, the design is to line up several of them and handle requests in parallel. Why I run this myself and how it is assembled are covered in other articles; here I stick to monitoring.

Once you pool four machines, you must not keep sending requests to one that has gone bad. Remove the unhealthy machine from rotation automatically, and bring it back when it recovers. How to make that call is the subject of this article. Caddy, the component doing the routing, periodically checks the state of each of the four machines behind it and sends traffic only to the usable ones. That "how it checks" is what I rebuilt once — and promptly reverted.

The symptom — alive, but not answering

A node would occasionally hang so that only inference stopped responding. The wedge in the title refers to this state: stuck, never returning.

What makes it nasty is that the machine itself looks fine. /api/version, a light query that returns the version, answers in 0.25 seconds. But /api/chat, which carries the inference, never comes back. A monitor that only watches process liveness judges this node "healthy" and keeps sending it requests. From the user's side, requests just get swallowed and never return.

What tipped me off was the unevenness: overall the system was responding, yet some requests never came back at all. The load balancer rotates across the four live machines, so when one of them is wedged, only the requests that land on that machine vanish. Even in the first measurement I observed the mismatch: the liveness check returns in 0.25 seconds while inference on the same node times out. The process is alive. But it is not doing its job. A liveness-only monitor cannot catch that difference.

Wedges tended to happen when several heavy requests piled onto the same node, or when a request suddenly hit a node that had been idle for a while (a cold state). The model has to be loaded back into memory first, so requests in that window are kept waiting. The load balancer mechanically keeps routing to any node that is alive, so requests keep landing on the wedged one and stack up, and only the users who happened to hit that node keep waiting. The only way to stop this is to see the wedge and take the node out of rotation.

I have written before that getting a "200" response (in Hypertext Transfer Protocol, HTTP — the conventions of web traffic — the number that means "success") and the thing actually working are two different matters. This article is the sequel: what happens when you apply that lesson to inference, work that is slow even when perfectly normal.

The first fix — measure inference itself

So I decided to measure "ready or not" by actually making the node infer. An active probe (a small request sent to feel out the state) against /api/chat. The idea: flush out the wedge that liveness checks cannot catch, by running one real inference. Ask the machine not "are you alive?" but "can you work?" — as intentions go, a straightforward one.

Here is the payload. A short, 115-byte request that makes the classification model produce exactly one token (the smallest unit of generation — one character's worth here):

{"model":"qwen2.5-coder:14b","messages":[{"role":"user","content":"1"}],"stream":false,"options":{"num_predict":1}}

The decision logic was built on Caddy's own features. Send this request every 15 seconds; time out at 10 seconds; after 3 consecutive failures, remove the node from rotation; a single success brings it back. A response containing a message field counts as success. The 10-second timeout came from the going rate for watching web endpoints.

Before rolling it out, I ran 12 Bats tests (a test framework for configuration) and Caddy's syntax check (caddy validate). I also verified several assumptions against the real environment: the liveness check returns in 0.25 seconds; the probe traffic actually reaches the Macs from Caddy; success can be judged from the message field; tests and validation pass. Those four I confirmed. But the last assumption — the latency of real production inference — I left at the 10-second going rate without measuring it. I thought I was prepared.

It collapsed in production — three miscalculations

It fell apart immediately after rollout. There were three miscalculations.

First, normal inference exceeded 10 seconds. A 14B model, when busy or freshly started (cold — the model not yet warm in memory), takes about 31.8 seconds even to generate a single token. This is not a hang; it is normal operation. The model must be loaded into memory and the conversation so far recomputed, so requesting just one character via num_predict=1 does not make it shorter. The cost sits in preparing the model to run, not in how many characters come out. The very worst case is the first request landing on a node that has been resting: it starts over from loading the model into memory. Checking all four nodes, three exceeded 10 seconds. Under this criterion, most of the healthy pool gets removed.

This is the heart of the incident. An active probe treats "slow to return" the same as "stopped." For fast endpoints, slow = broken is roughly right. But for inference, slow = normal happens all the time, so the same yardstick does not transfer. The monitor itself knocks out healthy machines. The mechanism meant to protect the pool was shrinking it.

Second, extending the timeout does not make the problem go away. Raise 10 seconds to 60 and healthy machines stop being removed — but now a genuinely wedged node is considered alive for a full 60 seconds. The line between "slow but normal" and "never returning" cannot be drawn by duration alone. Wherever you place the threshold, one kind of error grows. Longer protects healthy nodes but dulls detection; shorter detects fast but catches healthy nodes in the blast. There was no way out of that tug-of-war with a single time-based yardstick.

Third, the request body embedded in the configuration was not sent as written. Caddy has a substitution mechanism that splices runtime values into strings in its config. The curly braces {...} of the JSON above (JavaScript Object Notation, a format for writing data) were misinterpreted as substitution placeholders and replaced with empty strings. The 115 bytes became 28, and Ollama (software for running LLMs locally), receiving a broken request, returned HTTP 400 (the number for "bad request").

What made this worse is that caddy validate cannot find it. The config is syntactically correct, so static validation passes. I only discovered it by watching the traffic on the wire and checking the transmitted bytes one by one. On the screen where the config was written, the request looked correct.

Tests passed; the real machine failed

The original change had 12 passing Bats tests and a passing syntax check. It was still wrong on the real machines.

What the tests guaranteed was the syntax of the configuration and the shape of its output. They never checked "healthy inference does not get removed from rotation by mistake." This is the kind of failure that design and unit tests cannot catch — it only appears when the thing actually runs. The idea that the pass line is the real machine is something I have written repeatedly in other articles. This time it simply showed up in the form of a monitoring threshold.

Put differently: tests verify that things work as written. What I got wrong was whether what I wrote matched reality. The 10-second threshold behaved exactly as written. The premise was simply out of line with the real world. Adding more tests does not close that gap. The only prevention is ordering: measure the real environment's numbers first, then make them the premise.

Back to passive, the same day

What I chose was to remove active inference monitoring entirely.

In its place I put passive monitoring that observes real traffic. Stop sending probes; instead, watch how actual user requests are being answered. A node whose responses are abnormally slow, or which returns errors, gets picked out of the records and removed from rotation. It adds not a single extra request for monitoring's sake, so it puts zero additional load on the pool — it only counts traffic that is already flowing.

The thresholds were set like this. Any request that takes more than 120 seconds marks its node as unhealthy. Healthy-but-long inference measured up into the 10–60 second range, so I raised the bar past where it could catch those. If I had kept the going rate of 8–10 seconds, I would have been removing healthy nodes all over again. Errors (5xx responses) also count as unhealthy, and when failures persist within the most recent window (8 requests), the node comes out of rotation. It stays out for 30 seconds; after that, real requests probe the waters again.

The key point is that the numbers are split in two. The yardstick for what counts as unhealthy (a response over 120 seconds, or a 5xx error) and the duration a node stays removed once judged unhealthy (30 seconds) are separate settings. Making one number do both jobs — a number premised on fast work — is exactly the hole I had just climbed out of.

The liveness side went back to the light /api/version: sent every 5 seconds, cut off at 3. That is plenty fast for noticing a dead process. Whether inference is wedged is the passive monitor's job. Having failed at making one yardstick measure both, I split the roles in two: one watcher for "is it alive," another for "is the work stuck."

Why active inference monitoring was rejected is written down in the config file's comments and in the commit message of the change. A separate design document has not been written so far. Still, for the next time the same temptation comes around, the reasons live where my future self will see that a past self already tried this once and pulled it out.

With active gone, how do wedges get caught?

An obvious question remains: if I stop probing, don't wedged nodes slip through?

The answer: they get caught by the latency and errors of real requests. A real request routed to a node fails to return within 120 seconds, or comes back 5xx. When that persists within the last 8 requests, the node is judged wedged and removed. A wedged /api/chat shows up as real requests timing out — no probe needed. The wedge surfaces in the traffic on its own; monitoring only has to count what surfaces.

This approach has a real weakness. An active probe could preempt: declare "this node is not ready" before any user request arrives. Passive monitoring only knows after a few real requests have already gotten stuck. It cannot preempt.

I chose passive anyway, after weighing the two. The active probe was removing healthy machines wholesale. Passive monitoring delays the handful of requests that land on a wedged node. Dropping three healthy nodes outright, versus a few requests running late on one wedged node — the latter is the lesser harm. And in practice, removing healthy nodes concentrates load on whatever remains, which then starts to wedge as well. A monitor that spreads the outage is the outcome I most wanted to avoid.

There are also guards against misreading. Connection failures cap at 2, and health is judged over a window of the last 8 requests. This keeps "one request happened to be slow" from being mistaken for "persistently wedged." If a single fluke pulls a node out and pushes it back in over and over, that churn itself becomes the instability.

How far does this generalize?

Let me draw one line.

Active probes are not bad. For fast work, hitting the endpoint for real and checking the response content is still effective. Don't accept a bare HTTP 200 as health; look inside the response. That principle has served me well.

It only broke when applied to inference — work that takes tens of seconds while perfectly healthy. The criterion is a single number: how many seconds the target takes to respond when normal. Hit it with a yardstick shorter than that, and the active probe switches sides — it becomes the thing that drops healthy machines. So: measure the normal time first, then set the threshold. It looks obvious, and it was the single most effective ordering in this incident.

This may sound specific to self-built AI infrastructure. But work that is slow while healthy exists elsewhere: converting large files, heavy aggregation, long calls to external services. Monitor those and the same trap is waiting. Do not carry a yardstick premised on fast responses into slow work. That one point is the whole lesson.

Conversely, for an endpoint that answers in one second, active monitoring is still the right fit. In fact, on my own system liveness is still checked actively by hitting /api/version — I did not drop active checks, I only stopped applying them to slow inference. For fast work you can honestly say slow = broken, so hitting it for real and checking the content bothers no one. The dividing line is not the speed of the work itself but whether it can be slow while normal. Passive monitoring for work that is slow even when healthy; active monitoring for work that is fast when healthy. Same discipline, chosen by the nature of the target — and that is all there is to it.

What this incident decided

The time that counts as "unhealthy" gets decided after measuring the target's normal response time. Never place the threshold first and fit the measurements to it afterward. Treat going-rate values with suspicion — they were made for fast work.
Whether inference is wedged is judged by observing real traffic. Active checks are limited to a light "is the process alive." One yardstick never gets to cover both liveness and wedging.
Rejected options get their reasons recorded in both the config file and the change history. So the same choice never has to be made twice from scratch. Six months from now, I will not remember today's reasons — I plan for that.

The Real Reason AI Ignores Existing Decisions — Designing an ADR Context Auto-Injection Hook

uehara — Mon, 28 Sep 2026 11:51:03 +0000

TL;DR (about a 15-minute read)

In May 2026, I was reworking a Claude Code plugin that delivers a shared way of working to several development projects. One of its targets was a mechanism for carrying past design decisions forward into the next session.

It started with a choice of document-editing library. In one product, an Architecture Decision Record (ADR, a document that captures a design decision and its reasoning) had adopted BlockNote and rejected using TipTap directly. Yet about two weeks later, in the course of the work, the AI proposed newly introducing TipTap. The design record existed and its contents were clear. What was missing was a path to make its existence known at the start of a session.

So I wrote a script that reads the working project's docs/adr/, builds an index of only the number, title, and status from documents marked Accepted or Amended, and injects it into the context from a Claude Code hook. The first version ran on UserPromptSubmit, on every turn. The design does not inject every document in full; it passes only a list so that relevant documents can be chosen and read.

I cannot call this mechanism "a system that guarantees the AI follows existing decisions." What could be made deterministic goes only as far as the hook firing and delivering the index string into the context. Whether the model selects the relevant documents, reads their bodies, or follows the decisions is a separate matter. I have not measured by how many percent the violation rate dropped, either.

Revisions were needed after it went into use. In July I stopped the per-turn duplicate injection and moved it to SessionStart. In August I consolidated an implementation that read 121 files through individual commands into a single awk run, cutting the measured median from 3,842ms to 94.7ms. In September I stopped re-injecting the full index on compact and resume, inserting only a short pointer instead.

Running the current implementation on 2026-09-27, the plugin-base project had 144 ADR files, of which 107 were Accepted or Amended. Because the cap is 30, startup emits 30 of them, 3,510 bytes. On compact it becomes only a 262-byte pointer. Meanwhile, the product where it all started had 27 of 29 files in scope, and all 27 were shown. It works at small scale, but on projects where the count has grown, the later decisions fall out of the index.

What I built is a plugin that passes design decisions to the next session

The purpose of this plugin set is to make ways of working — spec authoring, review, testing, deploy verification, and so on — reusable across multiple development projects. Rather than a person pasting the same explanation every time, they are registered as Claude Code skills and hooks and run when the situation calls for them.

ADRs had the same problem. When you record a design decision in a document, people can search for it later. But a new AI session does not necessarily inherit the previous conversation as-is. Even if a repository holds 100 documents, unless they appear in the context as something to read, they remain undiscovered as far as the model is concerned.

What was needed was not a replacement for full-text search. It was to separate the following three stages.

Deliver existence: put a list of the currently effective design decisions into the context.
Read the relevant documents: select the ADR bodies related to the work at hand.
Reflect them in the implementation: turn the decisions and constraints into diffs and tests.

This hook handles only stage 1. Writing that the same mechanism guarantees stages 2 and 3 would be claiming a broader outcome than the implementation delivers.

Against an already-adopted BlockNote, TipTap was re-proposed

The product where it started was building a feature for writing your own notes and documents. What this needs is a rich text editor — the on-screen component for editing text with styling, headings, and lists. The two candidates, BlockNote and TipTap, are both libraries for embedding this editing capability into a React product, and BlockNote uses TipTap internally. A design record dated 2026-04-27 adopts BlockNote as the foundation for the block editor. This design record was added to the repository the next day, on the 28th.

The requirements were Notion-like block operations, pasting and copying Markdown, saving as JSON, and embedding into React. Using TipTap directly was also compared, but you have to assemble the slash menu, drag handle, indent, and Markdown conversion yourself. BlockNote had these operations from the start, so BlockNote — the smaller gap from the requirements — was chosen.

On May 12, in a session examining a different screen, the AI proposed introducing TipTap. This event is recorded in the context-injection design record created the same day. The original conversation transcript itself is not among the materials I verified this time, so I do not reproduce the exact wording of the statement. What can be confirmed is that a proposal contradicting the existing editor choice was recorded as the trigger, and that the hook implementation began that day.

Two days later, on May 14, a minimal editor using BlockNote 0.34 was implemented, exactly as the original decision called for. The final code returned to the existing decision. But a person noticing and reverting it is not the same as the AI being able to discover the decision from the start.

I did not settle for "the model forgot" as the cause

The design record at the time separates the candidate causes into four.

The ADRs were merely placed in a directory, with no explicit operation to read them.
Even writing "check the ADRs" inside a long project instruction did not always get referenced during the work.
For concrete implementation requests, the proposal moved ahead before existing decisions were checked.
Conditional instructions like "check at implementation time" left room to be interpreted as not applicable.

This is not to say I scientifically established that "recency bias (the tendency to overweight newer information) is the cause." There was no comparative experiment varying multiple conditions, and no tally of violation rates. The structural gap I could confirm is that there was no processing that fed the existence of the ADRs into each session.

The title's "real reason" refers to what was determined within this single case. It is not a general cause for all AIs failing to reference documents. Whether the document quality was poor, and whether the same countermeasure would have the same effect on a different project, cannot be decided from this record alone.

Even so, the point to improve can be made concrete. Instead of adding more sentences saying "please check the ADRs," build the list of what to check at runtime, and return it in a form that Claude Code can add to the model request.

I injected only the index of current decisions, not the full text

The first version's processing was in this order.

Determine the root of the working project.
Gather ADR files directly under docs/adr/.
Read each file's heading and Status.
Keep only Accepted or Amended.
Emit the number, title, and status for up to 30 entries.
Add them to the Claude Code context.

Draft is unsettled, Rejected is not adopted, and Superseded is already replaced. Mixing these into the same list would leave the model unable to distinguish a current constraint from an old proposal. Looking only at the target project's directory is to avoid mistakenly carrying in another product's decisions.

The reason for not injecting every body in full is information volume. The first version's budget was about 500 tokens for 20 entries and about 750 tokens at the 30-entry cap. The index carries only "what has been decided"; the detailed reasoning is obtained by reading the ADRs you need.

Even the current Claude Code hooks official documentation defines UserPromptSubmit as firing on each prompt submission and SessionStart as firing at start or resume. The additionalContext a hook returns enters the context as a system reminder and is read on the next model request. The first version used these official extension points.

Review of the first version found three implementation defects

Even if the idea was right, the first script was not safe as written. The first Pull Request touched 5 files, adding 273 lines and deleting 1. Automated review ran twice and pointed out 3 implementation defects in total.

First, the truncation to 30 entries was done before the Status filter. If Rejected entries were mixed into the top 30, valid ADRs further down would not be shown. After the fix, all entries are parsed, only Accepted and Amended are kept, and then it truncates to 30.

The second was the project root fallback. When the environment variable was absent, it used the hook script's own directory as the base. No matter where the plugin is installed, the ADRs of the project in use are not found there. I changed it to the order of environment variable, then the git root of the current directory, then the current directory.

The third was the internal delimiter. Separating number, title, and status with | breaks the columns when the title itself contains a |. I switched to tab separation and preserved the symbols and spaces within titles.

All 3 were fixed before merge. What this history yields is not the simple conclusion that "a hook is guaranteed." A hook firing does not pass through model judgment, but implementation defects in the script occur just as in ordinary code. Fixtures, review, and checking the execution results are required.

The range that could be made deterministic goes only as far as delivering the string

The first version's design record describes the hook as "deterministic." Without qualifying its range, this word invites misunderstanding.

If Claude Code fires the event, the plugin is enabled, and the script exits normally, generating the list does not depend on the model's spontaneous judgment. The Accepted and Amended filter, the cap, and the output format are also decided by the shell script. Up to here, the same processing can be applied to the same input.

After that it differs. The model that sees the index may not select the relevant ADRs. Even if it selects them, it may not read the bodies. Even if it reads the bodies, it may misjudge how to apply them to the work at hand. The "second stage that extracts relevant decisions from keywords" and the "third stage that requires ADRs be read before Write/Edit," which the first version had planned, are not implemented in this current hook.

Furthermore, in environments without jq it silently skips. Even on projects without docs/adr/ it emits nothing. If the plugin itself is not enabled, the hook does not run. I built a delivery path for design decisions, but that is not a guarantee that they always arrive in every environment.

Spelling out this boundary also changes how success is measured. The values to watch are at least these three.

The proportion of target sessions in which the hook fired.
The proportion in which the relevant ADR bodies were actually read.
How many proposals contradicting existing decisions there were, before and after the fix.

This history has no data that continuously tallies these three. That is why I do not write a post-introduction drop in the violation rate as an outcome.

Two months later, I stopped the per-turn injection

The first version was registered on UserPromptSubmit, injecting the same index every time the user sent a prompt. There is a benefit to having the list arrive near a new instruction. But at 30 entries it uses about 750 tokens every time. Over 20 turns, that means sending the same list 20 times.

In a revision dated 2026-07-15, I changed the registration target to SessionStart. The implementation was merged the next day, on the 16th. The index is injected once at session start and remains within that context, so the per-turn resend was stopped. The plugin version went from the first version's 0.4.0 to 0.21.0 with this revision.

There is a point that weakens in exchange. An ADR added mid-session is not automatically reflected until the next start. Also, in a long session, many turns come between the list injected at the start and the current exchange, making it harder for the model to reference the list. This revision did not "keep freshness intact and remove only cost." To reduce the resend cost, it is a decision that gave up both the immediate reflection of ADRs added mid-session and the benefit of placing the list right next to a new instruction.

In the current Claude Code, SessionStart fires not only at startup but also on resume, clear, and compact. So even the description of "exactly once per session" is not sufficient. The later compact handling became necessary because of this event characteristic.

As the file count grew, the start processing itself became slow

Even after reducing the frequency, the cost inside the script remained. The first version launched grep and sed multiple times per ADR. At the time of the August 2026 audit there were 121 files, and the structure created about 1,100 processes for a single start.

In a record of measuring the same input 10 times, the old implementation's median was 3,842ms, with a minimum of 2,236ms. Changing to an implementation that reads all files in a single awk run yielded a median of 94.7ms and a minimum of 71.4ms. About 40 times faster. The heading, Status filter, 30-entry cap, and output are preserved.

Reducing only the character count of the index still delays session start if generating it takes several seconds. Context cost and process cost had to be measured separately.

After compact, I stopped re-injecting the full index

In September 2026 another problem surfaced. Claude Code fires SessionStart even after it compacts the context. Re-injecting a list exceeding 3KB right after shrinking it grows the reduced context again.

The current hook looks at the input's source. For startup and clear it returns the normal list, and for compact and resume it returns only a short pointer saying "check the ADR directory if needed." The results of running the current code on 2026-09-27 are as follows.

source	output	measured bytes	ADR lines
startup	index of up to 30	3,510	30
clear	index of up to 30	3,510	30
compact	pointer to where to re-check only	262	0
resume	pointer to where to re-check only	261	0

There are 13 regression tests that check this branching, and all passed. They include boundaries such as: emit not even a pointer on an empty project; do not shrink when the pointer would be larger than a short body; and if source is absent, emit the normal list on the safe side.

The 30-entry cap is the next problem now

As of 2026-09-27, the project developing this plugin has 144 ADR files. Those whose current Status is Accepted or Amended numbered 107. The hook emits the first 30 in number order and warns that more remain. The 77 remaining do not enter the list.

The product where it started has 27 of 29 files in scope, and all can currently be shown. Even with the same implementation, coverage changes with project scale. The problem is not just that "there is a cap," but that "which decisions become invisible" is determined by number order. Because newer ADRs are further down, recent decisions are more likely to be dropped.

Another inconsistency remains too. The implementation's registration target is SessionStart, and the design record's Status is Amended. Meanwhile, the component README still says it runs every turn on UserPromptSubmit, and the upper-level document index still lists the Status as Accepted. The code and the descriptions are not updated in sync. In this article I treated the actual hooks.json and script as the current values.

If I fix it next, simply raising 30 is not enough. Injecting everything grows the startup context and runs counter to the compact countermeasure. A domain-based index, a rule that includes decisions with more recent change dates, and a method to mechanically extract candidates from work keywords need to be designed together with tests that ensure needed decisions are not mistakenly dropped.

The design boundaries I keep from this implementation

The judgment that remains from these four months is not to add more hooks.

Treat document storage, notification of existence, reading the bodies, and implementation compliance as separate stages.
Use hooks for processing that needs no semantic judgment, such as list generation or event firing.
Do not call a process where the model chooses "deterministic."
Measure not only injection volume but also firing frequency, process count, and re-injection after compact.
When you set a cap, verify coverage, not just the count.
Reconcile the current values of the code, the registration settings, the design record, and the README.

The mechanism that injects the index was an effective fix for the first accident. At the very least, it changes the state where the existence of design decisions is left entirely to the model's spontaneous directory exploration.

At the same time, it needs to be evaluated more narrowly than the first version's description. The current implementation does not read the bodies. It does not show anything past the 30th entry. It does not measure the violation rate. After compact it is only a pointer. Only by including these constraints does it become a design you can reuse.

Conclusion

When TipTap was proposed even though a document had adopted BlockNote, the problem was not whether the document existed. There was no processing to make that session aware of the decision's existence.

As a countermeasure, I generate the number, title, and status of the current ADRs within the project and inject them into the context from a Claude Code hook. It shifts from a path of waiting for the AI to notice, to a path that builds the list with the same script every time.

But what can be guaranteed goes only as far as delivery. The selection of the relevant bodies and compliance are verified separately. The per-turn resend was changed to SessionStart, the individual processes for 121 files were consolidated into a single awk, and after compact it was shrunk to a 262-byte pointer. And now, the cap that shows only 30 of 107 remains.

A design record is not finished the moment it is written. Only once you have designed how the next session discovers it, how many entries stay visible, and where the fact that a body was read is confirmed, does it become fit for operational use.