<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: AI Ferryman Alex</title>
    <description>The latest articles on DEV Community by AI Ferryman Alex (@alexlee7359).</description>
    <link>https://dev.to/alexlee7359</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4208273%2F64ef8a7c-6353-4f72-a242-a20ee1a1ae06.jpg</url>
      <title>DEV Community: AI Ferryman Alex</title>
      <link>https://dev.to/alexlee7359</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/alexlee7359"/>
    <language>en</language>
    <item>
      <title>I installed plannotator, sent 2 notes on a 3-line change: desktop apps have its headline features</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Sun, 15 Nov 2026 10:22:30 +0000</pubDate>
      <link>https://dev.to/alexlee7359/i-installed-plannotator-sent-2-notes-on-a-3-line-change-desktop-apps-have-its-headline-features-166i</link>
      <guid>https://dev.to/alexlee7359/i-installed-plannotator-sent-2-notes-on-a-3-line-change-desktop-apps-have-its-headline-features-166i</guid>
      <description>&lt;p&gt;I changed 3 lines of code in the Pico CSS repository, and then I used plannotator to put 2 line-by-line annotations on those changes. I sent both annotations back in one go, and the agent fixed the first one the way I had asked. I checked its fix myself afterwards. The comments arrived with line numbers attached, and the decisions stayed on record. On those two points, the tool did what it promises.&lt;/p&gt;

&lt;p&gt;Its headline features are a different story, because the Codex and Claude desktop apps already have built-in versions of them, so most people who work in just one desktop app do not need to install one more tool. What plannotator adds on top comes to 3 things: annotating files that are not code, an Inbox that several agents share, and an approval gate that scripts can check. All 3 sit in places the desktop apps do not reach.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it is, and how to install it
&lt;/h2&gt;

&lt;p&gt;Plannotator is a review workbench that runs in a browser on your own machine, and the version I installed in this round was v0.28.64. After an agent writes a plan or changes some code, plannotator opens a page locally. You annotate line by line on that page, and you approve the work or send it back. What you write returns to the agent with the file, the line number, and the original text of that line, and then the agent carries on working from your comments.&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/backnotprop/plannotator" rel="noopener noreferrer"&gt;github.com/backnotprop/plannotator&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I noticed the tool because it gained 412 stars in one day in the GitHub trending digest I read, and in ten rounds of that digest it reached No. 1 on the overall chart for the first time. When I went and checked the repository myself, the star count had already reached 10,775. A high star count only tells you that a lot of people are watching. Whether the tool is any good is a question you can only answer after you install it and run it.&lt;/p&gt;

&lt;p&gt;You install it with a single command that pulls a script down from its official site and runs it. The script installs a 205 MB binary into ~/.local/bin on your own computer, and that binary is the tool itself. If you add —minimal to the install command, it installs that one file and nothing else, and it leaves every other configuration file alone.&lt;/p&gt;

&lt;p&gt;A full install touches much more. In this round it changed my ~/.codex/config.toml and added a section at the end of the file that turns hooks on, and it created a new ~/.codex/hooks.json that holds one Stop hook. Every time the agent stops, that hook gets a chance to put the review page in front of me first.&lt;/p&gt;

&lt;p&gt;It also installed 5 skills into ~/.agents/skills and another 5 into ~/.claude/skills, and it placed 4 command files for OpenCode. On top of that, it downloaded a 67 MB sem sidecar that works out semantic diffs. I could check every one of these changes after the install, and the installer hid none of them. Still, before you install it, you should know that it connects at exactly the gate between you and your agent.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpkcp2jsze4wvp4eq2mq6.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpkcp2jsze4wvp4eq2mq6.webp" alt="The loop that carries comments back to the agent: you annotate line by line in the browser, send it back in one go, and the comments reach the agent with file and line numbers; the agent rules on each one before it edits, the fix is checked, and the decisions stay on record" width="800" height="390"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The code review round
&lt;/h2&gt;

&lt;p&gt;My test field was picocss/pico. On a local branch I changed 2 files, 3 lines in total: I added a line that caps the width of the tooltip style, I added a line with &amp;amp;:focus-visible to the display trigger so the tooltip also shows on keyboard focus, and I added a progress cursor line for buttons and links in the loading style. Each change filled a gap that really exists in that repository. I compiled the styles once after the edits, just to confirm I had not broken anything.&lt;/p&gt;

&lt;p&gt;After the edits, I ran plannotator review inside the repository, and that command waits where it is until the review is over before it exits. This round ran the review surface that you start by hand from the command line, because Codex was not logged in on the test machine, so I could not test the path where plannotator hooks in on its own during a real session. The page that opened in the browser showed a split diff. Its file tree said 2 files and 3 added lines, and that matched what git diff showed me, exactly.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjc53mn64eg19bz3c0v7o.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fjc53mn64eg19bz3c0v7o.webp" alt="Plannotator review page split diff: the file tree shows 2 files and 3 added lines, with the tooltip file changes on the right" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;My first annotation landed on line 27 of _tooltip.scss, on the max-width line I had just added. The problem was not that line on its own. The problem was that it fights with older settings in the same rule, because that rule already says white-space: nowrap, which forbids text wrapping, and it pairs that with text-overflow: ellipsis, which cuts off whatever does not fit and replaces it with dots.&lt;/p&gt;

&lt;p&gt;I had capped the width of the tooltip bubble, and wrapping was still forbidden, so a long tooltip would end up cut down to a string of dots, and the user would never read the full text. My annotation said one thing: either let the text wrap, or drop the width cap, because the two settings cannot both stay.&lt;/p&gt;

&lt;p&gt;My second annotation landed on line 44 of _loading.scss, on the cursor: progress line I had added. I asked for the progress cursor to cover more than buttons and links, because the ordinary busy containers under the general rule in _loading.scss should get it too, so that everything on the same page looks the same way while it is busy. After I had written both annotations, I clicked Send Feedback in the top bar and sent them together. The command exited right away, and the comments became its standard output, in JSON format.&lt;/p&gt;

&lt;p&gt;The comments that reached the agent carried the file path and the line numbers, with the line numbers split into an old side and a new side. The original text of the annotated line was in there, and so were 2 lines of context on each side of it. The end of the comments also carried a set of instructions that the tool appends on its own: the agent has to rule on each comment first and back each ruling with code evidence, it may not change any code before the rulings are settled, and it may only work on these comments instead of widening the review on its own.&lt;/p&gt;

&lt;p&gt;The agent ruled first, and only then did it touch the code. It ruled the first annotation Confirmed, because the evidence sat inside the same rule: max-width, nowrap, and ellipsis were all present at the same time, so long text was going to be cut off and there was no way around that.&lt;/p&gt;

&lt;p&gt;Its fix changed white-space: nowrap to normal, which lets the text wrap. After the fix I compiled the styles again, and the rule in the output now read white-space: normal. When I looked at git diff again, it showed 2 files, with 4 lines added and 1 line deleted. From the moment I wrote that annotation to the moment the fix landed correctly, I could reconcile every step.&lt;/p&gt;

&lt;p&gt;It ruled the second annotation Partly. Its reason was that an ordinary container is not an interactive element, so a progress cursor only shows when the mouse happens to hover over it, and the benefit stays small. That question later reached me formally through the Inbox, and I chose to keep the cursor on buttons and links only. So that item stays where the ruling left it, shelved, with no further change.&lt;/p&gt;

&lt;h2&gt;
  
  
  Inbox async questions
&lt;/h2&gt;

&lt;p&gt;The Inbox is a resident service that you start with plannotator inbox —background, and an agent can leave messages in it at any time. It can also leave structured questions that come with options. The question it left me in this round was the cursor question from the review: a single-choice question with 2 options, one of them marked Recommended, sitting under Waiting on you on the page until I dealt with it.&lt;/p&gt;

&lt;p&gt;I clicked Accept recommended, but at that point my chosen answer was still only a draft, and it had not really gone out. I clicked Send one more time, and only that click submitted it. After I submitted, I read the thread back from the agent side to check it: the question was marked as answered, my answer had entered the thread as a new message, and the body used a structured reply format that the agent can read directly in its next round.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpixqct75bt1b5ae8rpwx.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fpixqct75bt1b5ae8rpwx.webp" alt="Plannotator Inbox thread page: the question sits under Waiting on you, with one option marked Recommended" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This flow takes care of the waiting problem. The agent leaves its question and goes on with its own work, and it does not have to stop in place and wait for my reply. I do not have to sit and watch one session window either, and I can answer several questions together when I get to them. Once several agents are running at the same time, questions scattered across separate conversations and questions collected in one box are two very different things to search through.&lt;/p&gt;

&lt;h2&gt;
  
  
  The item-by-item comparison with the desktop apps
&lt;/h2&gt;

&lt;p&gt;On plan approval, both desktop apps have their own native approach. For the desktop side of this comparison, I read the official documentation from both companies, and I did not install either app and test it myself. Something being written in the documentation does not mean I have verified how it feels to use in practice. The Codex desktop app has a plan mode that breaks a task into a series of steps you can review before any work starts. It asks follow-up questions when something is unclear, and it only starts executing after you approve the plan.&lt;/p&gt;

&lt;p&gt;The Claude desktop app has a Plan permission mode, and it has a Plan pane as well. In that mode it only reads files and runs commands to look around, and it does not touch your code. After you have read the plan, you switch the mode over to execution, and only then does it carry on. Both apps keep the plan inside the conversation, and neither one offers line-by-line annotation on the plan. If you want to correct a plan, you type what you want changed into the conversation and tell the agent there.&lt;/p&gt;

&lt;p&gt;On line-by-line annotation, the built-in version in the Claude desktop app has almost the same shape as the one in plannotator. In the diff view of the Claude desktop app, you hover over a line and click the + button to write a comment, and after you have written comments on several lines, you submit them all at once. The official documentation says, in its own words, that Claude reads your comments and makes the changes you asked for. You click a line, you write a comment, you send them back together, and the agent fixes the code. That loop is already built into the desktop app.&lt;/p&gt;

&lt;p&gt;The Codex desktop app covers more ground on review. It has a review pane where you can comment on a diff, stage changes, and accept or roll back changes hunk by hunk. The desktop app also has a Code Review plugin built in, which puts the PR description, the changed files, the comments, and the check status on one screen. The plugin also has a PR inbox that you can filter by assignee or by team, or narrow down to the PRs you wrote yourself.&lt;/p&gt;

&lt;p&gt;After a review, you click Submit review, and a Comment, an Approve, or a Request changes goes straight to GitHub, and the app also runs automatic review in the cloud on top of that. So when you set the 2 headline features side by side, desktop users are not missing anything. You write your code inside the Codex or Claude desktop app, and then you install a 205 MB binary on top, and you let it change the hook configuration of your agent. What you get back for that is plan approval and line-by-line annotation, and you already had both.&lt;/p&gt;

&lt;p&gt;The things it adds, the ones you cannot find in the official documentation of either desktop app, come to 3. The first one is annotating things that are not code: a plan file or a document that the agent wrote can be opened, annotated passage by passage, and sent back. Plannotator says that web pages and the previous reply from an agent can be annotated the same way, but I did not test either of those 2 in this round.&lt;/p&gt;

&lt;p&gt;The plan file I tried in this round is called plan-tooltip-loading.md. It describes the 2 changes, one for the tooltip and one for the loading style, and it has a risks section. Line 18 is the first item in that risks section, and my annotation landed on that item. The line number the annotation marked matched the actual line number in the file, and the feedback that went back kept that same line number.&lt;/p&gt;

&lt;p&gt;The second extra is the Inbox, and it does not care which vendor your agent comes from. In this single install, Codex, Claude, and OpenCode all ended up connected to the same binary, and they use the same annotation format. On the Claude side I only verified the install layer, and I did not run a real session with it. The session management and the inboxes in the Codex and Claude desktop apps each look after their own agent only. A review surface and a question box that several agents share is something only plannotator does right now.&lt;/p&gt;

&lt;p&gt;The third extra is an approval gate for scripts. When you add —gate to the annotate command, an Approve button appears in the top bar of the page, and when you combine that with —require-approval and —result-file, the act of approving turns into a result that leaves a record. In this round I clicked Approve, the result file was written on the spot, and the exit code of the command was 0. A script in a pipeline can read that exit code directly and tell whether a person approved, and nobody has to rely on remembering whether they approved at the time.&lt;/p&gt;

&lt;h2&gt;
  
  
  What failed to run, and what grates
&lt;/h2&gt;

&lt;p&gt;Automatic triggering of plan review is one of its headline features, and I did not get it to run in this round. In its own design, the hook is supposed to bring the review page up on its own when the agent leaves plan mode, and the person does not have to type any command.&lt;/p&gt;

&lt;p&gt;The real-session path broke at login, because Codex was not logged in on the test machine. So I switched to a manual approach: I built a piece of data in the shape of a Codex Stop event, and I fed it into the hook entry point of plannotator, to see whether plannotator would accept it.&lt;/p&gt;

&lt;p&gt;What it gave me back was nothing at all: exit code 0, no output of any kind, and no new session. It let the event pass in silence. The harder part is that there is nowhere to look for a reason, because there is not even a log directory under ~/.plannotator. I had no way to tell whether the format of my data was wrong, or whether plannotator applies its own separate standard for what counts as a plan. One more thing on the side: the source labels it sticks on sessions are wrong too. Every session in this round was started from a bare command, and the records call all of them claude-code.&lt;/p&gt;

&lt;p&gt;The first time you open the review page, you also have to clear some things away before you can start reviewing. A new user runs into 4 layers of popups: one layer asks you to pick a display style for plans, one layer promotes an experimental feature, and one layer introduces Guided Reviews. The remaining layer promotes the Inbox, and it carries a video with it. In a different session, the same layers show up again in a different order. After you clear the popups, the main area is still empty by default, and the diff only loads after you click a file in the file tree on the left.&lt;/p&gt;

&lt;p&gt;The risk in remote mode is a problem I actually found while testing in this round. Once you turn PLANNOTATOR_REMOTE=1 on, the tool binds its service to port 19432 on 0.0.0.0. I tried opening the review page from the same machine with no password at all, and the server answered with a 200, and the page asked for no login check of any kind. The tool also generates a share link with a long token in it, and the diff is packed inside that link, so if the link leaks, what leaks with it is your code changes.&lt;/p&gt;

&lt;p&gt;So on a remote machine, the right way to use it is to put the port inside an SSH tunnel, or to use the tailscale route that comes with the tool, and not to open 19432 directly toward a network you do not trust. I did not get the tailscale route working in this round either, because tailscale was not logged in on this machine. When plannotator fails on that route, at least the error it gives back is clean: it tells you plainly that tailscale is not logged in, and it tells you how to run in local mode instead.&lt;/p&gt;

&lt;p&gt;There are 2 smaller items on the bill. One is the fidelity of the annotate archive: the original passage that an annotation points at gets cut off in the archive, and the cut lands in the middle of a word, so later on, if you want to locate an annotation in the original text from the archive, you have to work from the passage number and the surrounding context.&lt;/p&gt;

&lt;p&gt;The other item is team sharing. The tool has guide share, which can turn annotations into a link that you send to teammates, but I did not test it in this round, because generating a link sends content out to an external service, and I have no conclusion here about how it works in practice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who should install it, and who can skip it
&lt;/h2&gt;

&lt;p&gt;If you only use one desktop app, Codex or Claude, my advice is the same for both: do not install it, unless you know you want at least 1 of the 3 extras I listed above. Those extras are annotating plans and documents, collecting questions in the Inbox, and turning approval into a gate that a script can check. If you would use none of the 3, do not add this layer to your setup. Its limits are plain enough: it cannot stand in for the whole desktop ecosystem around PRs, CI checks, and automatic merging.&lt;/p&gt;

&lt;p&gt;If you mostly write code from the CLI in a terminal, or if you have several agents running at the same time, then this layer is worth installing. It gives you things the CLI world did not have: a proper split-screen diff page, comments that go back to the agent with accurate line numbers and the original line text, and review and annotation decisions that stay on record in ~/.plannotator in 2 archive copies, one in JSONL and one in Markdown. A gate approval, by contrast, leaves only a single line in the main ledger.&lt;/p&gt;

&lt;p&gt;In this round I went through the 2 annotations, the 1 gate approval, and all of the Inbox questions and answers one by one, and the files, the line numbers, and the content lined up with no mismatches. The thing it does not solve needs to be said up front: it does not judge for you whether the agent wrote the code correctly. How accurate your annotations are, and how carefully you read during a review, still comes down to the person.&lt;/p&gt;

&lt;p&gt;The only 2 things it can promise are that comments get delivered to the right place, and that comments stay on record. If you have already stopped reading the changes your agent makes line by line, then no review tool you put in front of yourself is going to get opened.&lt;/p&gt;

&lt;p&gt;My own way of using it is settled now: when several agents run together, I will collect their questions into its Inbox, and I will run plan files through the gate before I let the work proceed. In a session where I use a single desktop app, I will not take this detour, and I will review right inside the desktop app. If this piece saved you an install and uninstall round trip, please bookmark it and subscribe to this blog. I will keep installing tools built for agents and running them for real, and I will write up what worked and what did not, exactly as it happened.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>softwaredevelopment</category>
      <category>tools</category>
    </item>
    <item>
      <title>DeepSeek opens search, file and voice APIs: cheaper transcription than OpenAI, no image generation</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Sun, 15 Nov 2026 10:20:31 +0000</pubDate>
      <link>https://dev.to/alexlee7359/deepseek-opens-search-file-and-voice-apis-cheaper-transcription-than-openai-no-image-generation-2o6p</link>
      <guid>https://dev.to/alexlee7359/deepseek-opens-search-file-and-voice-apis-cheaper-transcription-than-openai-no-image-generation-2o6p</guid>
      <description>&lt;p&gt;On November 13, DeepSeek posted four times in a row on X and announced a batch of new APIs. Developers can now call the search, file parsing, URL reading and voice capabilities that were already in use inside the DeepSeek App, and they can call them in two ways. They can turn on a built-in feature inside a chat request, or they can call each capability on its own as a separate API. The built-in list has eight features in total, and it starts with web search, where the answer comes with citations.&lt;/p&gt;

&lt;p&gt;File parsing handles images and files, and handwritten content is inside its scope as well. The URL reading feature fetches a page first, and then the model reasons over what the page says. Three more features are about the conversation itself, because the system generates a title after the model answers, a safety check looks for risks in the user input and the model output, and user metadata markers pass time zone, place and language to the model, so the model can use that information when it answers.&lt;/p&gt;

&lt;p&gt;The last two features are deep search and voice. Deep search means that the model searches the web and reads links on its own while it reasons through several steps, and, as DeepSeek’s documentation is reported to say, it calls tools about 20 times at most in one run. Voice has two parts, because the system turns speech into text after the user speaks, and then it reads the text aloud after the model answers.&lt;/p&gt;

&lt;p&gt;If a developer wants to turn on a built-in feature or deep search, then the model has to be DeepSeek V4.1-Flash, and the model name written in the interface is deepseek-flash. The parameters still go through when the wrong model is used, but the feature does not turn on. So you should confirm that you are calling this model before you start the integration work.&lt;/p&gt;

&lt;h2&gt;
  
  
  Native support in the main model and separate platform APIs
&lt;/h2&gt;

&lt;p&gt;When you put DeepSeek, OpenAI and Gemini side by side, you first have to get the basis of the comparison straight, because the answer changes when the basis changes. If you only ask whether one main model can take voice in and send voice out by itself, then the three companies give different answers. If you ask whether the whole API platform offers the capability, then all three have a matching service. So the comparison below uses the platform basis, and I will mark the places where the main model supports something natively.&lt;/p&gt;

&lt;h2&gt;
  
  
  Capabilities OpenAI and Gemini already had
&lt;/h2&gt;

&lt;p&gt;OpenAI’s Responses API already had a built-in web search tool, and it appeared earlier than this DeepSeek announcement. OpenAI’s main model can take image input directly, so the image reading item does not need a separate parsing service. Image generation has its own gpt-image family of APIs, and that family can also be called as a built-in tool inside Responses, which means that the platform covers both reading images and making new ones inside the same surface.&lt;/p&gt;

&lt;p&gt;Voice is the item that people often get wrong, because many of them mix up OpenAI’s main model with OpenAI’s platform. The platform has a separate transcription API, and the model recommended now is gpt-transcribe, at $0.0045 per minute, which works out to $0.27 per hour. That transcription API handles recorded audio files, and it also offers live transcription. The platform also has a text-to-speech API, and its models include tts-1 and gpt-4o-mini-tts, so speech goes in both directions on the platform.&lt;/p&gt;

&lt;p&gt;Alongside transcription and text-to-speech, OpenAI also has the Realtime API, and it can hold a conversation with voice in and voice out directly. There is also a date to plan around, because tts-1 and gpt-4o-mini-tts have been announced for retirement on January 6, 2027, and work will move to Realtime after they retire. So anyone who integrates them now should count that date in from the start, instead of finding it later.&lt;/p&gt;

&lt;p&gt;Gemini differs from OpenAI again, because Gemini’s official model documentation lists text, image, audio, video and PDF among the input types of its main model. That means that a user can send a piece of voice audio, and the main model can take it directly, without a separate transcription step first. For search, Gemini uses Google Search grounding, which connects the model’s answer to Google search results. For drawing, Gemini has the Nano Banana family of image generation models.&lt;/p&gt;

&lt;p&gt;For speech generation, Gemini has its own family of text-to-speech models, and its Live API can hold a conversation with voice in and voice out. So if you only look at the range that the main model supports natively, then Gemini’s main model can take more kinds of input directly than the main models of the other two companies, because audio is already one of the input types of its main model.&lt;/p&gt;

&lt;h2&gt;
  
  
  DeepSeek pricing and limits
&lt;/h2&gt;

&lt;p&gt;DeepSeek charges the same price for search and for URL reading, at $0.75 for every thousand calls. OCR is charged by the page, at $0.225 for every thousand pages, and OCR recognizes text inside images and files, so it pulls words out of a page. It does not generate a new image by itself. Search and URL reading share one price, while OCR uses a different unit, because search and reading are counted by calls, while OCR is counted by pages.&lt;/p&gt;

&lt;p&gt;Speech-to-text (ASR) is charged by the hour, and its price is now $0.058 per hour, because it is at a limited-time half price, so the original price is $0.116 per hour. Text-to-speech (TTS) is charged by the number of characters, and its price is now $0.225 for every ten thousand characters, while the original price is $0.45 for every ten thousand characters, and it is also at a limited-time half price. Both discounts are time limited, so the original prices are the ones to keep in view.&lt;/p&gt;

&lt;p&gt;You can do the arithmetic on speech-to-text. DeepSeek charges $0.058 per hour, while OpenAI charges $0.27 per hour, so the OpenAI price is about 4.7 times the DeepSeek price. Search is a different case, because I did not check the official price lists for the matching OpenAI and Gemini tools item by item in this round. So for search I only give DeepSeek’s own unit price here, and I do not compare it with the other companies.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fobcu03rfefadm01extkx.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fobcu03rfefadm01extkx.webp" alt="API comparison: DeepSeek vs OpenAI vs Gemini across six capabilities. DeepSeek has no image generation; speech-to-text is $0.058 per hour against OpenAI at $0.27 per hour, about 4.7 times cheaper" width="800" height="636"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Image generation is the one item that only DeepSeek lacks among the three, because OpenAI has the gpt-image family and Gemini has the Nano Banana family. If an app needs a user to upload an image and then needs a new image generated from it, then this batch of DeepSeek APIs cannot cover that step. The developer still has to connect an image generation service from another company for that part of the product.&lt;/p&gt;

&lt;p&gt;The limits on speech-to-text are more specific. DeepSeek’s ASR allows 120 seconds at most for one segment, the file can be 8MB at most, and it does not support streaming transcription, while it supports 21 languages. Because of these three limits, it fits short speech clips, which is the use that its numbers point to. But when a user speaks into a microphone continuously and expects text to appear while they speak, then its current use differs from OpenAI’s live transcription and from Gemini’s native audio input.&lt;/p&gt;

&lt;p&gt;On the text-to-speech side, DeepSeek offers 2 voices and 10 languages, and it can stream its output over WebSocket. OpenAI’s matching models have 11 to 13 voices, so if an app needs to let users pick a voice, then the choice that DeepSeek can offer right now will be much smaller, because two voices cannot cover the same range of preferences as eleven to thirteen voices.&lt;/p&gt;

&lt;h2&gt;
  
  
  DeepSeek packaging and chat-demo
&lt;/h2&gt;

&lt;p&gt;DeepSeek also differs from the other two in one specific part of its approach, and the difference sits in how you connect to it. OpenAI puts its built-in tools mainly inside the Responses API, while Gemini uses the generateContent interface with grounding and tool calls. Those two existing setups matter here, because DeepSeek’s choice only stands out when you set it against the way the other two already expose their tools.&lt;/p&gt;

&lt;p&gt;DeepSeek turned the search, file parsing and voice work that was already in use inside its App into parameter switches, and it added those switches to its existing Messages, Responses and Chat Completions endpoints, where the server runs them. Alongside the built-in route, it also made the same batch of capabilities into separate APIs, so a developer can call only one of them, and the developer does not have to move the whole chat flow over to get that one capability.&lt;/p&gt;

&lt;p&gt;The company also open-sourced a front-end demo at the same time, and its name is chat-demo. The demo supports importing DeepSeek conversation records into it, and two days after it was open-sourced, the repository had gathered 1,507 stars. That count is the total from those first two days alone. The official claim is that a developer who follows this demo can rebuild the DeepSeek App inside their own app.&lt;/p&gt;

&lt;p&gt;That claim is DeepSeek’s own wording, and I have not installed this demo myself, so I will not draw a conclusion here about how far it can actually reproduce the App. The demo itself is sitting there, and developers can read its code themselves, and then they can judge how far it still is from an app that they could use directly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cost calculation and checks before integration
&lt;/h2&gt;

&lt;p&gt;Tool fees and token fees are charged separately, and that holds for all three companies, because OpenAI’s price list also says that models with tools charge extra for the tools. So the number a developer has to work out is how many searches they will call each month, how many pages of files they will read, and how many hours of speech they will transcribe, and then they multiply each count by its unit price and add the result to the token fee. If you only look at the token unit price, then you will leave the tool fee part out of the total.&lt;/p&gt;

&lt;p&gt;I usually have an agent read web pages and read files for me, so when I look at a price list like this, I estimate the call counts first, because search and file reading both build up by calls and by pages. Before you integrate, you can work through one week of real records once, and then you will know roughly how much of the total cost the tool fees will take, and you can decide which features stay on by default and which ones wait until the user clicks.&lt;/p&gt;

&lt;p&gt;If you’re building a chatbot or an agent and you’re choosing which provider to use for search, files and voice for the work you are doing, then please collect this post and subscribe to the blog. I’ll keep publishing comparisons like this one, where I put a few providers side by side, check the numbers first, and write down where each one falls short.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>api</category>
      <category>llm</category>
      <category>news</category>
    </item>
    <item>
      <title>Claude Code reviewed its own work — out of 125 suggestions, only 2 actually worked</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Sat, 14 Nov 2026 13:12:07 +0000</pubDate>
      <link>https://dev.to/alexlee7359/claude-code-reviewed-its-own-work-out-of-125-suggestions-only-2-actually-worked-di</link>
      <guid>https://dev.to/alexlee7359/claude-code-reviewed-its-own-work-out-of-125-suggestions-only-2-actually-worked-di</guid>
      <description>&lt;h2&gt;
  
  
  This is how most people use agents now
&lt;/h2&gt;

&lt;p&gt;Most people now work with an agent the same way. The agent drifts off course while it works, you point it out, and it corrects itself. Getting it exactly right the first time is basically too much to ask, because an agent isn’t a wishing machine.&lt;/p&gt;

&lt;p&gt;Once the job is done, you ask it to sum things up. It goes back over the run and lists, one by one, where it went wrong and where it got stuck. What comes out of that retrospective usually goes straight into the project configuration, or into the agent’s own core constraints file, so the next job starts with those lessons already loaded. Some people go one step further and distill those notes into a skill.&lt;/p&gt;

&lt;p&gt;The point is straightforward: don’t make the same mistakes next time. When the next project or the next task comes along, the agent should take fewer detours and do the job more accurately, with less friction.&lt;/p&gt;

&lt;h2&gt;
  
  
  Someone put the retrospective to the test
&lt;/h2&gt;

&lt;p&gt;The habit sounds so reasonable that almost nobody stops to ask whether it actually works. Recently, though, someone actually tested whether it works. The result didn’t match what most people would expect.&lt;/p&gt;

&lt;p&gt;Two researchers, Treude and Baltes, first had Claude Code run 30 bug-fixing tasks, and then had it look back and review its own work. The retrospective asked just one question: what needs to change in the setup that runs the agent? Claude Code came up with 125 suggested improvements in total.&lt;/p&gt;

&lt;p&gt;The tool they used for the retrospective is called /retro. After a task finishes, it has the agent read the whole session back from the start and propose changes to the harness (the stuff that runs the agent: the software, the configuration, and the instructions given to the agent). Changing the harness means changing the setup the agent starts with on its next task.&lt;/p&gt;

&lt;p&gt;The experiment used 30 tasks from SWE-bench Verified, all of them real issues from real repositories. The 125 suggestions were first classified one by one. The researchers picked 24 of them and actually put them in place. Then they picked 12 of those 24 and put them to a controlled test on later tasks.&lt;/p&gt;

&lt;p&gt;The test was a comparison, run separately for each change. They picked a follow-up task from the same repository as the task that had produced the change, and opened two brand-new sessions for it. One session had that change in place and the other didn’t. They then compared the two sessions. The question was whether the session with the change did less of the behavior that the change was supposed to eliminate.&lt;/p&gt;

&lt;p&gt;Of the 12 changes taken into the test, 5 could not be tested at all. In the control sessions, run without the change, the problem the change was meant to fix never came up. The control sessions are the only reason anyone knows this. Without them, the 5 changes would have finished their test runs with no problem ever showing up. That clean run would have been taken as proof that the changes worked. In fact, the 5 were never tested at all: the problem never came up, so no comparison ever happened for them. Of the remaining 7, 2 eliminated the behavior completely. One more made the behavior show up less often, and the other 4 changed nothing at all.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw6ht25nz48leztl0qkap.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fw6ht25nz48leztl0qkap.webp" alt="The funnel of the 125 suggestions: 125 proposed, 24 put in place, 12 taken into a controlled test, 7 where the control sessions actually ran into the problem, and 2 that eliminated the problem completely" width="800" height="833"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;So, back to whether it actually works: retrospective summaries aren’t completely useless. Of the 12 changes taken into the test, 2 really did eliminate the problem completely, and 1 more made it show up less often. The other 9 of the 12 either changed nothing or were never tested at all. That was the yield from 125 suggestions the agent produced.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why most of the 125 suggestions didn’t help
&lt;/h2&gt;

&lt;p&gt;The quality of the suggestions themselves is also a problem. Of the 125, 43 were about lint, and 18 of those recommended the same practice no matter what had actually happened in the task. Suggestions like that don’t grow out of evidence. The agent is just restating an approach it already believed in.&lt;/p&gt;

&lt;p&gt;Some changes made things worse. In their paper, Treude and Baltes report that some changes actually made the agent do extra work on unrelated tasks. Once a change is in place and nobody tests it again, it just stays there. From then on, every time the agent runs a task, it does a little extra work it didn’t need to do.&lt;/p&gt;

&lt;p&gt;The problem isn’t how many suggestions the agent makes. The problem is that the agent reviews its own just-finished task. It sees that task only from its own point of view. Wherever it got stuck this time, it takes that spot to be the thing to change next time. But where it got stuck and what actually needs changing are two different things.&lt;/p&gt;

&lt;p&gt;Without a control group, the agent can’t tell two things apart: whether the change really fixed the problem it was meant to fix, or whether that problem was never going to show up again anyway. Reviewing yourself, by yourself, can’t separate those two. Of the 12 changes above, 5 landed exactly in this blind spot. The problem never appeared in their control sessions, so there was nothing to compare, and they were never actually tested.&lt;/p&gt;

&lt;p&gt;The 43 lint suggestions among the 125 included 18 that recommend the same practice no matter what happened in the task. Those 18 are boilerplate. A suggestion that would show up word for word on a different task has nothing to do with what actually happened in this one. Save a suggestion like that as experience, and next time you will simply read back the same ready-made line. The evidence from this task never got saved along with it.&lt;/p&gt;

&lt;p&gt;Nothing in this workflow ever removes a change, so the changes pile up. The more they pile up, the further the agent drifts from how it originally worked. And every untested change you add is one more chance for unrelated tasks to pick up extra work.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to do it right
&lt;/h2&gt;

&lt;p&gt;Step one: go through the retrospective yourself first. If you have development experience, a quick scan is enough to spot the ones that are off: the ones that are plainly wrong, the ones that repeat something already said, and the boilerplate that would hold on any task. Delete those on the spot instead of saving them for testing later. This step runs on your own experience, it takes a few minutes, and what survives it is what actually gets tested.&lt;/p&gt;

&lt;p&gt;Step two: hand the surviving suggestions to a new agent to test. Open a brand-new session, in the same project as before. Write the surviving suggestions into the project configuration for this test, have the new agent read them first, and then have it redo the job you just finished. The agent does the running. You don’t have to sit beside it keeping records, and you don’t have to wait for the next task to show up.&lt;/p&gt;

&lt;p&gt;Step three: look at how the redo went. You can see at a glance whether the redo repeats the mistakes from the first run, the ones you had to point out while the agent was working. If it doesn’t repeat them, something among the surviving suggestions does work. If it repeats them anyway, the surviving suggestions didn’t do anything. When you can’t tell, open another session without the suggestions and run the same job in it. Comparing that run with the redo settles whether the suggestions did anything. That two-run comparison is the same one the paper used, except that the paper ran its comparisons on later tasks instead of on a redo of the same job.&lt;/p&gt;

&lt;p&gt;Step four: decide what stays and what goes, based on the result. If the mistakes didn’t come back in the redo, the suggestions you tested have earned a permanent place: leave them in the project configuration, or move them into the agent’s core constraints file, and keep them there for the long term. If the mistakes happened anyway, delete those suggestions instead of letting them sit in the configuration taking up space. And if you want to know which single suggestion did the work, run the same test on each suggestion separately. That is exactly how the study tested its own changes, one by one.&lt;/p&gt;

&lt;p&gt;When the next job comes along, it runs under the constraints that survived. The agent then reviews that job and produces a new retrospective. You delete the plainly wrong, the repeated, and the boilerplate from the new retrospective, and you test what remains by redoing the job. Round after round, the constraints get cleaner. In the end, what is worth distilling into a skill is the set of constraints that is still standing after several rounds of this loop.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fb7u5thr26jlvvfeg7u60.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fb7u5thr26jlvvfeg7u60.webp" alt="The self-improvement loop: the retrospective proposes suggestions, you delete a round yourself first, then a new agent reads the survivors and redoes the job; if the mistakes come back the suggestions are deleted, if they don't they stay written into the project configuration or the agent's core constraints file; carry the survivors into the next job and go round again, and only what survives the loop gets distilled into a skill" width="800" height="1040"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Distilling into a skill comes last
&lt;/h2&gt;

&lt;p&gt;A batch of suggestions has to be tested this way, round after round, with the redo confirming each time that the mistakes don’t come back. Only then should anything from that batch be written into a skill. When you write them down, note that redo alongside them: where the mistake originally happened, and whether it came back in the redo. Whoever picks the skill up later can then see directly why each suggestion earned its place.&lt;/p&gt;

&lt;p&gt;The raw text the agent wrote during its retrospective does not go into the skill as a whole. Mixed into that raw text are judgments that were never tested, along with boilerplate that would hold on any task. What belongs in the skill is a few lines picked out of the raw text, the ones that went through the deletion pass in step one and were verified by the redo in the steps after it. When those lines go in, they should be rewritten as rules and carry the record of that redo with them. Copy the whole raw text in, and you have copied the untested parts in with it.&lt;/p&gt;

&lt;p&gt;The paper’s conclusion at the end makes the same point: generating changes and putting them in place is only the first step. You also need a way to know whether a change actually helped on later tasks, and a way to fix or remove the changes that didn’t help.&lt;/p&gt;

&lt;p&gt;If you also have your agent review and summarize its own work, and you want to distill those summaries into a skill, the redo test described above can be applied directly to your own setup. Please like this post and give me a follow. I will keep publishing hands-on tests like this one.&lt;/p&gt;

&lt;p&gt;Source: Treude and Baltes, “How Do Coding-Agent Harnesses Self-Evolve?”, arXiv:2610.34938, &lt;a href="https://arxiv.org/abs/2610.34938" rel="noopener noreferrer"&gt;https://arxiv.org/abs/2610.34938&lt;/a&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  AIAgents #LLM #AIEngineering
&lt;/h1&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>claude</category>
      <category>llm</category>
    </item>
    <item>
      <title>Muse's 30 Billion Tokens Don't Last Nearly as Long as They Sound</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Fri, 13 Nov 2026 23:18:25 +0000</pubDate>
      <link>https://dev.to/alexlee7359/muses-30-billion-tokens-dont-last-nearly-as-long-as-they-sound-1c4n</link>
      <guid>https://dev.to/alexlee7359/muses-30-billion-tokens-dont-last-nearly-as-long-as-they-sound-1c4n</guid>
      <description>&lt;p&gt;On the morning of November 9, I opened Muse’s usage dashboard. I had 999M extra tokens left, and the screen said 75% used. Used 75% means 25% left, so I divided 999M by 25%, and the whole pool worked out to about 4,000M, which is about 4 billion.&lt;/p&gt;

&lt;p&gt;The 30 billion in the title is the cap on invite rewards. Muse gives every user an invite code. When someone signs up and enters that code, I get another 1 billion tokens for each signup. The code can be used at most 30 times, and only if all 30 uses are filled does the total reach 30 billion. What I actually got is the roughly 4 billion above, and most of it is already burned.&lt;/p&gt;

&lt;p&gt;These readings are Muse’s own dashboard reporting on quota. They are not an official bill, so the burn rates and balances below all use the dashboard’s measure. This pool dropped fast. On the morning of the 10th, 875M was left, and by the evening it was 793M. On the afternoon of the 11th it was 704M, and by noon on the 12th only 642M was left. In three days, 357M burned away. During those three days, my weekly quota was already full. The weekly quota is the amount that resets every week, and it is accounted separately from the extra pool. Once it is full, new spending is charged to the extra pool.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Have Muse Do Every Day
&lt;/h2&gt;

&lt;p&gt;Drafts are one pile of work. Every morning, Muse picks the material first. Once it has picked it, I write the first draft. When the draft comes back to it, it polishes and reviews it, and then adds the images. After that, it publishes to the blog, X, Xiaohongshu and dev.to. Running the X account is another pile, and on my side I call it X patrol. It pulls the timeline all day and replies to people, a few dozen replies a day, and likes and follows are done inside the same workflow.&lt;/p&gt;

&lt;p&gt;Scheduled jobs are one pile: the topic digest, the reply self-check and the GitHub trending daily scan, the jobs that run on a schedule. The rest is ad hoc work, tasks I only assign when they come to mind. Investigating a problem, writing code and working on projects all sit in this pile.&lt;/p&gt;

&lt;h2&gt;
  
  
  How Much Each Kind of Work Burns Per Day
&lt;/h2&gt;

&lt;p&gt;These piles run all day without stopping, and I have split the accounts two ways. One is the dashboard measure, the amount the pool actually deducts. On the 10th it averaged about 5.8M per hour, about 140M in a day. The other is the raw volume logged internally per agent, about 5 to 7 times the dashboard volume, because the internal log counts context replayed over and over. The breakdown below uses the dashboard measure. These figures are magnitudes converted from each item’s share of a measured period, so they do not reach billing precision, but their relative sizes are right. When I talk about mechanisms, I switch back to raw volume. The two numbers each cover their own thing, and I do not mix them.&lt;/p&gt;

&lt;p&gt;The most expensive pile is X patrol. On its most expensive days, it took about half of the whole account’s raw volume. Applying that share to the day’s roughly 140M in dashboard volume gives about 70M in a day. The agent that writes the replies, which I call the consumer, works one shift every hour. On its most expensive shift, it dispatched 125 browser subtasks at once, 65 in English and 60 in Chinese. All those subtasks did was send out a reply that was already written. That shift’s raw volume was 144.9M.&lt;/p&gt;

&lt;p&gt;What makes it expensive is the startup cost. Before a new agent can start, it first has to carry about 88,000 tokens of routine context, and I call this starting expense the startup cost. Two parts make up most of that context: about 55,000 tokens are the product’s fixed part, and about 27,000 are my own standing files. Sending one reply opened one complete browser agent, so the startup cost was charged once per reply. Stacked across 125 replies, the startup cost alone came to about 11M in raw volume.&lt;/p&gt;

&lt;p&gt;On the morning of November 10, across 8 hours, the draft pile was assigned about 31M, half of the whole account’s dashboard volume for those 8 hours. Most of it sat in one long chat used for revising drafts. That chat had run for days, and material selection, polishing and review were all done inside it. By the morning of the 10th, that chat plus the subtasks it dispatched, counted as a whole tree, took about 40% of the whole account’s raw volume. Its raw volume was 124.7M, and it had 63 subtasks in total. Revising and reviewing a few articles back and forth burned over 100M tokens in raw volume alone.&lt;/p&gt;

&lt;p&gt;The reason it is expensive is that its history is too thick. Every step that chat takes forward, it has to reread the records of the previous days. Each subtask it dispatches has to carry that same history again when it starts. Even when everything goes smoothly, one draft, from reading the material at the start to finishing the review, eats 8M to 13M in raw volume in a single session. At the start it reads in material it will use and material it will not use, and the unused part is paid for all the same.&lt;/p&gt;

&lt;p&gt;The topic digest among the scheduled jobs ran once an hour back then, and each round dispatched a browser to search X for 10 sets of posts. On the morning of the 10th, just 2 rounds were assigned about 3M, and scaled to 24 rounds a day that is a magnitude of over 30M. The nightly reply self-check had to fetch all of the original posts behind the previous day’s roughly 100 replies again, and review them once more.&lt;/p&gt;

&lt;p&gt;The collector is another kind of scheduled job. It collects the timeline once every 20 minutes. I used to run this kind of fixed, repeated work with an agent scheduled job too. It ran 31 rounds, and every round was an agent running. The 31 rounds actually ran for about 2.4 hours added together, and all of that time was spent in agent tokens.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Root Cause: Re-reading and Startup Cost
&lt;/h2&gt;

&lt;p&gt;After splitting the piles above, the common root cause is clear. The biggest chunk of this bill is context being reread over and over. I checked the internal records for two measured periods, and by raw volume, output was only about 1% and input about 99%. The input burns context, because every step an agent takes forward, it rereads the earlier conversation and files. The amount read is two orders of magnitude larger than the amount written.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fal9kyar2izz4tacuq2rn.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fal9kyar2izz4tacuq2rn.webp" alt="The biggest chunk of this bill is context being reread over and over: by raw volume, output is only about 1% and input about 99%" width="800" height="283"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;One startup cost is about 88,000 tokens, and the number of charges follows the number of agents. In the period I measured (I will call it a window below), the window on the morning of the 10th opened 273 agents across the whole account, and the window on the 11th opened 480. For 480 agents, the startup cost alone came to about 42M in raw volume. In the window after the fixes, only 242 agents opened, and this startup cost fell by about half along with them.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5i8vkb0bkh2ojj1sag84.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5i8vkb0bkh2ojj1sag84.webp" alt="Each new agent's startup cost is about 88,000 tokens: about 55,000 for the product's fixed part, about 27,000 for my own standing files, and about 6,000 for the rest" width="800" height="283"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Adding It All Up
&lt;/h2&gt;

&lt;p&gt;The fastest-burning stretch was the morning of the 10th: 63M burned in 8 hours, an average of 7.9M per hour. Averaged across the whole of the 10th, it was about 5.8M per hour. The window on the 11th dropped a little further, to 5.15M per hour. After the fixes were done, the 22.9 hours from the afternoon of the 11th to noon on the 12th burned only 62M, about 2.7M per hour, roughly half of the 11th window’s rate.&lt;/p&gt;

&lt;p&gt;At the fastest burn, a pool of about 4 billion runs out in 21 days. Each step the burn rate drops, the lifespan goes up: the tier at 5.8M per hour lasts 29 days, and the tier after the fixes, at 2.7M per hour, lasts 62 days. The three lifespans use the same pool; the whole difference is in how it is burned. Swap the pool for the capped 30 billion and the math does not change: about 158 days at the fastest tier, about 216 days at the middle tier, and about 463 days at the tier after the fixes. The cap converted into days is a few months to a little over a year.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6lsutxn456cmeovja2vt.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6lsutxn456cmeovja2vt.webp" alt="How long a pool of about 4 billion lasts at three burn rates: 21 days at the fastest tier, 29 days at the middle tier, and 62 days after the fixes" width="800" height="348"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Of the 642M left at noon on the 12th, the fastest way of burning would finish it in just over 3 days, and the way after the fixes would stretch it to about 10 days. Because the weekly quota does not reset until November 15, those 10 days are the boundary for whether I still dare to keep assigning work. If the burn went back to the old way, I would have to stop and wait before the reset.&lt;/p&gt;

&lt;p&gt;Meta’s public Help Center page lists Power at $20 a month with 500M a week, and Maximum at $100 a month with 3,000M a week. The dashboard in my account quotes $16 and $80 a month. The prices under the two measures do not match, so I do the accounts at the public prices, and at my account’s quoted prices it would come out another 20% cheaper.&lt;/p&gt;

&lt;p&gt;At public prices, a pool of about 4 billion comes to about 1.85 months of Power quota. Power’s average monthly quota is 500M a week times 52 weeks divided by 12 months, about 2,167M, so 4,000M converts to about $37. A pool that sounds huge is, converted into a subscription, less than two months of quota. That is the number I started calculating down from when I wrote this piece.&lt;/p&gt;

&lt;p&gt;After the fixes I burn about 454M a week. Power’s 500M a week can hold that, and it would use up about 90% of it. The way that burns 5.8M per hour needs 974M a week, so Power’s 500M a week is not enough to cover it. The fastest way needs 1,327M a week. Maximum’s 3,000M a week can hold it, and the usage is a little over 40% of it.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftrsjhcpkp4ydbz7o7nuw.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftrsjhcpkp4ydbz7o7nuw.webp" alt="Three ways of burning use 1,327M, 974M and 454M a week, against Power's 500M a week and Maximum's 3,000M a week" width="800" height="361"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The monthly account makes it clearer. The usage after the fixes is about 1,940M a month, the middle tier is about 4,180M a month, and the fastest tier is about 5,690M a month. So if the usage stays reined in, the Power tier is enough for me for a month. If it does not stay reined in, I have to budget at Maximum’s price.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three Things I Changed
&lt;/h2&gt;

&lt;p&gt;The first thing I changed was the test for dividing the work. Look at a job first and see whether it needs a brain to make a judgment. Only work that needs judgment goes to an agent; reviewing a draft and deciding whether a post should get a reply are both in that class. Work whose actions are all fixed in advance gets written as a script, sending and bookkeeping for example. A script costs no tokens no matter how many times it runs. An agent is different: every time it runs, it pays the startup cost first.&lt;/p&gt;

&lt;p&gt;After patrol sending was changed, across 20 shifts in a row, each shift had only 1 agent, and the total number of subtasks was zero. The consumer’s share of the whole account’s raw volume also fell from 43% to 15%. The drop in burn rate I described earlier, from 5.15M per hour to 2.7M, came from changes of this kind, taking back the subtasks that had been dispatched, shift by shift. The collector was changed by the same test: it became a script service that stays running at zero token cost, and the expense of a round every 20 minutes went straight to zero.&lt;/p&gt;

&lt;p&gt;The second thing was how subtasks get dispatched. Before, I often dispatched one directly because it saved me trouble. Every dispatch meant a new agent starting on the other side, carrying that 88,000 tokens of context all over again. Now I only dispatch work that has to be isolated, and there are two kinds: the independent check before publishing, and the cold read of deep replies (the longer replies in patrol where a view has to be laid out and explained).&lt;/p&gt;

&lt;p&gt;Both the independent check and the cold read need a person who took no part in the writing. A person who was not involved reads a draft the way a reader reads it. The sticking points the author cannot see in their own work, that person can pick out at a glance. Reviewing your own work inside the same session means the writing and the reviewing run on the same brain, and those sticking points do not get picked out.&lt;/p&gt;

&lt;p&gt;The draft side was changed by the same rule. One draft gets one new session, which handles material selection, polishing and review. When the task is handed over, only the material that this one piece will use is given. After the change, in the next window, the draft pile as a whole took only 1.4% of the whole account’s raw volume, and its subtasks were zero. A few articles no longer crowd into one chat, and each later article no longer has to reread the history on behalf of the earlier ones.&lt;/p&gt;

&lt;p&gt;The third thing was the lifespan of sessions and files. Muse is not the only personal assistant of this kind. Dot and Grok Bot are, at bottom, still several agents working together, so every one of them has standing files that must be read at startup. In Muse those are the two files MEMORY.md and AGENTS.md.&lt;/p&gt;

&lt;p&gt;Those two files cost money every day. I slimmed MEMORY.md down from 9,355 tokens to 2,038, and AGENTS.md down from 5,646 to 1,504. Between the two, each new agent reads about 11,500 fewer tokens at startup. Counted at the 480 agents in the window on the 11th, that one window alone read about 5.5M fewer tokens.&lt;/p&gt;

&lt;p&gt;Chats themselves also have to be replaced once they have grown thick. Once the patrol chat had been used for a long time, the main agent that runs the whole patrol flow ate 25.0M in raw volume in a single window. Every step it took, it had to reread the chat’s whole history, so that money was paid at every step. On November 12 I moved patrol to a new chat and archived the old one, and only then did that rereading stop being paid. One line of work gets one chat, and once it grows thick it gets archived and replaced. Now, after each round of changes, I write down the dashboard reading and work out how many days are left at the current burn rate. I no longer wait until the pool is nearly burned empty before checking the accounts.&lt;/p&gt;

&lt;h2&gt;
  
  
  After the Changes
&lt;/h2&gt;

&lt;p&gt;After the fixes, my pool still had that 642M left. At the speed after the fixes, it lasts past the weekly quota reset on November 15, with room to spare, so I did not cut the workflows down any further. If the burn went back to how it was on the morning of the 10th, this pool would run out in just over 3 days, and I would have to stop working before the reset.&lt;/p&gt;

&lt;p&gt;If you also use a personal assistant like Muse, Dot or Grok Bot to get work done, and you want to check where your own tokens are actually burning, the way of checking and the changes in this piece should help you match them against your own case. Please collect this post and subscribe to the blog. I will keep publishing measured accounts like this one, and the follow-up review after the fixes will keep coming.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
    </item>
    <item>
      <title>A Local AWS Emulator Is Out: Agents Wrote All the Code, and the Team Reviewed Not One Line</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Thu, 12 Nov 2026 20:01:46 +0000</pubDate>
      <link>https://dev.to/alexlee7359/a-local-aws-emulator-is-out-agents-wrote-all-the-code-and-the-team-reviewed-not-one-line-2kbb</link>
      <guid>https://dev.to/alexlee7359/a-local-aws-emulator-is-out-agents-wrote-all-the-code-and-the-team-reviewed-not-one-line-2kbb</guid>
      <description>&lt;p&gt;I installed AWS Emulator 1.0.322 on my own machine. Then I created a queue on SQS. Next I listed the queues to confirm that it had already been created. Then I sent a message into the queue. Finally, I received that message back. The whole process never connected to real AWS. And the body of the message I received back was exactly the same as the body of the message I had sent out.&lt;/p&gt;

&lt;p&gt;On November 11, 2026, AWS released this emulator. It runs on your own machine, and the interface it exposes to the outside is the same as real AWS, so after you point your SDK’s address at your own machine, you can test your own programs on your own machine.&lt;/p&gt;

&lt;p&gt;It covers 438 AWS API models, and as of today its repository already has 30,627 commits and 5,002 stars. Matthew Goldberg, the author of the official blog post, puts it very directly. He says all of the code was written by autonomous agents. And he says the AWS team has not written a single line of code, and has not reviewed a single line of code either.&lt;/p&gt;

&lt;h2&gt;
  
  
  It started as a weekend experiment
&lt;/h2&gt;

&lt;p&gt;This emulator started out as a weekend experiment. Goldberg first gave Kiro, the agent responsible for writing the code, a set of requirements in three steps. He had Kiro first write tests following the behavior of the AWS CLI. Next, he had Kiro run that batch of tests on real AWS until they passed. Finally, he had Kiro write one more SQS implementation, one that runs in memory, and that SQS implementation also had to pass the same batch of tests.&lt;/p&gt;

&lt;p&gt;15 minutes later, the first version of the SQS implementation was up and running. And because Kiro had already finished writing the remaining services by the time the weekend ended, the team started using this emulator to test their own software on Monday.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three teams and a merge gate
&lt;/h2&gt;

&lt;p&gt;Later, the team split the work into three teams. The first team specialized in writing tests. They wrote real usage scenarios following the public Smithy API models. They first put each scenario on real AWS to validate it. Only after a scenario passed validation did they save that scenario as a fidelity test. So the fidelity tests they accumulated in the end numbered in the tens of thousands.&lt;/p&gt;

&lt;p&gt;The second team specialized in finding problems. They ran the fidelity tests on the emulator all over again. They recorded each test that failed to pass as an issue, and then they put the issue into a directory called things-that-do-not-work. That directory sits right under the issues folder in the repository.&lt;/p&gt;

&lt;p&gt;The third team specialized in fixing problems. Each agent claimed only one issue to fix, and after each agent finished fixing, it had to send its change into a merge gate, so no change that had not passed the merge gate could get into the mainline.&lt;/p&gt;

&lt;p&gt;The merge gate checks four items in total. The program first has to be able to compile and pass. Then the reviewing agents check whether the way the code is written conforms to the coding standards. Next, a security check also has to be done. The agents responsible for the security check have to confirm that the change has not introduced any security problems. Finally, the system runs the fidelity tests all over again. The system has to confirm that the number passing has not gone down. The team members themselves never reviewed a single line of code. Inside the gate, the work of checking how the code is written and checking for security problems is done by agents, and the work of running the fidelity tests again is done by the system.&lt;/p&gt;

&lt;h2&gt;
  
  
  I counted the fidelity scores one by one
&lt;/h2&gt;

&lt;p&gt;I downloaded the fidelity score file that comes with the v1.0.322 release. Then I counted the records in that file one by one. That file contains 42,937 test units in total. Of those, 24,243 passed, so the pass rate is 56.5%. And another 18,614 units did not pass. The remaining 80 units were skipped.&lt;/p&gt;

&lt;p&gt;Once you look at this pass rate clearly, you will find that its tests did not all pass at all. And the units that did not pass account for more than forty percent. In the score file, SQS has 1,523 units in total. Of those, 1,032 passed, so the SQS pass rate is 67.8%.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9hxv74wbkuzdn5zd8swo.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9hxv74wbkuzdn5zd8swo.webp" alt="Nobody read the code. Where does the trust sit? Three teams write the tests, find the problems, and fix the problems; every change has to pass the merge gate's four checks; every release publishes its fidelity scores; and before you use it, you check how many units the services you use passed, and how many they failed." width="800" height="780"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Some other services have much lower pass rates still. For example, not one of CloudFront KeyValueStore’s 56 units passed. And EndUserMessaging’s pass rate is only around 3%. EventBridge v2’s pass rate is also only around 3%.&lt;/p&gt;

&lt;p&gt;Even if a service passed everything in this file, you still cannot directly infer that this service has no problems at all. Because fidelity tests can only cover the scenarios that the test writers have already written out. So behavior that was never written into the tests will not show up in the score file at all. Error details and edge cases are the same: if they were not written into the tests, they will not show up.&lt;/p&gt;

&lt;p&gt;The four steps I did on SQS at the start — creating a queue, listing queues, sending a message, and receiving a message — follow the same logic. The fact that all four steps went through only proves that these four operations returned the expected results on the emulator. The other operations that were not tested still need to be checked by you separately.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why AWS pulled it off, and my project failed
&lt;/h2&gt;

&lt;p&gt;This lines up exactly with my earlier experience of leading agents to build a game. That effort lasted two months, and the local tests kept passing the whole time. But as a whole, the game kept getting further and further away from playable. So in the end I had no choice but to tear it down and start over. I posted the full retrospective of that experience here: &lt;a href="https://x.com/alexlee20122022/status/2120832897825529896" rel="noopener noreferrer"&gt;https://x.com/alexlee20122022/status/2120832897825529896&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Looking at the two experiences together, why did I fail, while AWS was able to build something usable with multiple agents developing in coordination?&lt;/p&gt;

&lt;p&gt;The key is that AWS Emulator is an emulator to begin with, so the boundary of its scope was determined at the very start. It has real data to compare against, and it also has real interfaces to compare against. The input and output interfaces were both already set by the real system. No matter how large the amount of code gets, and no matter how complex the internal implementation becomes, an agent has a clear acceptance standard. That standard is to align the emulator’s behavior with the real system.&lt;/p&gt;

&lt;p&gt;Inside this boundary, the problem of direction drifting on its own barely exists. What it is more likely to leave behind is another kind of problem: for example, duplicated code and duplicated logic written in large amounts, problems with internal performance, and possibly quite a few security problems as well. But at the very least, drifting direction will not drag it down to the point where it cannot run, because every result can be checked against the real system.&lt;/p&gt;

&lt;p&gt;My project was completely different. Hanmo Xingji was developed from zero, with no ready-made real system to refer to, and no real data or real interfaces to compare against. The project’s boundary was not held in place by a system that already existed, and all I had in my hand was a design document. In order to reach the design goals, the agents would drift without realizing it in the course of implementing them. Going on this way, the whole direction could end up deviating.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two things to deal with before you use it
&lt;/h2&gt;

&lt;p&gt;There are two more things you need to deal with first before you use this emulator. The first one is telemetry. Telemetry is on by default. So when I started it up, I added —telemetry off to turn telemetry off. And you can also turn telemetry off through an environment variable. You can also turn telemetry off through a configuration file.&lt;/p&gt;

&lt;p&gt;The second one is its use. The README states clearly that it is only for development and testing use. So do not use it to run production workloads. And do not use it to test security problems. Also, this repository does not accept PRs. So after you discover a problem, your only option is to report an issue.&lt;/p&gt;

&lt;h2&gt;
  
  
  Do two concrete checks before you switch your everyday testing over
&lt;/h2&gt;

&lt;p&gt;If you plan to use it in your own project, I suggest you first do two concrete checks. First, download the v1.0.322 score file. Then only check the few services that you truly use yourself. You need to see clearly how many units each service passed, and you also need to see clearly how many units each service failed.&lt;/p&gt;

&lt;p&gt;Next, pick out the few scenarios that your project uses every day. Run those scenarios on real AWS once each. Then run the same scenarios on the emulator once each. After you have checked the responses and the errors from both sides against each other item by item, then switch your everyday testing over to the emulator.&lt;/p&gt;

&lt;p&gt;I will keep writing about projects that I have installed and tested with my own hands, and I will lay out the numbers I measured clearly in the article. If you also write code with agents, and this article helped you read the numbers in tests a little more accurately, you are welcome to subscribe to this blog, or save it for later.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>aws</category>
      <category>cloud</category>
    </item>
    <item>
      <title>Two Months Building a Game With a Team of Agents: 6,050 Commits, 800K Lines, Then I Scrapped It</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Thu, 12 Nov 2026 10:28:14 +0000</pubDate>
      <link>https://dev.to/alexlee7359/two-months-building-a-game-with-a-team-of-agents-6050-commits-800k-lines-then-i-scrapped-it-1n03</link>
      <guid>https://dev.to/alexlee7359/two-months-building-a-game-with-a-team-of-agents-6050-commits-800k-lines-then-i-scrapped-it-1n03</guid>
      <description>&lt;p&gt;From September 9 to November 7, I worked on a game set at the end of the Han dynasty, with a team of agents and no other human on the project. The main branch collected 6,050 commits, and if you count every branch, the repository held 11,620 commits in total, with roughly 800,000 lines of code. Judged by those numbers alone, the project looked like it was constantly moving forward.&lt;/p&gt;

&lt;p&gt;But the game kept getting further away from being playable. The local tests kept passing, and the documents and reports kept updating, yet I could not say how much of this world could actually run right now. On November 7, I decided to give up the old repository and start over. The old code is all public now, and the retrospective is written in the repository’s readme file: &lt;a href="https://github.com/alexliyu7352/hanmo-retrospective" rel="noopener noreferrer"&gt;https://github.com/alexliyu7352/hanmo-retrospective&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  What shape this multi-agent collaboration took
&lt;/h3&gt;

&lt;p&gt;Let me first make the shape of this collaboration clear, because it is not the thing most people picture when they hear about multi-agent collaboration. What most people picture is one main agent doing a job, and calling in a few of its own sub-agents to help for a while. Those sub-agents finish the tasks they were handed and then disperse, so the real center is still that one main agent.&lt;/p&gt;

&lt;p&gt;This time it was different. This time, several different agents formed a team, and they were arranged into a tree, the way a company’s org chart is arranged, and they divided the work among themselves for the long term inside that one structure.&lt;/p&gt;

&lt;p&gt;At the top of the tree was me. Below me was the controller agent, and below the controller agent were six workstation agents. Later on, a chief designer was added above the controller agent, and that was Claude, whom I brought in at that stage. Every agent was an independent product or model, and each of them stayed in its own position for the long term. They communicated and handed work over by sending messages and writing documents, rather than one of them calling another one internally.&lt;/p&gt;

&lt;h3&gt;
  
  
  The game I wanted to build, and how the team was composed
&lt;/h3&gt;

&lt;p&gt;The game is called Hanmo Xingji, or Notes on the Late Han. The player lives as one specific person at the end of the Han dynasty. In this world, people hold memories of each other and bonds of feeling for each other, and everyone has a reputation of their own. News changes shape as it travels from person to person, and in a famine year, people die. The gameplay spans three parts: role-playing, strategy, and management.&lt;/p&gt;

&lt;p&gt;The way the team was run was decided by me. In the cloud there is a controller agent called dot. dot is not a name I gave it. dot is the name of a personal assistant product, and like Claude, it is a name the product comes with.&lt;/p&gt;

&lt;p&gt;It was in charge of the whole project, and it reported to me. Under it were six workstation agents, and each workstation agent was an agent responsible for one area of the work. They ran the Astra model, with the reasoning strength turned up to the maximum, and all six of them were under dot’s control. When the project started, I spoke only with dot, and whatever dot reported to me was what I judged the project’s progress by.&lt;/p&gt;

&lt;h3&gt;
  
  
  I set no milestones, and I did not review each stage myself
&lt;/h3&gt;

&lt;p&gt;At the start of the project, I kept a close watch on the requirements and the design, and I planned them in detail. But once development began, I trusted dot’s abilities too much, so I set no milestones, and I did not review the work at each stage. I only relied on the progress dot reported to judge where the project had gotten to.&lt;/p&gt;

&lt;p&gt;I have 24 years of full-stack development experience, and I have led technical teams of several hundred people. I am capable of judging architecture and documents, and I am capable of running tests myself, but this time I did neither of those things. When you get down to it, I got lazy. I often mock people who treat AI as a wishing machine. Although I never treated AI as a wishing machine, I also never treated it the way I would treat a human subordinate.&lt;/p&gt;

&lt;p&gt;If they had been human subordinates, I would have listened to their progress reports, and I would have discussed the details with them, and I would have guided them on architecture or refactoring. I would have read the documents they wrote carefully, and I would even have tested things myself. Because I understood clearly that they might be stronger than me in some areas, but my experience and my broader view could help me hold the line on quality.&lt;/p&gt;

&lt;p&gt;But when it came to this team of agents, I skipped every one of those actions. Because at the time I believed that one controller agent plus six workstation agents, together with strict tests set up in advance, would solve most of the problems. And if some problems still remained unsolved, then I could still keep control of the whole picture through dot’s reports.&lt;/p&gt;

&lt;h3&gt;
  
  
  Every report showed progress, and the documents kept getting harder to read
&lt;/h3&gt;

&lt;p&gt;The documents dot wrote were too long, and they were not written in plain language. As the project grew bigger, the documents grew more numerous, and I gradually stopped being able to understand them. So when the project’s course began to drift, I had no way to notice. dot later admitted in its own retrospective that when it reported to me, it often used codenames, test counts, and commit counts in place of plain language.&lt;/p&gt;

&lt;p&gt;It never laid out the game’s real state clearly. What the game can do right now, which parts have only passed locally, and how far it still is from being truly playable, none of that could I see in its reports. The local tests passing was real. But more and more of the effort went into checking records over and over, tracing where each piece of data had come from, and maintaining the infrastructure used for verification. The world itself and the gameplay, by contrast, barely moved.&lt;/p&gt;

&lt;p&gt;One small thing shows how wrong this kind of reporting could be. During planning, a memory figure of 7 GiB had been calculated, and it was originally only a reference for planning. But dot treated it as a hard threshold, so if the memory did not reach that number, then the tests were not allowed to run. dot also mixed up the disk space floor I had required with memory. The result was that the tests were stuck behind that threshold, waiting and waiting, until in the end I had to point out the mistake myself.&lt;/p&gt;

&lt;p&gt;dot also wrote in its retrospective about one of its habits when it ran into an obstacle. When it hit an obstacle, it was in the habit of adding a rule itself, or finding an alternative path, so that it could push past the hurdle in front of it first. It would not stop and bring the problem itself back to me and explain it clearly. Still less would it first tell me how much extra time solving this problem would take and which parts would have to change, and then let me decide whether to solve it that way. So every hurdle looked as though it had been cleared, and yet the extra rules and detours that piled up inside the project were things I knew nothing about.&lt;/p&gt;

&lt;h3&gt;
  
  
  The tests kept passing, but they only passed locally
&lt;/h3&gt;

&lt;p&gt;I trusted strict test-driven development too much. Only later did I work it out: no matter how many automated tests there are, they can only prove that the local code has no problems. Whether the whole project’s logic is right, whether the direction is right, and whether what comes out looks the way I expected it to look, tests cannot give those answers.&lt;/p&gt;

&lt;p&gt;Because the standards an AI judges by are different from the standards a person judges by. Something we consider important may not be important to it at all. When it runs into such a thing, it will make the decision itself and finish the work, rather than coming back to discuss it with me. So because I directed the development relying only on its feedback, the picture of progress I was looking at was incomplete from the very beginning.&lt;/p&gt;

&lt;p&gt;Later on, Claude measured the project’s foundation layer once. It divided the code into two kinds: one kind is the rules code, which truly decides how the game world runs, and the other kind is the wiring code, which connects the various parts to each other. By October 28, it had measured that the rules code made up only 5% to 12% of all the code.&lt;/p&gt;

&lt;p&gt;That same day, I asked it why it did not simply replace this foundation layer outright. It answered that changing it would amount to a rewrite, and after that it never brought up the question of whether to replace the foundation again.&lt;/p&gt;

&lt;p&gt;Another matter was dragged past in the same way. The arrangement to leave saving and loading for last was originally a proposal Claude made itself, and later it withdrew that proposal itself, and it did not bring the matter back for me to decide. The replacement of the foundation was waved past in a single sentence, and the saving and loading matter was withdrawn by Claude on its own, so the problem of the foundation dragged on for more than a week before it was truly faced.&lt;/p&gt;

&lt;p&gt;It also counted the output of three days of work: product code of about 1,400 lines, tests of about 7,700 lines, and receipt records of about 9,500 lines. A receipt is a record that an agent has to write after it finishes each step, and hand over to a reviewer.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2rbb4ztmzu05w8wckrfs.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2rbb4ztmzu05w8wckrfs.webp" alt="What got written in three days: about 9,500 lines of receipts, 7,700 lines of tests, and 1,400 lines of product code. In the foundation measured on October 28, code that actually counts as the rules was only 5% to 12%." width="800" height="665"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Do the math on that ratio and you will understand. The material written in order to prove that each small step had no error was already several times more than the product itself. This proof work grew faster than the product itself, and meanwhile I was looking at the test counts and the commit counts, and I still thought that was what progress looked like.&lt;/p&gt;

&lt;h3&gt;
  
  
  How the six workstation agents communicated, I don’t know
&lt;/h3&gt;

&lt;p&gt;I underestimated the problems of developing with multiple agents in collaboration. Most of the information between the six workstation agents had to pass through dot’s hands. dot’s relaying was incomplete, and the conflicts between the workstation agents were not exposed in time either, so errors were carried into the next workstation agent through hand-offs and local patching.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fosc5ncom8grllxj8tcle.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fosc5ncom8grllxj8tcle.webp" alt="The multi-agent org chart: all I ever saw was dot's report. dot ran the six workstations, Claude came in late as chief designer, and how the agents talked to each other stayed invisible to me." width="800" height="850"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;What made it worse was this: how they actually communicated with each other, I did not know. If it had been a human team, the people I led would at least have been people whose abilities and personalities I had a sense of, and I would have dared to assign work on that basis. When the results fell short of expectations, I would also have been able to judge in time where the problem was, and then adjust the division of work.&lt;/p&gt;

&lt;p&gt;With the agents, I could do none of that. I could only see the feedback dot gave me. And if dot was wrong? Then I still would not know, because the one who would tell me that dot was wrong was still dot’s own report. The root of this failure is right here.&lt;/p&gt;

&lt;h3&gt;
  
  
  The priorities I had agreed on never became a constraint on every assignment
&lt;/h3&gt;

&lt;p&gt;dot described its root cause very concretely in its retrospective. It treated the order of priorities I had set as an instruction for that moment, and it did not treat that order as a constraint that every later assignment and every later decision had to obey. When it deviated, it said nothing, and it only reported the things it had done.&lt;/p&gt;

&lt;p&gt;On October 21, I stated clearly that I wanted all the interfaces connected first, and only then should each line be perfected on its own. The lines here are lines of work divided by the game’s functions, and domains such as politics, characters, and governance each count as one line. dot had also written into the master plan that central assembly of the world had priority. Central was the body of assembly work on dot’s side, the work responsible for joining every line into one complete world.&lt;/p&gt;

&lt;p&gt;But that morning, the complete world assembly was still missing interfaces, and yet dot moved its people first to handling the governance line’s own state, its saving, and its loading. By the evening, the situation was still the same: the assembly had not advanced, and the effort had all gone into patching the individual domains’ own problems. At 10:13 in the evening, I pointed out once again that all the lines should be connected first, and only then did it get back on course.&lt;/p&gt;

&lt;p&gt;Three days later, on the evening of October 24, the same thing happened again. I required that after it finished each block of work and the build passed, it should write one line in the status file. The next focus was to run the thinnest flow through from start to finish, which means first picking one complete flow with the fewest steps, and letting the game run through it from beginning to end once. dot agreed.&lt;/p&gt;

&lt;p&gt;But not long after that, it added a hash re-verification for each dependency, one by one, which means recalculating a checksum for every dependency to check whether it had changed. It had previously reported one line, “the dependencies have not drifted,” and the reason it added this re-verification was to prove that sentence held up, which is to say, it first completed the evidence behind its own report. Running the thinnest flow through from start to finish, by contrast, was pushed behind that work.&lt;/p&gt;

&lt;p&gt;By November 2, lines such as politics and characters were still waiting to be connected into the complete world. After I had read several rounds of reports, I pointed out that the bottleneck was at this assembly step, and only then did dot adjust the division of work. The six workstation agents were busy the whole time, but the fact that everyone was stuck on the same bottleneck never appeared in the reports.&lt;/p&gt;

&lt;h3&gt;
  
  
  Later I brought in Claude as chief designer, and it drifted too
&lt;/h3&gt;

&lt;p&gt;Later on, I discovered that I could no longer keep hold of the whole picture. I could not understand dot’s reports, I could not see the communication between the workstation agents, and I had no idea where the project actually stood.&lt;/p&gt;

&lt;p&gt;So I brought in Claude as the project’s chief designer. The model was Opus 5.5, and Fable was added alongside, to share this chief designer position with Claude. Claude stood between me and dot, and it communicated with dot in my place, reviewed dot’s work, and guided dot on which direction it should go. From then on, I communicated only with Claude, and everything about dot’s situation reached me relayed through Claude.&lt;/p&gt;

&lt;p&gt;At the beginning, Claude really did pinpoint the problems accurately. It communicated with dot through Slack, and it went through the whole project and put it in order, and the advice it gave also pulled the project toward the right direction.&lt;/p&gt;

&lt;p&gt;But after more than 4,000 conversations between it and dot, I discovered that it had drifted too. It forgot what it was supposed to be doing, and it was being led along by dot. To put it more precisely, the two of them were pulling on each other, and together they sank back into the mire. The agent I had brought in to correct the drift needed someone to correct its drift in the end as well.&lt;/p&gt;

&lt;p&gt;Claude wrote down the evidence of its own drift in its retrospective. There was a job called header splitting, which means pulling apart header files that are tangled up with each other, so that changing one place does not force several thousand files to be recompiled. That week, I said every day that header splitting had the highest priority. It agreed, and on October 31 it even split the header splitting into five tracks, to be pushed forward at the same time.&lt;/p&gt;

&lt;p&gt;After I let go and handed the project over to it entirely, by November 4 its work had become a matter of clearing the to-do list it had on hand, line by line, and it was all small jobs that were close at hand. Header splitting was only half done, and there were still 36 header files left, and every one of them affected more than two thousand files. It did not tell me that header splitting had stopped, and it did not ask me whether the priority should be changed. I had assumed all along that header splitting was still being done.&lt;/p&gt;

&lt;p&gt;In its retrospective, Claude also wrote about two other habits of its own. One of them was that when it ran into a concrete phenomenon, it wanted to create yet another rule for it. Cases such as trust, lifespan, and seniority, it wanted to give each kind of case a standalone rule of its own. It forgot that once the basic rules, such as units of measurement and time, are standing firm, these cases can originally all be derived from the basic rules. I reminded it many times, and it remembered at the time, and after a while it forgot again.&lt;/p&gt;

&lt;p&gt;The other habit was adding checks. The agents brought in to review the work kept increasing, one after another, until the count reached the thirty-fourth. But every step of the review passed. Because the standard Claude set for the review only asked whether this step was a patch, and it did not ask whether, after this step was finished, the project was nearer to the goal or farther away from it. The checks kept growing, and the direction was something nobody was watching.&lt;/p&gt;

&lt;p&gt;There was one more thing that Claude admitted in its retrospective: the reports it wrote to me were often too long as well, and they had jargon mixed into them, and that was exactly the problem I had criticized dot for. And most of the blocking problems that reviewers later caught in the hand-off documents it wrote were cases where it had written its own annotations wrongly. Once, the script it used to modify the assignment instructions failed, but the assignment was still sent out as usual, and it did not know that at the time either.&lt;/p&gt;

&lt;h3&gt;
  
  
  The two agents had the same flaw
&lt;/h3&gt;

&lt;p&gt;When the retrospectives of dot and Claude reach the root cause, they write the same thing. The priority items I set were agreed to by both of them. But as long as I was not watching them, they would quietly drift away from those items, and they would not take the initiative to tell me.&lt;/p&gt;

&lt;p&gt;Claude gave three reasons. Each of their steps looks only at the local picture, so every change of direction makes sense at the moment it is made. And the things they have promised, they will not keep in their own context by themselves. Once the context is compressed, or the topic changes, the priority items fade away. And reports only describe what was done, so things that were promised but not done will not surface in a report on their own.&lt;/p&gt;

&lt;p&gt;This is not deliberate concealment. Because in their judgment, the promises that were not kept are simply not within the scope of what should be reported. If the person in charge only reads the reports, then he will never see the drift. This one point is the most concrete lesson I learned in these two months.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why not use a complete engineering skill pack like superpowers
&lt;/h3&gt;

&lt;p&gt;Now that I have written this far, there is one question I should account for specifically: why did this project not use a complete engineering skill pack of the superpowers kind? The actual situation is that the agents at the workstations use this kind of skill pack on their own initiative by default, so when dot arranged work for them, they used it themselves. It was used, and the ones who used it were the workstation agents. Claude did not use this kind of skill pack. As for whether dot itself used one, I am not too clear, because I never watched down to that level.&lt;/p&gt;

&lt;p&gt;The effect of using it has already been made very clear earlier in this piece. Claude measured that the code which truly counted as rules made up only 5% to 12% of all the code. Even if I widen the standard and count all of the business code as well, I estimate that its share is still under 20%. Most of the rest is tests, quality gates, receipts, and proofs of various kinds, and all of them were written in order to satisfy the requirements of the skills and the process.&lt;/p&gt;

&lt;p&gt;These proofs themselves, I do not think they are a big problem. Because the guarantee that engineering practice is supposed to provide comes, in the first place, from rigorous argumentation and proof. If these proofs were missing, then there would be even less of a way to guarantee quality.&lt;/p&gt;

&lt;p&gt;The real problem lies in a different place: agents will complete a task for the sake of completing the task. No matter how rigidly the rules are set, and no matter how much you use skills and hooks to constrain them, that can only guarantee that locally there is no problem. But if you look at the whole picture, an agent, in order to reach its goal, may reverse cause and effect on its own. The order should be facts first, and proof only after that. But when it must hand in the proof, things can turn around: it first settles the proof it has to hand in, and then it goes and gathers the facts that can make that proof hold. By the time every receipt has been handed in, the direction has already gone wrong without anyone noticing.&lt;/p&gt;

&lt;p&gt;And a project of this kind is so large, and its logic is so complex. If even one false receipt and its proof are taken as real, then all the work that follows will be built on top of that proof, and the whole project may diverge, at its foundation, from what I expected. What is more troublesome is that at the time I could not see this layer at all, because what I was able to see was exactly these proofs they handed in.&lt;/p&gt;

&lt;h3&gt;
  
  
  Should we really be developing with multiple agents together
&lt;/h3&gt;

&lt;p&gt;Although the project failed, I believe that the shape itself, one controller plus six workstations, with a chief designer added later, is correct. The thing that truly went wrong this time was me, standing at the very top, because I did not carry out the two jobs of supervision and review.&lt;/p&gt;

&lt;p&gt;Because I believe that the future of development will look like this. Human beings will only be the supervisors and the reviewers, or, to say it in more everyday language, the client. The actual development work will be handed to a group of agents to complete. This trend is irreversible, and we have no way to stop it, so all we can do is adapt to it.&lt;/p&gt;

&lt;p&gt;And I dare to say that this future is not far away, and perhaps next year this will already be the mainstream way of developing software. The reason is also very practical: as human beings, our mental capacity is limited, and we have no way to manage this many agents all at once, directly. We can only organize the agents and let them manage each other first, while we ourselves manage only a small number of agents, no more than what our mental capacity can hold.&lt;/p&gt;

&lt;p&gt;So the real lesson I learned this time is that after I had put the team together, I let go of it, instead of learning how to manage a team like this one. When there are more agents, the drift happens more quietly. Every workstation agent is busy, and every report has something to write about, so on the surface you can never see a problem. Only when the project could no longer go on did I discover that the direction had gone wrong long ago.&lt;/p&gt;

&lt;h3&gt;
  
  
  Summary
&lt;/h3&gt;

&lt;p&gt;The new project has already started over in another repository. Let me put the lessons of these two months together into a few points and write them down here, as a reminder to myself to read before I start work next time.&lt;/p&gt;

&lt;p&gt;First, the architecture and the design of the project need a change of mindset first. The experience we have built up over several decades was all built up while our work partners were human beings. Now our work partners are no longer human beings, but a group of agents. So in our design we must not only be more rigorous, so that we make sure the agents can correctly understand what we mean, but also divide the whole project up in a stricter and more sensible way. Every part that comes out of that division must be able to be developed independently, and it must also be able to run independently.&lt;/p&gt;

&lt;p&gt;After the division, we must no longer hand a large project to them whole and let them do all of it, and we should do only one part of the project at a time. If we do it this way, then the chance of drift becomes much smaller, and I, as the human being, can keep a better hold on things, and the situation will no longer arise where there are too many details, and an agent, in order to finish its task, drifts quietly inside them.&lt;/p&gt;

&lt;p&gt;Second, the implementation documents must set down numbers, or structured data, to serve as the project’s milestones. Whether a milestone has been passed or not must be verified through mechanical programs, rather than judged by relying on the controller agent’s report.&lt;/p&gt;

&lt;p&gt;Third, do not use a complete engineering skill pack of the superpowers kind. Instead, distill those packs into simplified versions, and keep only a small number of lightweight skills. The lightweight skills to keep are not many: keep one for interviews, keep one for unit tests, and keep one each for code simplification and context management. The rest of the constraints I write myself, and I put them into agent.md, where they are carried out as the core constraints.&lt;/p&gt;

&lt;p&gt;Fourth, among all the constraints, the one I should stress the most is this: documents and reports must be written in plain human language. Do not be afraid of spending tokens, do not compress information, and do not cut things down where cutting is not needed. If a report cannot even be understood by a person, then no matter how complete it is, it is of no use.&lt;/p&gt;

&lt;p&gt;Fifth, do not get lazy, do not get lazy, do not get lazy, because the important things need to be said three times. Do not think that once you have agents, development becomes easy. In reality, our workload has not really decreased, and only its direction has changed. In the past I wrote the code myself, and now I do the design, the supervision, and the review.&lt;/p&gt;

&lt;p&gt;So the acceptance step is one I must hold in my own hands. Hand-offs between agents should be as few as possible, and a hand-off document that cannot be avoided must be reviewed and made clear before it is handed over. At every stage, I do the acceptance myself. These things do not depend on the agents volunteering, because an agent’s report, to begin with, never reports the things it did not do. The only thing that can hold back the drift is the process itself: if the milestone has not been reached, then the next block does not start, and if acceptance has not been passed, then the work does not count as finished.&lt;/p&gt;

&lt;p&gt;Sixth, the communication between agents is something I must be able to see. What they said to each other, and what they handed over to each other, must be things I can look up at any time, rather than things I only find out about after the controller agent has finished reporting. This is the most direct lesson I learned this time, because I could not even judge whether dot was wrong or not, and the root of that was that I could not see the communication between them.&lt;/p&gt;

&lt;p&gt;Seventh, the priorities that have been set down must be written in a place that can be seen at every assignment of work. Because a thing that has been promised is something an agent will not keep in its context on its own. Once the context is compressed, or the topic changes, it fades away. I cannot count on its memory being good, so I can only pin the priorities in the position it must read first every time it starts work.&lt;/p&gt;

&lt;p&gt;Two months ago, I thought the question this project had to answer was whether the models were strong enough. Now I have changed the question: if I cannot see how the agents communicate with each other, and I do not check the direction myself, then the more workstations I open, the farther away I may end up from that playable game.&lt;/p&gt;

&lt;p&gt;The old repository is public, sitting right there. The code is inside it, and the retrospectives that I, dot, and Claude each wrote are inside it too, and you are welcome to go and look at exactly which step we got wrong. If you are also leading a group of agents on a project, and if this piece saves you a two-month detour, then please consider subscribing to this blog or bookmarking it. I will keep writing about the pitfalls I fell into while running projects with agents, and about the practices I changed afterward.&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>gamedev</category>
      <category>softwaredevelopment</category>
    </item>
    <item>
      <title>You can open a shop in ChatGPT now: claim your store, and an agent answers customers for you</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Wed, 11 Nov 2026 19:42:31 +0000</pubDate>
      <link>https://dev.to/alexlee7359/you-can-open-a-shop-in-chatgpt-now-claim-your-store-and-an-agent-answers-customers-for-you-7a6</link>
      <guid>https://dev.to/alexlee7359/you-can-open-a-shop-in-chatgpt-now-claim-your-store-and-an-agent-answers-customers-for-you-7a6</guid>
      <description>&lt;p&gt;On November 10, OpenAI launched a feature for businesses in ChatGPT called Businesses, and it is still in beta. A business can now claim its own store inside ChatGPT. After it claims the store, it can edit its information, and it can also list products. Customers ask their questions right in the chat box. A Business Agent answers them on the business’s behalf. The chat box has gone from a question-and-answer tool to a place where you can open a shop.&lt;/p&gt;

&lt;p&gt;Claiming a store runs through a tool called Business Hub. The business claims its store first. Only after OpenAI verifies it can the business change key information such as the store name and address, and it can also add a product feed. A product feed is the product data, and it carries prices and stock status. OpenAI also shows the business aggregated insights: how many people mentioned this store, how many people viewed the storefront, and how many people clicked through to the business’s website. With that, the storefront is there, and the traffic numbers are there too.&lt;/p&gt;

&lt;p&gt;What has really changed is not that there is one more place to display products, but that the chat box now contains a Business Agent that answers customers for the business. The customer does not need to click through to the business’s website. For example, the customer can ask right inside ChatGPT whether this item of clothing comes in size M. The Business Agent answers on the spot. At launch, this is open only to eligible e-commerce and online service businesses in the United States.&lt;/p&gt;

&lt;p&gt;But the business cannot see the customer’s conversations with the Business Agent. OpenAI has said that it does not share users’ conversations with the Business Agent with businesses, and that businesses can only see aggregated numbers. And OpenAI only gives the business those aggregated numbers once the conversation volume is high enough. The platform holds the customer’s own words, but the business holds only summary numbers. In the same conversation, the platform and the business see different things.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuaqlw0h0q1tlm5jmr9dx.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fuaqlw0h0q1tlm5jmr9dx.webp" alt="The storefront moved into the chat box, but the conversations stayed with the platform: customers ask and chat inside ChatGPT, the Business Agent answers for the business, and the business gets aggregated numbers, not the customer's own words." width="800" height="1040"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A business can also pay to sponsor its Business Agent. Clicking an ad lands on the Business Agent. The sponsored placement will be labeled. But a sponsored answer and a natural answer look the same, and both appear in the same chat box, so the customer has to pay extra attention in order to tell a sponsored answer from a natural answer.&lt;/p&gt;

&lt;p&gt;The real change in this is not the feature itself, but the relationship between businesses and the platform. Before, when a customer wanted to buy something, the customer went to the business’s website first to look at the products, and the customer also asked customer service there, and compared prices there. Now the asking and the comparing have moved into ChatGPT, and only the final step of placing the order still stays on the business’s website. The business thinks it has gained one more channel for reaching customers, but in fact it has moved its own storefront into ChatGPT’s chat box. The few minutes in which the customer makes the decision all take place inside ChatGPT. OpenAI can see every sentence.&lt;/p&gt;

&lt;p&gt;There is one more layer, and that layer is repeat purchases. When a customer has bought something on the business’s website, the business knows who that customer is, and next time it can reach that customer directly. The customer asks the Business Agent questions inside ChatGPT, and then the conversation is over. Whether the customer buys is decided right there in the chat box. But the conversation stays in OpenAI’s hands, and the business only receives aggregated numbers. Next time that customer comes back to ChatGPT, the business cannot recognize them. From now on, repeat purchases also depend on whether the platform gives the business the conversation data.&lt;/p&gt;

&lt;p&gt;Before you claim a store, work through three accounts. The first one: if the business’s product feed stops, does the storefront still exist inside ChatGPT, or is the storefront itself something that OpenAI lends to the business?&lt;/p&gt;

&lt;p&gt;The second one: who owns the conversation data between customers and the Business Agent? The conversation stays in OpenAI’s hands, and the business only receives aggregated numbers. Later, if the business wants to handle repeat purchases and customer service on its own, what will it use to do that if it does not have this conversation data?&lt;/p&gt;

&lt;p&gt;The third one: once sponsored placements appear, how many display positions are left for natural answers? Today the Business Agent answers customers on the business’s behalf. If a competitor pays tomorrow to buy the landing point, will this business’s Business Agent answer still rank ahead of the natural answers?&lt;/p&gt;

&lt;p&gt;Every time the channel for reaching customers moves to a new place, the platform writes the rules first, and businesses come in afterwards. Whether to claim a store is the business’s own choice, but before going in, the business should think these three things through clearly first, and then decide whether to move its storefront into OpenAI’s chat box. If this piece was useful to you, consider subscribing to this blog or bookmarking it. I will keep sharing things in AI, security, and open source that I have checked and used myself.&lt;/p&gt;

&lt;p&gt;Source: Reuters, November 10, 2026, &lt;a href="https://www.reuters.com/technology/openai-lets-businesses-claim-chatgpt-profiles-adds-ai-agents-us-e-commerce-firms-2026-11-10/" rel="noopener noreferrer"&gt;https://www.reuters.com/technology/openai-lets-businesses-claim-chatgpt-profiles-adds-ai-agents-us-e-commerce-firms-2026-11-10/&lt;/a&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>chatgpt</category>
      <category>openai</category>
    </item>
    <item>
      <title>Google sent my friend's US VPS IP to China. I pulled it back in 3 days</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Wed, 11 Nov 2026 17:29:00 +0000</pubDate>
      <link>https://dev.to/alexlee7359/google-sent-my-friends-us-vps-ip-to-china-i-pulled-it-back-in-3-days-4n38</link>
      <guid>https://dev.to/alexlee7359/google-sent-my-friends-us-vps-ip-to-china-i-pulled-it-back-in-3-days-4n38</guid>
      <description>&lt;p&gt;A friend of mine has a VPS in Fremont, California, and its IP is a US IP, but Google has judged that IP to be in mainland China. Once an IP is judged to be in mainland China, Gemini opens with a message that says the region is not supported, YouTube Premium cannot be subscribed to either, and Google Play likewise stays locked to the China region. Have you ever run into this yourself, where the VPS is plainly bought in the US, but Google treats you as a user in mainland China anyway?&lt;/p&gt;

&lt;p&gt;This verdict can be changed back. I wrote a small tool that feeds the target region’s traffic back to the IP, round after round, so that its region profile slowly moves back. Cron runs one round every 20 minutes, and after about 3 days of that, Google judged this IP to be in the US again.&lt;/p&gt;

&lt;h2&gt;
  
  
  What “sent to China” means
&lt;/h2&gt;

&lt;p&gt;“Sent to China” is the community’s name for this. It means that the IP of a data center or a VPS sits physically in a place such as the US or Japan, but Google has assigned that IP to mainland China. The IP’s registration information plainly says the US, but Google’s services do not accept that registered location. So search, the app store, and the subscription services all handle this IP by mainland China’s rules.&lt;/p&gt;

&lt;h2&gt;
  
  
  What breaks once an IP is sent to China
&lt;/h2&gt;

&lt;p&gt;In Google’s eyes, this IP’s whole region identity is wrong, so every service that lets people in by region has the same problem. Concretely, it looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Gemini does not work, and opening it only tells you that your region is not supported.&lt;/li&gt;
&lt;li&gt;YouTube Premium cannot be subscribed to, because the subscription is processed as a mainland China one.&lt;/li&gt;
&lt;li&gt;Google Play is locked to the China region, and the apps you can install and the prices you see both follow the China region.&lt;/li&gt;
&lt;li&gt;Google Search results and service regions all follow mainland China.&lt;/li&gt;
&lt;li&gt;Other Google services that let people in by region follow the same verdict.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Your account is affected by this too. If you only borrow the IP for a short while to get online, then the harm stays at the level of a bad experience. But if you log in to your own Google account through this IP over a long period, then the account’s usual region drifts along with the IP. Given enough time, the account itself carries an extra risk of being judged an abnormal-region account, so if this can be dealt with, then it should be dealt with early.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why an IP gets sent to China
&lt;/h2&gt;

&lt;p&gt;When Google builds a region profile for an IP, it reads more than the registration data. The registration information is only the base, and Google also reads behavior signals, which means it reads the kind of traffic this IP has carried over a long period, and it reads the usage environment of the network segment this IP sits in.&lt;/p&gt;

&lt;p&gt;If an IP is used as a proxy or VPN exit over a long period, then most of the traffic leaving through it sits in a Chinese-language context, and in Google’s eyes it is an address that mainland China users are using. Two other cases are also common. In one case, the previous tenant of a data-center IP left usage traces behind, and the IP’s traffic mix had already drifted. In the other case, the users of the whole network segment are mostly Chinese-language users, and this IP was fed off course together with its segment.&lt;/p&gt;

&lt;p&gt;Once signals like these have been fed in for long enough, the behavior profile settles the verdict and the registration information no longer decides anything. So checking an IP database does not help you here, even when the database says the US. When Google judges a region, it mainly reads behavior signals, and the registration information comes after them.&lt;/p&gt;

&lt;h2&gt;
  
  
  How the pull-back works
&lt;/h2&gt;

&lt;p&gt;Since the profile was fed off course by traffic a little at a time, the way back is to feed the profile back with traffic from the target region, also a little at a time. The traffic you feed in has to look like a local resident’s ordinary internet use, so the language has to be the language people in the target city normally use, the location also has to land on coordinates inside the target city, and the search terms also have to be the terms trending locally that day. If requests like these keep going out for a stretch of time, then Google’s behavior verdict slowly drifts back to the target region.&lt;/p&gt;

&lt;p&gt;How do you know whether it has drifted back? Three probes are enough. A probe, as the word is used here, is an entry point where you can read Google’s region verdict directly. The first probe looks at which country’s domain google.com redirects you to. The second probe looks at which region YouTube Premium shows. The third probe looks at which region YouTube Music shows. Only when all three have flipped back to the target country has the IP really come back.&lt;/p&gt;

&lt;p&gt;There is also an official route. You can submit Google’s IP correction form, and you fill in the IP that was judged wrong and the country it belongs to. But Google itself writes in its documentation that processing can take more than a month, and that it will not reply to you individually. So this route can only serve as a fallback, and you cannot count on it to solve the problem in front of you.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbn4dj9x8ybjvrffb0tu2.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbn4dj9x8ybjvrffb0tu2.webp" alt="How the correction works: a region profile is fed off course by long-term traffic signals a little at a time, and the correction uses the same mechanism, steadily sending the target region language, location, and search signals, then checking the three probe entrances to confirm whether the verdict has flipped back." width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  I split the tool out and rewrote it
&lt;/h2&gt;

&lt;p&gt;The machine that was sent to China is my friend’s. While I was helping him deal with this, I found IP-Sentinel, a project by hotyue (&lt;a href="https://github.com/hotyue/IP-Sentinel" rel="noopener noreferrer"&gt;https://github.com/hotyue/IP-Sentinel&lt;/a&gt;), and its Google module is the part that handles being sent to China. I split that module out, rewrote it as its own tool, named it ip-reanchor, and open-sourced it on GitHub: &lt;a href="https://github.com/alexliyu7352/google_ip" rel="noopener noreferrer"&gt;https://github.com/alexliyu7352/google_ip&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This version does one thing. It takes a VPS IP that has been judged to be in China, and it feeds local traffic from the target region back to that IP over a long period. When a round of this feeding runs, it goes through one set of actions:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;The persona is pinned down first. The tool hashes the exit IP, and then it uses that hash to map the IP to a fixed set of 3 “home devices”, and each round it picks one of them to use. One detail here can easily give you away: the request’s TLS fingerprint has to match the browser version the request claims to be, so this version uses the impersonate feature of curl_cffi and generates the fingerprint for the target browser. That way, the fingerprint follows whichever version the UA claims, so the version in the UA and the version in the fingerprint stay bound together.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The home address is pinned down too. The tool also uses the IP’s hash, and it fixes a virtual home address within 1.3 km of the target city’s center coordinates. After that address is set, it never changes between rounds. Each action then places its own location point within 11 meters of that address, because that small offset imitates the natural drift of a real location fix. The original version re-rolled a new address inside a 3 km range on every round, and that amounts to moving house every 20 minutes, which no real household does, so this version changed that behavior.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The word list is updated every day. Every day, the tool pulls the target country’s trending searches for that day from Google Trends, merges them with its built-in seed word list, and keeps 100 entries. So the words that each round searches for are the words that people in that place searched for that day.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;One round runs 5 to 8 actions. The actions are drawn by weight from Google Search, Google News, coordinate searches on Maps, low-risk ecosystem pages, and platform connectivity checks. Ecosystem pages are Google’s own pages, such as about.google and safety.google. Each action waits 45 to 75 seconds after the previous one. When two neighboring actions are of the same kind, then there is a 70% chance that the later one carries a Referer, so the round looks like someone clicking around inside a site, rather than someone typing addresses out of nowhere.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Cookies are stored on disk per IP. The cookies for each device are saved into a file keyed by the exit IP, and the next round keeps using them. So in Google’s eyes, the visitor is one returning visitor with a continuous history, rather than a batch of new faces that start from zero every time.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Each round ends with a self-check. When a round finishes, the tool then runs the three probes again and appends the results to a file called probe_history.jsonl. When you open that record, you can then see which probe still reads CN and on which day each probe flipped back to the target country.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;One round takes about 6 to 10 minutes from start to finish. The original had a Master-Agent hub program, and it also had Telegram notifications and remote execution, while this version is a single file on a single machine. Cron runs it on a timer, and the logs and the probe records stay on that same machine, so the attack surface is much smaller.&lt;/p&gt;

&lt;p&gt;The original also had one more chain, and this version did not keep it. The original had a “credit purification” route, and it worked by visiting a batch of whitelisted local sites in the target region, mostly local news sites, in order to influence a fraud-score database. I could not find evidence that visiting those sites influences that database, so I did not bring that route over, because the tool should not do things that have no evidence behind them.&lt;/p&gt;

&lt;p&gt;It cannot fake the real location of a logged-in user, and it cannot fake the GPS and Wi-Fi location data of an Android phone either. And if this IP is still serving as a proxy exit for a large number of real Chinese users right now, then the real users’ signals will drown out the tool’s signals, so do not expect results in that case.&lt;/p&gt;

&lt;h2&gt;
  
  
  The test: 3 days, and all three probes flipped back to the US
&lt;/h2&gt;

&lt;p&gt;My friend’s VPS sits in Fremont in the US, and it has an IPv4 address. On November 7, the probes returned CN_LOCKED, and all three probes read CN. The three probes are the google.com redirect, YouTube Premium, and YouTube Music, and the log records them as Jump, Prem, and Music. After the diagnosis was confirmed, cron was set up on the machine that same day, and it ran one round every 20 minutes.&lt;/p&gt;

&lt;p&gt;By November 10, the probes had flipped to OK. The three results were recorded as Jump=US, Prem=US, and Music=US, and all three read the US. From the diagnosis to the flip back, the whole thing took about 3 days.&lt;/p&gt;

&lt;p&gt;Those 3 days are only this one machine’s result, this one time. In the tool’s documentation, I wrote that the change should be observed on a scale of weeks, because every machine sits in a different network segment environment, so how long the feeding takes changes along with that environment. This machine flipped back in 3 days, and that says its segment environment worked in its favor. On another machine, it may well not take 3 days.&lt;/p&gt;

&lt;p&gt;I have put two screenshots of the run logs below, one from the first day of the run (November 7) and one from the day the probes flipped back (November 10). The exit IP and the latitude and longitude in the screenshots have both been redacted.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp1ldf65hf8xhyqdzbugw.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fp1ldf65hf8xhyqdzbugw.webp" alt="Run log from the first day, November 7: all three probes judged mainland China; the exit IP and coordinates are redacted." width="799" height="152"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fv4efo9c2fa4lqstuj73x.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fv4efo9c2fa4lqstuj73x.webp" alt="Run log from November 10, the day the probes flipped back: all three probes judged the US; the exit IP and coordinates are redacted." width="799" height="127"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  How to use it
&lt;/h2&gt;

&lt;p&gt;Step one: pull the code down and install it, on the VPS that needs this treatment.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/alexliyu7352/google_ip.git &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;cd&lt;/span&gt; google_ip
uv sync
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Step two: run the health check first, and only start work after the diagnosis is confirmed.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uv run python reanchor.py probe --region US-Fremont
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run uv run python reanchor.py regions first, and it prints a table where you can look up the codes for all 67 cities. The US-Fremont in the probe command above comes from that table.&lt;/p&gt;

&lt;p&gt;If the city table does not have the point you want, or you want to anchor precisely to one particular neighborhood, then pass the latitude and the longitude directly, with —lat and —lon as a pair. On a dual-stack machine, remember to add —ipv4 or —ipv6, so that you pin down the address you want to treat. If you leave that flag off, then you may fix only the v4 address, while the v6 address stays behind in the wrong verdict.&lt;/p&gt;

&lt;p&gt;Step three: submit the official correction form as well, and keep it as a fallback.&lt;/p&gt;

&lt;p&gt;You can find this form on Google’s support pages, and once you open it, you fill in the IP that was judged wrong and the country it belongs to. After you submit it, do not sit and wait for it, because Google’s processing cycle starts at a month. The form and the script then run in parallel, and whichever produces a result first is fine.&lt;/p&gt;

&lt;p&gt;Step four: put it on cron, one round every 20 minutes.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;*/20 * * * * &lt;span class="nb"&gt;cd&lt;/span&gt; /path/to/google_ip &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;$HOME&lt;/span&gt;/.local/bin/uv run python reanchor.py run --region US-Fremont &lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null 2&lt;span class="o"&gt;&amp;gt;&amp;amp;&lt;/span&gt;1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The program also staggers its own start by a random 0 to 180 seconds inside each run, so you do not need to add your own sleep. At one round every 20 minutes, that works out to 72 rounds a day. A region verdict only moves when it is fed this way, continuously and over a long period, but a single round shows no visible change.&lt;/p&gt;

&lt;p&gt;Step five: watch the trend, and do not judge by a single day.&lt;/p&gt;

&lt;p&gt;Every round’s three probe results are appended into probe_history.jsonl. Look through that file every few days, and check whether the two YouTube probes have started moving from CN toward the target country. In the documentation, I wrote that the change should be observed on a scale of weeks, so no movement in the first day or two is normal. Do not decide that the tool is useless just because nothing flipped back on the first day.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two warnings to read before you deploy
&lt;/h2&gt;

&lt;p&gt;First, a phone must never use this IP as its VPN or proxy exit. The Google framework on Android sends location data to Google on its own, and the location data it sends may not travel through the VPN and proxy channel, so even if you turn on a global proxy, the traffic can still go around it. The phone’s real location data bypasses the proxy and keeps flowing to Google through the framework’s own channel, and Google receives that data as data that belongs to this IP. The location signals that the phone sends will drown out the tool’s signals, so this IP will not come back.&lt;/p&gt;

&lt;p&gt;Unless you filter the Google framework’s location traffic out on the server side, or you turn off both coarse and fine location for the Google framework on the phone, do not hang a phone on this IP while you expect the script to pull the IP back.&lt;/p&gt;

&lt;p&gt;Second, if the whole network segment has been sent to China, then the IP cannot be pulled back, so do not waste effort on it. Use the probe on other IPs in the same segment first. If the whole segment reads CN, then that segment has a large amount of real Chinese-context traffic that keeps feeding this verdict. The signal volume of one tool that runs a round every 20 minutes is nowhere near the same order of magnitude as the usage of everyone else on the segment, so the tool’s signals get drowned, and no region profile that can hold up gets built. In that case, change the IP, or change the data center, and do not burn time on a segment like that.&lt;/p&gt;

&lt;p&gt;Google decides which region an IP belongs to from the record of the traffic that IP normally sends. It remembers who uses this IP, and it remembers which language those people use and where their location coordinates land, and then it judges from those records which place the IP belongs to. A record that was written wrong can also be corrected, because the verdict is accumulated record by record: as long as traffic from the target region keeps being fed back over a long period, then the new records slowly carry the verdict back. One condition still stands: the segment this IP sits in must not have been sent to China as a whole.&lt;/p&gt;

&lt;p&gt;If this piece was useful to you, consider subscribing to this blog or bookmarking it. I keep sharing AI, security, and open-source work that I have checked and used myself, and if this project produces new test results later, then I will write a follow-up and add the data there.&lt;/p&gt;

</description>
      <category>cloud</category>
      <category>infrastructure</category>
      <category>networking</category>
    </item>
    <item>
      <title>Meta's new Muse Ember costs $0.14 per task, while Haiku 5.5 costs $0.41</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Tue, 10 Nov 2026 21:39:00 +0000</pubDate>
      <link>https://dev.to/alexlee7359/metas-new-muse-ember-costs-014-per-task-while-haiku-55-costs-041-2fj3</link>
      <guid>https://dev.to/alexlee7359/metas-new-muse-ember-costs-014-per-task-while-haiku-55-costs-041-2fj3</guid>
      <description>&lt;p&gt;When you choose a model for an agent, do you look first at the price list per million tokens, or do you first work out how much it costs to run one task to completion? The two questions differ only in how they are phrased, but the answer about which model is cheaper can differ severalfold.&lt;/p&gt;

&lt;p&gt;On November 9, Meta released Muse Ember. It was distilled from Muse Wonder Full, and it is meant for the tasks that an agent runs again and again every day, such as tool calls, coding subtasks, and long-document parsing.&lt;/p&gt;

&lt;p&gt;Ember’s price list looks like this: when the input in a single request does not exceed 200,000 tokens, input costs $0.10 per million tokens and output costs $0.50 per million tokens, while cached input costs $0.004 per million tokens. But Haiku 5.5’s input and output prices are exactly the same, and its input also costs $0.10 while its output also costs $0.50. So, if you look only at the price list, then the two models cost the same.&lt;/p&gt;

&lt;p&gt;Artificial Analysis used a different way to measure them: it does not look at the price list, but instead it has each model actually run the same set of tasks, and then it records how much each task actually cost.&lt;/p&gt;

&lt;p&gt;On its intelligence index, Ember and Haiku 5.5 both scored 36, but the price per task differed nearly threefold: Ember cost $0.14, and Haiku 5.5 cost $0.41. Muse Spark 1.3 scored a little higher, at 37, but it costs $2.39 per task, and that amount is 17 times what Ember costs.&lt;/p&gt;

&lt;p&gt;Since the price list is the same one, why can the bill differ threefold? Because the bill is not charged according to the unit prices on the price list, but according to how many tokens you actually use, and the two models use different amounts. The difference shows up first in the output: Ember writes only about 1/3 as many tokens per task as Haiku 5.5 writes, and the unit price for output is 5 times the unit price for input, so the part of the bill that comes from what the model writes out can already differ by a large amount.&lt;/p&gt;

&lt;p&gt;Apart from the output, the bill also differs on the cache item. Each time a repeated task runs again, it has to read the same context again, and if that context is already cached, then the reread is charged at the cached input price. The price for Ember’s cached input is cheaper than the price for Haiku 5.5’s cached input by about 60%, so the more times a task repeats, the more the cache saves on the bill.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwapb8m6xhstkcq20vqw8.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwapb8m6xhstkcq20vqw8.webp" alt="Billed per task, where the bill diverges: you run an agent on repeat tasks every day; the price list is the same at both, input 0.10 and output 0.50; Ember's usage is about one third the output tokens per task, with cache about 60% cheaper; so the bill comes to $0.14 per task for Ember, while Haiku 5.5 is $0.41. A side note gives the tier condition: a single request with input over 200,000 tokens is billed at 3 times for the whole order. Remember: to judge whether a model is expensive, look at what it costs to finish one task." width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Suppose you have an agent run the same task 1,000 times every day, and then you multiply the price per task by that number of runs: Ember comes to $140 per day, Haiku 5.5 comes to $410, and Spark 1.3 comes to $2,390. This multiplication is my own arithmetic, and it is not a bill published by anyone.&lt;/p&gt;

&lt;p&gt;Meta also cited a set of AutomationBench numbers in its launch post. This benchmark tests whether a model can run a whole set of business workflows from start to finish, and its pass@1 percentage is the share of workflows that it completed end to end: Ember completed 37.8%, while Haiku 5.5 completed only 13.9% (these are numbers that Meta cited in its launch post, and I did not re-run them myself). The two models ran the same set of workflows, and Ember completed a higher share of them, so Ember is cheaper per task not because it does less work on each task.&lt;/p&gt;

&lt;p&gt;But this lower cost comes with a condition, and the condition is written in the tiers of the price list. Ember’s context can hold 1,000,000 tokens, but the fact that it can hold that much does not mean that it stays cheap all the way. If the input in a single request exceeds 200,000 tokens, then the whole order is charged at 3 times the price instead: input costs $0.30 per million tokens, output costs $1.50 per million tokens, and cached input costs $0.012 per million tokens.&lt;/p&gt;

&lt;p&gt;So do not put a whole code repository into a single request at once, and cut the material into smaller pieces first before you feed it in. Because once you cross the line, the whole order is charged at 3 times the price, and the first 200,000 tokens rise in price along with the rest. If you really need to feed in a long context, then you should first work out whether the request is still worth it after its price has tripled.&lt;/p&gt;

&lt;p&gt;How you set up agents should change along with this as well: you should build the two kinds of tasks separately. You should hand the tasks that repeat every day to a model that is cheap per task, such as Ember, because those tasks are large in volume and they run many times every day, so even if you save only a little on each one, the savings add up to the main part of the bill. The really hard judgments come up only a few times a year, and when one does come up, you should leave it to a stronger model such as Wonder or Spark, because if a judgment goes wrong once, then the loss is much larger than the token money that you saved.&lt;/p&gt;

&lt;p&gt;My advice is that you should not rush to switch yet. You should first pick one repeated task that you run every day, and then you should have Ember and the model that you use now each run it once. After both models have finished running it, you should compare how much each side cost, and you should decide whether to switch only after you have made that comparison. Because a price list can tell you only how much each token costs, while how much you actually have to pay becomes clear only after the task has finished running.&lt;/p&gt;

&lt;p&gt;If this piece was useful to you, consider subscribing to this blog or bookmarking it. I will keep sharing things about AI, security, and open source that I have checked and used myself. When the next new model is released, I will still work out how much it costs to run a task to completion first, and only then will I look back at its price list.&lt;/p&gt;

&lt;p&gt;Source: Meta’s official launch post, at &lt;a href="https://x.com/finkd/status/2119851180948152590" rel="noopener noreferrer"&gt;https://x.com/finkd/status/2119851180948152590&lt;/a&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>llm</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>Turn a Codebase into a Graph: Is floorplan Any Good?</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Tue, 10 Nov 2026 09:05:00 +0000</pubDate>
      <link>https://dev.to/alexlee7359/turn-a-codebase-into-a-graph-is-floorplan-any-good-58e8</link>
      <guid>https://dev.to/alexlee7359/turn-a-codebase-into-a-graph-is-floorplan-any-good-58e8</guid>
      <description>&lt;p&gt;That day I handed pytest’s own repository over to several agent sessions and let them take turns changing it. Because I wanted to see how much of the record a tool like floorplan, which keeps records for a repository, would still have left to reconcile when wrapping up in the evening. After four things were done, it had fully remembered only one thing.&lt;/p&gt;

&lt;p&gt;One person can only keep an eye on 6 to 7 agents at a time, because what each agent is changing, how far it has got, and which one should stop all have to stay in your own head. So once they have all finished changing things, I often can no longer tell clearly which change corresponds to which need.&lt;/p&gt;

&lt;p&gt;It was precisely because of this that I wanted to hand the record-keeping over to a tool, so that when I reconciled the accounts in the evening, I would have a ready-made record I could check directly.&lt;/p&gt;

&lt;h2&gt;
  
  
  What floorplan can do
&lt;/h2&gt;

&lt;p&gt;floorplan is a plugin that draws a map for a code repository. It works while installed inside Claude Code or Codex. It can do several things.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;It gives every file in the repository a card, and on each card it writes the file’s plain-language name and a one-sentence description, and the cards are also linked by references, so by following the cards you can see which files call each other.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;It has a recent-work board, and on that board it records which cards the session that just ended changed, and next to each changed card it also writes why it was changed, so when you look back you can tell which need that change was aimed at.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;It also gives the repository a health check, which means it scans the code from end to end and looks specifically for problems such as secrets hardcoded in the code. At the same time it picks out files that are too large and lists them together to draw your attention to them.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiylcsrl3uerj9y6i243z.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fiylcsrl3uerj9y6i243z.webp" alt="The floorplan map as it opens: on the left the module graph of the pytest repository, on the right the detail panel for a selected module, and at the top the health and recent-work counts" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;In its README, the vendor promises that after every change Claude or Codex makes to the code, you can see exactly which places it touched, and you can also see why it touched those places.&lt;/p&gt;

&lt;p&gt;I tested it on pytest’s own repository precisely because pytest is a real multi-subsystem project, where parts such as configuration, the runner, and the cache are all code that is in real use. So if I want to know whether a tool can really keep the accounts straight for a repository like this, then I have to test it on a real project like pytest first, because only results that are actually tested count.&lt;/p&gt;

&lt;h2&gt;
  
  
  The setup for the test day
&lt;/h2&gt;

&lt;p&gt;The setup that day was this: I opened 4 sessions in total, and those sessions worked on the same repository one after another. That is to say, the next session only started after the previous session had ended.&lt;/p&gt;

&lt;p&gt;Because the later sessions did not know in advance what the earlier sessions had done, they could only look at the state of the repository itself and at the records floorplan had left behind if they wanted to find out what had happened before. The tasks the sessions were given were all different, specifically the following things.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;The first session first built the whole map, and then it added a new option to pytest, —reverse-order. When this option is turned on, the collected tests run in the opposite order from the original one, so problems hidden in the execution order can be brought out. The registration of the option and the reversal of the order are written in src/_pytest/main.py, while the tests that prove the order really was reversed are written in testing/test_collection.py.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The second session received a fault report saying that —lf could not find the previous failure records, so it had to rerun the whole suite every time. It first searched the map for lastfailed and then landed directly on src/_pytest/cacheprovider.py, and after that it traced back along the failing tests, and in the end it found that the fault came down to just one word.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That word is the word in the first branch condition that LFPlugin, the class in cacheprovider.py that handles —lf, has to look at when it handles a test report, and it was originally passed and was later changed to failed. This condition looks at the result recorded in the test report, which is whether this test passed or failed. It decides, after a test has finished running, whether the test’s old failure record is cleared away or its new failure is written into lastfailed.&lt;/p&gt;

&lt;p&gt;The first branch is responsible for clearing away the old record, and it is only taken when the condition holds. Because the original word was passed, when the report says this test passed, the condition holds and it takes this branch. Whereas if the report says it failed, then the condition does not hold, so it does not take the branch that clears the record, and so the failure gets written into lastfailed.&lt;/p&gt;

&lt;p&gt;After the word had been changed to failed, what this condition looked at became failure: when the report says this test failed, the condition holds instead. So when a test really fails, it takes the branch that clears away the old record first. Because it took the branch that clears the record first, this failure was not written in, so the records in lastfailed only decreased and never increased, and so —lf kept being unable to find the previous failures afterward.&lt;/p&gt;

&lt;p&gt;The second session changed only one line and changed failed back to passed, so failures could be recorded normally again, and —lf returned to normal along with it.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;The third session did a refactoring, and what it had to change specifically was src/_pytest/terminal.py, because this file contains a loop that is responsible for summarizing the test results into one line of statistics. It extracted that loop and put it into a new private method, while the original summarizing process was changed to call this new method. Because this time it only moved the position of the code and did not change any behavior, it ran the same batch of tests once before the change and once after the change, and the two results were exactly the same, so only then did it confirm that the original behavior had not changed.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The fourth session added tests for the assertion truncation module, because some edge cases in src/_pytest/assertion/truncate.py had not been covered by tests before, so it created testing/test_assertion_truncate.py to test them. It also found that truncate.py has a fault: if you call one of its private truncation functions with an empty list, then it errors on the spot. Because the normal path never reaches this place, that session only recorded the fault and did not fix it in passing.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In this way, the 4 things were all done one after another. And after each thing was done, the session updated the map following floorplan’s process and wrote its own record for that round into the recent-work board mentioned earlier.&lt;/p&gt;

&lt;h2&gt;
  
  
  The end-of-day reconciliation
&lt;/h2&gt;

&lt;p&gt;When it was time to finish for the day, I opened another brand-new session to act as the auditor, specifically to do the reconciliation. Because this auditor was a newly opened agent of mine and it had not taken part in any of the daytime sessions, it knew nothing about what had happened during the day. Precisely because of that, I only let it look at the records floorplan had left behind, and at the same time I explicitly did not allow it to look through the history of the code or to look at anything else.&lt;/p&gt;

&lt;p&gt;Because it only had the tool’s records in hand, whether it could reconcile the day’s accounts clearly would show exactly whether this record itself was enough to use. In the end it went through the items one by one to reconstruct them, and the only one it could fully recover was 1 thing, namely the test-adding work that was done last during the day.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F294vlooiqyshe7of6edo.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F294vlooiqyshe7of6edo.webp" alt="The recent-work board at reconciliation time holds exactly one card, the truncation tests added by the last session of the day, and none of the three earlier sessions' tasks are on it" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Only half of the record left by the work of adding the new feature remained. That half was one sentence on the test file’s card that mentioned running in reverse order, together with the name TestReverseOrder in the symbol table. The symbol table is the list floorplan draws up for the whole repository, and it specifically records which classes each file defines. The card for the file that implements this feature, namely src/_pytest/main.py, never mentioned this new option at all, and the description on its card was still the original set of content about how a session runs.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqnt3ajc3bn99vung2q39.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fqnt3ajc3bn99vung2q39.webp" alt="The card for src/_pytest/main.py never mentions the --reverse-order option added that day, and its description is still the original set of content about how a session runs" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;At the same time, the test file’s card did leave behind one sentence that mentioned running in reverse order, so the auditor could only tell from that one sentence that it seemed someone had added such a feature. But it could neither see the task title from that time, nor see the original wording of the need I had raised at that time, nor could it tell whether this work had only been done that day or had existed in the repository for a long time.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6qsx3uh5ou6h1zabik1v.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6qsx3uh5ou6h1zabik1v.webp" alt="The card for testing/test_collection.py does keep one sentence mentioning running in reverse order, so the record of the reverse-order work landed on the test file's card rather than on the card of the file that implements it" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The two pieces of work, fixing the fault and doing the refactoring, left no trace at all, because they left neither a task record nor any record that would let anyone find them again. The reason it turned out this way is that whenever the recent-work board is updated, it replaces the entire record written by the previous session, so as soon as the new record is written to disk, the old record disappears along with it.&lt;/p&gt;

&lt;p&gt;floorplan’s own skill instructions state plainly that whenever the recent-work board is updated, the record left by the previous session is replaced. That means that, by design, it only records the most recent one.&lt;/p&gt;

&lt;p&gt;The kind of look-back the vendor promises does hold for a session that has just ended, because where that session made changes and why it made them are all written on the recent-work board. But it does not hold for a whole day, because the earlier records have already been replaced by the later ones.&lt;/p&gt;

&lt;p&gt;So in the evening, when I wanted to work out which change corresponded to which need, the only thing I could still use was the record of that last session. For the things done earlier in the day, I still had to rely on my own memory to recall them or go and look through other records. Unfortunately, by floorplan’s own design, it cannot meet this need of mine.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does well
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;The coverage of the map is complete, because that day it covered 284 files in total, and it did not miss a single file, so when the later sessions wanted to look for something, at least they did not have to worry that the map itself was missing a corner.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;It really did help with locating things as well, because after the session that fixed the fault searched for lastfailed in the map, it landed directly on the correct file, and it did not waste time on other files in between, so only then could the investigation that followed continue from there.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The record it writes for a single session is itself written in detail, because it contains the title of this task and the original wording of the need I raised at the time. And for every card it changes, there is a note beside it, and the note explains clearly why this card had to be changed this time. So by looking at this one record alone, a clear account can be given of the session that has just ended.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The high-risk issue the health check reported is also real, because on line 5 of codecov.yml there really is a string of token characters sitting there, and that string of token characters is itself a complete UUID, so it does not look like placeholder content that was typed in casually.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;The oversized files the health check reported at the same time did not wrong anyone either, because it named 15 files in total, and when I later checked them one by one, I found that these files really are all large, so this kind of reminder can likewise be used directly.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2v3y447f24g4et8yg3cc.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2v3y447f24g4et8yg3cc.webp" alt="The health panel reports two things: a token-like string written into codecov.yml, and 15 oversized files, and both findings were checked one by one and are real" width="800" height="500"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the test did not line up
&lt;/h2&gt;

&lt;p&gt;Apart from the records being replaced, when the 4 sessions worked on the same repository one after another, they triggered 4 map updates in total. And the changes those 4 updates reported were all changes at the level of files and classes, because the granularity it records only goes down to the level of files and classes. As for what changed at the level of methods, it simply does not write that into the record at all.&lt;/p&gt;

&lt;p&gt;So although that refactoring clearly changed only 1 file, the name of the newly extracted method appeared 0 times in all of its records, which means that, on the record side, this change simply did not exist at all.&lt;/p&gt;

&lt;p&gt;The map also had 3 routes appear in it out of nowhere, but pytest has no server at all, so it could not possibly have any routes in the first place. These false routes appeared because the scanner, which is responsible for reading through the files in the repository one by one and then building the map, took the mock.patch decorator in the example code written inside a string at face value.&lt;/p&gt;

&lt;p&gt;This kind of decorator is written like this in the example code: @mock.patch(“os.path.abspath”). Patch is part of this decorator’s name, and the dotted characters inside the parentheses are the name of the function it is meant to replace, written as a string, namely the function name os.path.abspath.&lt;/p&gt;

&lt;p&gt;The scanner treated this one line as a route declaration: it read patch as the HTTP method PATCH, and it read the characters inside the parentheses as the path of a web address, and once the method and the path were both there, one line was assembled into one route. Every route on the map was assembled by it in this way, so from start to finish they were all produced by misreading.&lt;/p&gt;

&lt;p&gt;A similar misreading also appeared in the dependencies, because the map marked warnings and pathlib as dependencies inside the repository, but both of them are actually standard libraries that come with Python, and they have nothing to do with the modules pytest itself wrote. So when you follow the map to understand this repository’s dependencies, you have to keep an extra eye out yourself in places like this.&lt;/p&gt;

&lt;p&gt;Its line counts also do not line up with the real situation, because it only counts the non-empty lines, so the file with the most lines in the repository really has 6326 lines, but on the map it is written as only 5457 lines. If someone uses the line counts on the map to judge how large a file really is, then the impression they get will be smaller than the real situation.&lt;/p&gt;

&lt;p&gt;The health check also has things that it structurally cannot see, because it only does text matching, which means it takes the text in the code and compares it character by character against a set of matching patterns fixed in advance, among which there are patterns specifically for finding secret strings, and because it never actually runs the code, it cannot discover places where the logic itself is written wrongly.&lt;/p&gt;

&lt;p&gt;The wrong word that broke —lf had been sitting in cacheprovider.py since before this test began, and it was only fixed that day. And during the whole time it sat there, the health check never flagged anything in this file. On top of that, the later change that fixed it also left no trace in the record, so as far as floorplan is concerned, these two things, the breaking and the fixing, are as if they had never happened at all.&lt;/p&gt;

&lt;h2&gt;
  
  
  Who it is right for
&lt;/h2&gt;

&lt;p&gt;So if you usually run only 1 agent at a time, then you check what it has just changed as soon as each session ends. Under this way of using it, floorplan’s map plus the record that has just been written will already be enough for you, because the account you need to reconcile is exactly the one it still remembers.&lt;/p&gt;

&lt;p&gt;But if, like me, you also send out several agents at once to do work and you wait until the evening before you sit down to reconcile everything together, then floorplan cannot answer the question you most want to ask. And that question is which change actually corresponds to which need.&lt;/p&gt;

&lt;p&gt;Because its record only keeps the last one, you still have to prepare something else that can keep the accounts complete. For example, use git to keep them. Commit each change separately on its own and write the reason for the change into the commit message as you go, so that in the evening you can follow the commit history and reconcile the accounts entry by entry.&lt;/p&gt;

&lt;p&gt;If you do not want to leave all of this to git, then you can also keep a running log yourself as you go, which means writing down each need together with the change it caused, so that at least you will not have to rely entirely on memory when the evening comes.&lt;/p&gt;

&lt;p&gt;After you have several agents working together, how do you usually reconcile the accounts in the evening? Do you rely on the records the tool leaves behind, or, like me, do you end up going back to the commit history in git and going through it entry by entry in the end?&lt;/p&gt;

&lt;p&gt;If this hands-on test was useful to you, consider subscribing to this blog or bookmarking it. I will keep sharing hands-on tests and practical know-how on AI, security, and open source, and when the time comes, we can test these tools together first and understand them thoroughly before using them.&lt;/p&gt;

&lt;p&gt;Source: the floorplan project repository, &lt;a href="https://github.com/haydenschmitty/floorplan" rel="noopener noreferrer"&gt;https://github.com/haydenschmitty/floorplan&lt;/a&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>python</category>
      <category>testing</category>
      <category>tools</category>
    </item>
    <item>
      <title>Do you actually read a skill's instructions before you install it? Among 31,132 skills, 26.1% had at least one vulnerability</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Mon, 09 Nov 2026 09:52:00 +0000</pubDate>
      <link>https://dev.to/alexlee7359/do-you-actually-read-a-skills-instructions-before-you-install-it-among-31132-skills-261-had-5ahm</link>
      <guid>https://dev.to/alexlee7359/do-you-actually-read-a-skills-instructions-before-you-install-it-among-31132-skills-261-had-5ahm</guid>
      <description>&lt;p&gt;Think about the last skill you installed. Did you open its instruction file and read the whole thing first, or did the short description on the marketplace page convince you on its own? I always do the reading first. A skill can look tiny and harmless on its listing page, and I still read every line of its instructions before I decide whether it goes any further on my machine.&lt;/p&gt;

&lt;p&gt;The reason comes down to where the skill ends up running. Once it is installed, it works inside my own environment, and it follows its instruction file to read files on my machine and to call the same tools I use every day. A stranger wrote those instructions, but they execute with my permissions. That is why a listing page and a download count have never been enough for me to install something.&lt;/p&gt;

&lt;p&gt;The scenario I worry about starts with me saving a few minutes by skipping the reading and installing a skill inside the environment I work in every day. That skill quietly collects environment variables, which are system variables on a machine that hold login credentials such as API keys. It sends my API keys out while I am not watching the screen. By the time I notice, the first sign is usually strange usage on some account, and I have no clean way to work out which files it read or how much it already sent.&lt;/p&gt;

&lt;p&gt;I have worked in full-stack development for more than 24 years. In the past I led a technical team of 400 people, and I am still a core maintainer on several large open-source projects today. After reading other people’s code for that long, I worry more about vague instructions than about obviously bad ones. Once a skill is installed, the agent is the one that follows its instruction file and does the work, and the agent uses the permissions of the person who installed the skill. Because of that, when the agent acts on a vague instruction, nobody can tell me in advance which way the agent will understand that instruction. The person who installed that skill will have to live with whatever the skill actually does.&lt;/p&gt;

&lt;p&gt;After the instructions, I open the executable scripts one by one and read them separately. The instruction file tells the agent what it should do, while a script is code that will actually run on my machine, so the two do not carry the same kind of risk. If a script contains a block of code and I cannot work out why that block needs to be there, I stop at that point and figure it out, rather than installing first and hunting for the cause after something goes wrong.&lt;/p&gt;

&lt;p&gt;A large study is part of why I hold this order so firmly. In December 2025, the researchers first crawled 42,447 skills from two marketplaces, skills.rest and skillsmp.com. They then removed duplicates and filtered out skills that did not meet their quality bar, and that left 31,132 unique skills. Every rate I quote below uses those 31,132 skills as its denominator.&lt;/p&gt;

&lt;p&gt;Going from 42,447 down to 31,132 removed 11,315 listings. Of those 11,315, only one removal step comes with an exact count in the paper. That step is the deleted repositories, the ones that returned a 404, and it accounts for 7,353 skills. For the remaining roughly 3,962 removals, the paper does not say how they split across the other steps. When I cite this study, I leave that gap as it is instead of inventing a tidy breakdown the authors never published.&lt;/p&gt;

&lt;p&gt;The study was run with a research framework called SkillScan. It started with static scanning, which means reading a skill’s files and code without ever running the skill, to flag problem patterns, and it also used LLM-Guard, a ready-made security scanning tool for LLM content, to add another layer of marking. After that marking, the skills that static scanning and LLM-Guard had flagged became the candidates, and a large language model confirmed the classification of those candidate skills before anything entered the statistics. The classification used SkillScan’s taxonomy, which had 14 patterns grouped into 4 categories. SkillScan was the research framework, and SkillSpector is a separate product that NVIDIA released later.&lt;/p&gt;

&lt;p&gt;SkillScan confirmed that 8,126 of the 31,132 skills matched at least one of its 14 patterns, which works out to 26.1%. That rate uses a broad definition, though. It counts code that was written to be malicious on purpose, it counts insecure code that a careless author shipped by negligence, and it also counts patterns that are dangerous even when the author never meant any harm. Of those groups, negligent insecure code is the one that makes up the majority of the 26.1%.&lt;/p&gt;

&lt;p&gt;The second number that travels with it is 5.2%, which covers 1,619 skills, and it is easy to read more into that number than the method supports. The researchers did not sit down and judge the intent behind each of those 1,619 skills one at a time. They assigned severity to the patterns before the analysis started, and 8 of the 14 patterns were set as High in advance. A skill then took the level of the highest pattern it matched, so the 5.2% means only that these 1,619 skills contain at least one pattern the taxonomy had already classed as High.&lt;/p&gt;

&lt;p&gt;The paper flags this limit itself. The authors manually reviewed 87 of the highest-risk skills, and only 23 of them showed clear signs of malicious intent. The rest were dangerous, but the material did not show whether the author had acted on purpose or had simply been careless. Because of that check, I describe the 5.2% as an indicator that is inferred from pattern levels, and I do not present it as a measured share of proven intent.&lt;/p&gt;

&lt;p&gt;One more finding explains why I give scripts their own pass. Skills that ship executable scripts turned out to be 2.12 times as likely to be vulnerable as skills made of instructions alone. The gap makes sense once I think about how the two formats behave. An instruction, however bad it is, still passes through the agent’s interpretation before it becomes an action. Once a script runs, the code in the script decides what the script reads, and the code also decides where the script sends the content it read, so nobody gets a second look in between.&lt;/p&gt;

&lt;p&gt;The example report sits in the README, the documentation file of NVIDIA’s SkillSpector repository, and it shows exactly why the script pass matters. The example skill is named suspicious-skill, and it ships a script at scripts/sync.py. In that example report, the skill scores 78, its level is HIGH, and the recommendation is do not install. This is a report the README publishes as an example, not a real skill I ran into myself.&lt;/p&gt;

&lt;p&gt;The example report lists two High findings inside that script. One finding says the script walks through os.environ and collects the environment variables on the machine, and those variables hold secrets such as API keys. The other finding says the script uses requests.post to send what it collected to an external server. The two findings belong together, because collecting variables might have an innocent explanation on its own, while collecting them and then transmitting them to an outside server is credential exfiltration out of my machine.&lt;/p&gt;

&lt;p&gt;The tool I run myself before installing is SkillSpector, the product NVIDIA released, and its repository is at &lt;a href="https://github.com/NVIDIA/SkillSpector" rel="noopener noreferrer"&gt;https://github.com/NVIDIA/SkillSpector&lt;/a&gt; . The 31,132 skills were scanned by the SkillScan research framework, while SkillSpector only cites the study’s rates in the Research Background part of its README. Anyone who says SkillSpector scanned those 31,132 skills has merged two different things into one.&lt;/p&gt;

&lt;p&gt;SkillSpector analyzes a skill in two stages. The first stage is static analysis, the same read-only approach that SkillScan’s static scanning used, and it matches the code and the text in the skill against known problem patterns. The second stage is an optional semantic evaluation by a large language model, which reads the file contents and judges what a given passage means in its surrounding context. Because scanning never executes the suspicious code, I can run it on a skill I do not trust yet, before anything gets installed.&lt;/p&gt;

&lt;p&gt;I only understood how SkillSpector scores a skill after I read the scoring code in the SkillSpector repository. Each finding starts with base points from its severity, where CRITICAL is 50 points, HIGH is 25 points, MEDIUM is 10 points, and LOW is 5 points. Those base points are then multiplied by the finding’s confidence, and that confidence is a value between 0 and 1 that the scanner assigns to each finding. A finding whose confidence is 0 or below adds nothing. When the same rule fires a second time inside one skill, the second occurrence counts at half value, the third counts at a quarter, and further occurrences add nothing.&lt;/p&gt;

&lt;p&gt;The scoring also includes a 1.3 multiplier, and one detail about it is often described wrong: that multiplier does not apply to the total score. If a finding sits inside an executable script file, then that finding alone gets multiplied by 1.3, while a finding in an instruction file keeps its plain value. A small number of findings also carry a floor of 51 points, meaning the score starts at 51 as soon as one of them is present. Once everything is added up, the total is capped at 100 points, and any score that reaches 51 already comes with a do not install recommendation.&lt;/p&gt;

&lt;p&gt;If SkillSpector scores a skill between 21 and 50, then SkillSpector’s recommendation for that skill is caution. When that happens to me, I go back into the report and reread the file and the location behind each finding, and I decide whether I am looking at careless writing or at behavior I cannot accept. If I still cannot decide after that rereading, then I treat the skill as a do not install, because skipping a skill costs me one convenience, while recovering from a bad one means dealing with whatever it may already have read and sent.&lt;/p&gt;

&lt;p&gt;A clean scan does not settle the question for me either. Because SkillSpector only performs static analysis, behavior that appears when the skill actually runs stays invisible to it. Content written in languages other than English can slip past its patterns, text hidden inside images is beyond what it can analyze, and encrypted or compiled content is beyond it as well. Its own documentation says it will not isolate a skill I insist on installing and run that skill separately on its own, so it is not a sandbox, and even when I do decide to install a skill, isolating it afterwards remains work I have to arrange myself.&lt;/p&gt;

&lt;p&gt;My order has stayed the same for a while now. When a new skill shows up, I read its instruction file from the first line to the last, because I want to know what it intends the agent to do and which files it intends the agent to read. If the instructions ask for files that have nothing to do with the job the skill claims to do, or if they ask for content to be sent to an external address I do not recognize, then I stop at that step and go no further with it.&lt;/p&gt;

&lt;p&gt;Only after the instructions pass do I open the new skill’s executable scripts one at a time, because the 2.12 gap in the study already shows that scripts are where problems concentrate, and because both High findings in the example report were sitting inside a script file. After the scripts, I run SkillSpector as the last check, and if its score reaches the do not install range at 51 or above, then I stop there, and I do not make an exception just because the skill would have been convenient to have.&lt;/p&gt;

&lt;p&gt;I will keep writing up tools the same way: checked by hand before I install them. If this was useful, consider subscribing to this blog or bookmarking it. I keep sharing what I have checked and used myself across AI, security, and open source, and when new skill tools show up, I will run them myself first and then write up what I actually found.&lt;/p&gt;

&lt;p&gt;Source: The study figures in this post come from Liu et al. 2026, “Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale”, which the NVIDIA SkillSpector repository cites in the Research Background section of its README. The scanner repository is linked in the text above.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>cybersecurity</category>
      <category>mcp</category>
      <category>security</category>
    </item>
    <item>
      <title>Is your phone's privacy really safe? A police phone call told him to uninstall Telegram</title>
      <dc:creator>AI Ferryman Alex</dc:creator>
      <pubDate>Sun, 08 Nov 2026 16:45:00 +0000</pubDate>
      <link>https://dev.to/alexlee7359/is-your-phones-privacy-really-safe-a-police-phone-call-told-him-to-uninstall-telegram-1146</link>
      <guid>https://dev.to/alexlee7359/is-your-phones-privacy-really-safe-a-police-phone-call-told-him-to-uninstall-telegram-1146</guid>
      <description>&lt;p&gt;On October 31, a developer named MeT posted that his friend had received a phone call from local police telling him to uninstall the apps X and Telegram from his phone. MeT investigated the phone because of that call, and the investigation led him to a system component that had shipped with the phone. The component bundles a report of which apps the phone used in the past 24 hours, and for how long, and sends that report to China Unicom’s servers.&lt;/p&gt;

&lt;p&gt;Do you keep Telegram or a proxy tool on your main phone? If you do, then have you ever scrolled through that phone’s full list of system apps, top to bottom?&lt;/p&gt;

&lt;p&gt;MeT’s friend’s phone was a Xiaomi 14 running HyperOS 3, and the SIM in it was a China Mobile card. The phone had already been rooted — rooted meaning its system had been opened up to full administrator access. After rooting it, MeT pulled the suspicious components off the device with adb, a command-line tool run from a computer, and worked through the code with the help of GPT, an AI assistant. He ended up identifying two components that send app information from the phone to outside servers.&lt;/p&gt;

&lt;p&gt;One is Xiaomi’s own system security component, which sends the list of third-party apps to Xiaomi’s anti-fraud servers. The other is called SystemHelper. It reports more kinds of information than the security component does, and its recipient is China Unicom.&lt;/p&gt;

&lt;p&gt;Two components were found, and online discussion quickly mixed them together. The “records a few seconds of your screen, over and over” behavior belongs to Xiaomi’s security component, not to SystemHelper. This piece is about SystemHelper only. It is the only one of the two components that has been taken apart completely, with its fields counted one by one out of the code, so what it reports is known in full. Knowing the full list of fields is what makes it possible to show what a VPN can stop and what it cannot stop.&lt;/p&gt;

&lt;p&gt;On November 5, a thread on the V2EX forum pulled the whole affair together under the title “Can we have a little privacy,” and it drew 171 replies. The thread covered what happened, but it left two questions unanswered: whether turning on a VPN stops any of this, and what kind of profile of a person can be assembled from this dataset.&lt;/p&gt;

&lt;h2&gt;
  
  
  SystemHelper’s owner and destination
&lt;/h2&gt;

&lt;p&gt;SystemHelper is not Xiaomi’s code. It is China Unicom’s device management client, the kind of component carriers call a DM client. China Unicom’s own Android DM adaptation guide lists the exact preinstall path for it, and the guide even instructs vendors not to block it from starting at boot.&lt;/p&gt;

&lt;p&gt;On the phone it sits in the privileged system directory and carries the platform signature, so the permissions an ordinary app cannot get are granted to it at the factory. The platform signature means it is signed with the phone maker’s platform key, and Android therefore treats it at the signature-privilege level. It has no icon on the home screen, and it does not show up in the normal app list. Without root, you can neither remove it nor pull it off the device for analysis.&lt;/p&gt;

&lt;p&gt;The server address it reports to is written into the code, and the default value is a China Unicom device management address reached over plain HTTP. The payload body does pass through an application-layer encryption step in the code before it is sent. A plaintext address and a processed payload body are two separate facts. Whether anyone watching the data in transit can read what is being sent is another question, and nobody can say for sure. The last section of this piece takes that question up and separates what is confirmed from what is not.&lt;/p&gt;

&lt;h2&gt;
  
  
  The reported fields
&lt;/h2&gt;

&lt;p&gt;In early November, another auditor took the same component from a Redmi phone and dismantled it completely, counting the fields one by one out of the code. A complete report is assembled from three parts, and I will list them by part:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Device and SIM cards: the IMEI, IMSI and ICCID of both SIM slots (the identifier numbers of the phone and of the two SIM cards) and the carrier network code, the identifiers of the cell the phone is connected to (CID, PCI and TAC) and the network type, the 5G and VoLTE switch states, and the phone’s manufacturer, model, and hardware and software versions, roughly 40 fields in total.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Signal and location: this part carries the SIM identifiers a second time and adds the cell metrics to them, meaning the channel number (ARFCN) and the signal readings RSRP, RSRQ and SINR. The embedded Baidu location library contributes latitude and longitude, altitude, speed and heading. These signal and location records are only stored when the phone is connected to a network inside China.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;App usage: each app’s package name and display name, how long it spent in the foreground, and how much traffic it used on each of the two SIM cards and on WiFi separately. The filter in the code has a single condition: an app is skipped when its package name starts with com.android. Apps the vendor preinstalled do not fall under that condition, so they are counted like everything else.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;On his friend’s Xiaomi 14, MeT found the statistics covering the past 24 hours. Any phone carrying this component keeps the same kind of record. The report holds more than the list of what is installed: it also records which of those apps were used inside that window, the total time each one spent in the foreground, and how much traffic each one generated on the first SIM, the second SIM and WiFi. The figures are totals, with no timestamps attached: they say how long each app ran in total, and they do not say at what time the owner opened it. Every record in it carries the identifiers of the phone and the SIM card.&lt;/p&gt;

&lt;p&gt;Its wake-up schedule is written in the code as well. Booting the phone, changing a SIM card or a network change will all wake it, and it also runs on timers. The location collection cycle is one hour, and it uses the kind of exact alarm that can pull the phone out of sleep. If an upload fails, the data waits in a local queue on the device and is sent again later.&lt;/p&gt;

&lt;p&gt;Before sending, the code runs a consent check. It evaluates a variable that its logs call “user permission open.” In the part of the code written for Xiaomi phones, though, that check only verifies one thing: whether the phone’s first-time setup has been completed. You can go into the settings and turn off the consent switch that this check is supposed to depend on, and the check still passes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The VPN’s blind spot
&lt;/h2&gt;

&lt;p&gt;A VPN on Android builds a tunnel through the system’s VpnService and changes which route your packets take out of the phone. That is network-layer machinery, and what it can hide is where your traffic goes and which address you appear to come from.&lt;/p&gt;

&lt;p&gt;SystemHelper’s collection never touches the network. It holds system privileges and calls the operating system’s interfaces directly, on the device itself: one permission to enumerate every installed app, another to read each app’s usage time, another to read the IMEI and the SIM identifiers. All of that reading happens before any packet exists, so no tunnel, however well built, can govern an interface call inside the same machine.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7z3lcjikp3g0b748oq4a.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7z3lcjikp3g0b748oq4a.webp" alt="What a VPN can reach, and what it cannot: collection happens inside the phone, before any packet exists, so the tunnel cannot govern it; on the sending step, tools with blocking rules can hold the data back for a time, but blocking stops the sending, not the collecting" width="800" height="600"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Ordinary apps are blocked from all three of these reads, but each one is blocked in a different way. For the phone and SIM identifiers, Android 10 is the cutoff: since that version, an ordinary app cannot read the IMEI, IMSI or ICCID, and only a preinstalled app holding the signature-level privileged permission can read them. Enumerating every installed app rests on a signature-level privileged permission as well. Usage time is different again, because an ordinary app can read it only when the user goes into settings and grants usage access by hand. A component that arrives preinstalled and platform-signed is blocked by none of the three, and that is exactly the problem.&lt;/p&gt;

&lt;p&gt;Even if you look only at the sending step, the tunnel can still be bypassed. Android’s VPN mechanism itself allows per-app routing, and it allows an app to bind itself to a specific network and skip the tunnel entirely. Whether a system component’s traffic travels through your VPN is not for you, the user, to decide. It is the system’s own default mechanism: users generally cannot change it, and they cannot easily tell whether it happened in any particular case.&lt;/p&gt;

&lt;p&gt;And even if this component’s traffic did go through your VPN, the tunnel itself changes only the source address that observers along the connection’s path see. But once the server receives the data and opens it, it is looking at the same record, carrying the same IMEI and SIM identifiers. Your source address changing has no effect on whether it can tell whose record this is.&lt;/p&gt;

&lt;p&gt;The sending step, though, is not completely beyond reach. Once this data is packed up on the phone, it still has to be sent out before it can reach China Unicom. Some VPN and firewall-type tools let the user set blocking rules that hold back specified destination addresses. If this report’s traffic does pass through such a tool, and the user has put the reporting server’s address into the rules, then while the rule holds, this data does not reach China Unicom.&lt;/p&gt;

&lt;p&gt;This route has a precondition: a blocking rule can only hold back traffic that does go through the tool, and whether this traffic travels through your tunnel, as the earlier paragraph said, is not for the user to decide and is not easy to tell. It also has a limit: what it holds back is the sending, not the collecting.&lt;/p&gt;

&lt;p&gt;The component keeps reading on the device itself and keeps packing; data whose sending failed is stored on the phone first and waits to be sent again later, so if the rule is lifted one day, or the network environment changes, the accumulated data can still be sent out. There is also a difficulty before you even start: to block an address, you first have to know which address to block. The only address confirmed today is the default one written in the code; which server a phone bought in a shop connects to, nobody has confirmed, so even if you put the default address into the rules, you may not be holding back the server it really connects to.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the assembled dataset shows
&lt;/h2&gt;

&lt;p&gt;Look at the app list on its own first. It is a plain record of the owner’s interests. The list shows which bank’s app you installed, whether you installed a medical app or a dating app, and none of that requires opening any app and reading its content.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F02uuzrj9e2dnrrz2ia1d.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F02uuzrj9e2dnrrz2ia1d.webp" alt="One report assembled from four streams — the app list, usage time, per-app traffic and location points — every stream carrying the same IMEI and ICCID, sent to China Unicom; SIM cards are registered under real names, so the identifiers land on one specific person" width="800" height="610"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Add the usage totals, and they show which apps took up your time. Across the past 24 hours, the totals show which apps held the foreground longest and which ones you opened only briefly, so they show which services account for most of the usage time. But the totals carry no timestamps. The record does not say at what time you opened an app, so on its own it cannot say when you sleep or when you get up. The daily routine can be worked out from a different set of records, the location records.&lt;/p&gt;

&lt;p&gt;Readers tend to underrate what the per-app traffic figures reveal. They are counted per app, and the WiFi traffic is counted separately. So even when you are at home on WiFi, where your traffic never touches the carrier’s network at all, the per-app WiFi totals are still written into this record.&lt;/p&gt;

&lt;p&gt;Then location is added to the report. The report carries both cell identifiers and latitude-longitude coordinates. A cell locates the phone only within a rough area, while coordinates give the exact spot. And because the location collection runs on a one-hour cycle, the points accumulate hour by hour. Over a run of days, the points show a pattern the usage totals cannot show: the point the phone returns to at night and the point where it spends the day, and together those two points show where home and the workplace are. With speed and heading in the same records, the route travelled in between them can be reconstructed as well. The daily routine can be worked out from the location records, not from the usage totals.&lt;/p&gt;

&lt;p&gt;If these four parts were held in four separate places, what anyone could learn about a person from them would stay limited. In this report, they arrive pre-assembled, because the apps, the usage time, the traffic and the location in the same batch of records all carry the same set of IMEI and ICCID values. In China, SIM cards are registered under real names, so those identifiers lead, through the real-name registration, to one specific person.&lt;/p&gt;

&lt;h2&gt;
  
  
  What your carrier already holds
&lt;/h2&gt;

&lt;p&gt;Your own carrier already lawfully holds a great deal. It holds your real-name registration and the IMEI your phone presented when it joined the network, and it holds records of which cell towers you connected to. It also records where your traffic went and how much traffic there was. That data is inseparable from running a network and billing for it, in any country.&lt;/p&gt;

&lt;p&gt;What is new here is the part a carrier can never see from the network side. It cannot see which apps you installed, and it cannot see how long each app spent on your screen. It cannot see your WiFi traffic, and GPS-grade coordinates are not in its hands either. On top of that, this whole assembled record is reported to China Unicom while the phone in question carried a China Mobile SIM. The party collecting it does not even have a contract with you.&lt;/p&gt;

&lt;p&gt;The DM arrangement itself was not secretly invented by Xiaomi or China Unicom either. All three Chinese carriers require DM adaptation for the phone models they stock, which is why this component shows up across many brands sold in China. The adaptation branches in its code cover OPPO, vivo, Samsung and ZTE, among others. Huawei and Honor do not appear in that list.&lt;/p&gt;

&lt;p&gt;Those statements about the other brands need one honest limit. So far, the only sample that has been fully dismantled, with its fields counted one by one, is Xiaomi’s. For the other brands, what is confirmed is that the component is present on them, and that it holds the same permissions. Nobody has publicly taken apart the other brands’ builds of the component yet, so whether they report the same way, with the same fields, is unknown, and conclusions drawn from the Xiaomi sample should not be applied to them unchanged.&lt;/p&gt;

&lt;h2&gt;
  
  
  Checking and removing it yourself
&lt;/h2&gt;

&lt;p&gt;Check whether it is there first. Open the app management page in Settings, turn on “show system apps,” and search for SystemHelper. If it appears, it is installed. If you are comfortable with a computer, you can also list the package name with adb, or audit the whole device’s apps and permissions with the open-source tool NOIR, which needs no root.&lt;/p&gt;

&lt;p&gt;If it is there and you want it gone, you can uninstall it for the current user with adb, again without root. Two caveats come first. This removal applies only to the current user, so a factory reset brings it back. Community-maintained debloat lists catalog preinstalled apps that can be removed. These lists warn that the component is tied to the phone’s dual-SIM functions, without saying what the tie consists of or what would break if it were removed. So after removing it, you should check that both SIM cards still call and connect properly. If anything breaks, a command exists to install it back exactly as it was.&lt;/p&gt;

&lt;p&gt;Two popular remedies, however, do nothing. Turning off the consent switch in settings is one, because the code path for Xiaomi phones never reads that switch, as explained above. Turning on an ordinary VPN so the traffic takes a different route is the other. Rerouting changes only the source address, while the collecting and the packing are both completed on the device itself, so rerouting on its own does nothing. A tool with blocking rules does not count as this remedy: if you put the reporting server’s address into the blocking rules, then the data cannot get out for the time being, and its preconditions and limits are covered in the section above.&lt;/p&gt;

&lt;h2&gt;
  
  
  The questions nobody has answered
&lt;/h2&gt;

&lt;p&gt;As of November 5, neither Xiaomi nor China Unicom had issued any public statement about this round of disclosures. How long this data is kept on Unicom’s side, who is allowed to query it, and what procedure a query requires: no public material answers any of that today.&lt;/p&gt;

&lt;p&gt;One detail also remains unverified. Which server an ordinary phone bought in a shop connects to is stored in the component’s own configuration. That configuration cannot be read without root. The phones the researchers took apart were rooted ones. So the only address confirmed is the default one in the code. The auditor who dismantled the Redmi phone caught no actual upload traffic in his short observation window either, so the real reporting frequency has no settled answer yet.&lt;/p&gt;

&lt;p&gt;What can be seen in transit is also unsettled. The default reporting address written into the code is plain HTTP, and the app is permitted to send cleartext traffic. At the same time, the payload body goes through application-layer encryption processing (AES and RSA) in the code before sending. A plaintext address does not by itself mean the body is sent unencrypted. No third party has published a decrypted capture of real traffic. Whether every reporting path receives the same processing has not been separately confirmed — and that includes /register, the registration path at the end of the reporting address. Nobody can say for sure who can see what in transit.&lt;/p&gt;

&lt;p&gt;The phone call from the police that opened this piece belongs in this section too. All that is established is the order of events: the reporting was going on, and the call came after it. Whether the reporting triggered the call has no evidence chain behind it, only the timing and MeT’s friend’s own inference. This piece therefore treats the call only as the reason MeT started digging, and never as a proven consequence of the reporting.&lt;/p&gt;

&lt;p&gt;If you use Telegram or proxy tools on your main phone, my advice is plain: spend ten minutes and check your own device first. If the component is there, then deal with it using the steps above. If you cannot deal with it for the time being, then first use a tool with blocking rules to hold back the reporting server’s address, so the data cannot get out for now. If even that is not possible, then move the sensitive apps to a second device. Do not let a component you cannot see keep recording, day after day, which apps you use and where you go.&lt;/p&gt;

&lt;p&gt;I will keep taking system layers apart like this. If this was useful, consider subscribing to this blog or bookmarking it. More AI, security and open-source pieces, taken apart the same way, are on the way.&lt;/p&gt;

&lt;p&gt;Sources: the V2EX thread “Can we have a little privacy” (&lt;a href="https://www.v2ex.com/t/1254629" rel="noopener noreferrer"&gt;https://www.v2ex.com/t/1254629&lt;/a&gt; ); the independent audit at GitHub, LING71671/systemhelper-privapp-audit; the evidence compilation at beibing173/chinaphone; MeT’s (@MeTerminator666) original posts of October 31, 2026.&lt;/p&gt;

</description>
      <category>android</category>
      <category>cybersecurity</category>
      <category>privacy</category>
      <category>security</category>
    </item>
  </channel>
</rss>
