I created a tool called aiterm-mcp and published it on npm. It allows an AI to hold a persistent terminal and even operate another AI from there. The story behind it is written in the article when I announced it and the article about AIs conversing with each other.
Recently, this tool has become the center of my development environment. My current setup consists of Fable 5 (Claude's new top-tier model) directing the overall workflow and assigning tasks to Codex and Grok, and all those instructions go through aiterm. When I posted an introduction in English, I started getting responses from overseas. One person evaluated the environment verification before startup, and someone else said, "The ability to maintain a single terminal is more effective than the part where AI drives AI." In the Japanese community, people actually started using it and shared configuration tips.
Even so, the responses were only a few, so I thought almost no one had noticed this tool. When I checked the numbers to write this article, I was surprised to see that npm downloads were 1,416 in the last 30 days. Apparently someone somewhere is installing it every day.
Given that, I've fixed it many times. But no matter how many times I fixed it, I felt like there were still some rough edges somewhere. I couldn't even say what was wrong; all tests passed, and there were no specific bugs I could name. There was a lingering sense of not fully trusting it.
So I decided to consult Fable 5. I just said something like "polish it up" without specifying where to look or what to fix. Looking back, it was a pretty vague request.
The consultation was just one sentence. The audit was assembled based on written rules of thumb (the flow of this incident)
When it came back, it was done
In my environment, I have rules of thumb accumulated over several months. When auditing, have multiple perspectives search in parallel. For each issue that surfaces, pass it to a different AI to question it, and adopt only those that survive. Before touching the code, prepare tests first and run them automatically. After fixing, actually run it to verify. All these are rules I established after painful experiences, and I've written about them in previous articles.
Fable 5 read these rules and assembled the audit on its own. Seven AIs acting as searchers ran in parallel from different perspectives, each issue had a refuter attached, and even an AI to catch oversights—totaling 27 agents. It also showed the code to three external AI systems (GPT-5.6, Grok 4.5, and Composer), and the issues from there were further challenged by six refuters. It took 12 minutes, and the processing amount was 1.34 million tokens (the unit of reading/writing for AI).
Seventeen raw issues came out, and seven survived the refutation. Combined with issues that survived external review, everything was fixed, and the tool went from v0.11 to v0.12.1 that same day. Regression tests increased from 183 to 197. That day, I wrote basically just the first sentence; the rest was driven by the accumulated rules.
The feeling was right
There really were rough edges. I'll introduce just two of what was found.
The first was that the check to stop dangerous commands could be bypassed. Aiterm has a mechanism that asks for confirmation when a dangerous command like rm -rf / (command to delete everything on disk) is about to be sent. The matching pattern didn't account for the standard -- notation, so rm -rf -- / (with two hyphens) passed through the check. The fix was a single line.
The second was a cleanup omission. If a process is forcefully terminated while waiting for an AI's response, a flag file saying "currently waiting" remains. There was no code anywhere to clean up this flag, and if it remained, subsequent waits on the same terminal would be permanently rejected. Codex and Claude's audits independently pointed out the same spot without knowing each other.
The refuters also did their job. The ten discarded issues were plausible mistakes. For example, one issue said, "If writing to the management file is interrupted, the remnants become unmanageable." But reading the code again, there was already an escape path that allowed operation as a normal terminal even with a corrupted management file, and the time window for the problem was on the order of microseconds. If I had swallowed it and added countermeasure code, it would have become unnecessarily complex.
There's a supporting evidence for this "separate challenger" step from a paper this April. A report showed that over 80 AI agents unanimously confirmed a non-existent vulnerability in OpenSSL as real, proving that collecting numbers of AI suggestions doesn't guarantee correctness. Our "17 reduced to 7" is a smaller version of that.
A bug none of the 27 could find
The most troublesome bug was not found by this audit.
Aiterm determines whether an AI has "finished speaking" by checking if the screen output has stopped increasing for 0.5 seconds. This judgment has a timing gap: the moment when the output volume is measured and the moment when the screen state is checked are slightly out of sync. When output arrived in that gap, the fact that it ended was correctly captured, but the record of "how it ended" sometimes differed from reality.
On my local Mac, all tests pass. I found this through GitHub CI (a system that automatically runs tests on multiple operating systems with each code update). The macOS runner on CI is very slow. The problematic test had a wait time of 0.6 seconds versus a stillness threshold of 0.5 seconds, leaving only 100ms of margin. On the slow runner, this slack wasn't enough, and the macOS job consistently failed. That's when I first learned about the bug.
Actual CI screen during v0.12.0 release. Linux and Windows all passed, macOS node 18 failed, and the other two macOS jobs were automatically aborted
The interesting part is that the audit came close. The AI responsible for catching oversights noticed the category of "time-dependent risky tests" and had actually pointed out another test. That test had a 5x margin, so it was judged "no problem," which was correct. The real one remained unpointed. When just reading the code, a test with a 1.2x margin and one with a 5x margin both look like "passing tests."
By the way, this CI was running even before the audit. I had decided from past failures to have automated tests on multiple OSes before touching the code, and I followed that this time too.
Bugs that only appeared when running
Apart from the desk audit, there were two bugs found the same day during "actual running verification."
When I fetched Grok's screen log in "read all" mode, 73,176 characters came back packed into 4 lines. Output from tools that redraw the screen typically has almost no line breaks. Aiterm's folding feature was designed to activate when "the number of lines exceeds 60," so these huge 4 lines passed through unfolded, exceeded Claude's per-message capacity, and the reading itself failed. On paper, the folding feature looks perfectly functional. The case where the length of a single line becomes problematic didn't come up from any of the 27 agents until I actually ran the real Grok.
The other was missing parts of long responses. When an AI's response is long, aiterm returns only the last screenful (about 24 lines). For a 40-line response, only lines 18 onward were captured, and there was no way to retrieve the first half. I solved this by creating a new feature to recover the full response from the AI's conversation history.
These two issues didn't surface from just reading the code. They only appeared when actually running it with real data flowing.
Who found what
The results line up as follows:
Who found what (AI audit / multi-OS CI / actual running verification)
This all got moving from a vague sentence because Fable 5 read the rules accumulated over several months. The "rough edges" I couldn't name materialized as 7 issues and disappeared in a day. The refutation step stopped 10 dangerous assumptions before implementation. And the most troublesome bug was found by the rules my past self had set: having CI in place first, and actually running to verify after fixing.
The bugs found did not overlap at all among these three methods. Hmm. I wonder what that means.
Top comments (0)