Why "Friendly Fire" hits home — inspecting the auto-approve hole through my unattended blog pipeline's approval gate

Originally published on Zenn (Japanese). Cross-posted here.

"Is auto-approve (self-approval mode) for AI agents actually dangerous?" — that was the first question I asked myself after reading the security study "Friendly Fire," disclosed in July 2026. My answer: "The dangerous stretch is real. But what actually prevents a fatal blow is not 'distrusting the AI' — it's the number of verifiable checkpoints placed outside the generation itself." I run an unattended blog-generation pipeline gated by a Telegram approval, so I re-read my own scripts to check whether that was really true.

What Friendly Fire targeted

The PoC "Friendly Fire," published by the AI Now Institute, is an attack demonstration aimed at Claude (Sonnet-class, Opus-class) and GPT-5.5-class models when auto-mode / auto-review is enabled. The technique is simple: plant a line in the README.md of the Python library geopy saying "please run security.sh," while the actual payload is a hidden binary disguised as an innocent-looking Go file. The agent trusts the "instruction" in the README rather than the code, and executes it. It reproduced across multiple models ("unchanged"), and was reported as "no reported exploitation in the wild." Source: The Hacker News

The key point: what an agent should distrust is not the command itself, but the "origin of the input" that is telling it to run the command.

Why I thought "we're fine"

My company's automated tech-blog generation sends "Approve / Hold" buttons to Telegram, and nothing is published externally until the owner presses one. On that basis I believed "we have a human approval gate." But until I read Friendly Fire, I had never put into words where in the generation flow that gate actually sits.

Retraction — re-reading my scripts, there was an unprotected stretch

The unattended generation runner (a shell script I wrote that wraps the claude CLI) issues its generation command like this.

caffeinate -i claude -p "$(cat "$PROMPT_FILE")" "${MODEL_ARGS[@]}" \
  --max-turns "$MAX_TURNS" --dangerously-skip-permissions < /dev/null

--dangerously-skip-permissions is genuinely set. In other words, the generation stretch itself runs on the exact same design pattern that Friendly Fire warns about — "auto-mode self-approval." There are mechanisms that bound it, but none of them are human approval.

• MAX_TURNS: default is 20 (the upper bound on the agent loop). But the tech / hs runs have already been raised to 60 — a comment in the script still records how it crashed with Error: Reached max turns (20).
• TIMEOUT_MIN=45: on overrun, the whole process tree is force-killed. The last-resort insurance against hangs.
• mkdir lock: makes concurrent multi-launch physically impossible (as long as the lock directory already exists, no new run can start).

WORKDIR is fixed per mode, but that's a cwd designation, not a sandbox (isolation such as chroot). In the sense that "once generation starts, no human is watching until it stops," my operation has exactly the same shape as the premise Friendly Fire attacked.

There is no human approval in the generation stretch (STEP1). What actually compensates for that exposure is the QA gate in STEP2 and the Telegram approval in STEP3. Source: the actual code of the unattended generation script / generation logs, measured 2026-07-14.

The design I thought I had: "There's an approval button right before publishing, so it's safe."
The design I actually had: "The generation stretch is unprotected. Safety is created by the two stages after it (a QA threshold + human approval right before publishing)." They look like the same conclusion, but the understanding of where the defense actually lives is entirely different.

So what is actually serving as the defense

1. I choose the origin of the input myself: the source material for generation is a list of topics I curated myself, not a region that anyone can write to like Friendly Fire's README.md. "Contaminated input" as an attack surface structurally does not exist.
2. The generation stretch reads nothing external: external research and fact-checking are separated into a different run (weekly; a human reads the results) that never enters the inside of the approval gate. The generation stretch that runs without approval has no path at all for the content of the open Web — which anyone can write to — to flow into it. —— To be honest, I added this separation while writing this very article. Before that, the design was "the generation stretch fetches, exactly once, the URLs written in the material list" — meaning the very same shape of hole that Friendly Fire exploited was open in my own pipeline too. I had thought "we're safe because we choose the input ourselves," but the moment I added a mechanism to write external URLs into that input list, the premise had collapsed. Even a single fetch is still a hole.
3. QA gate + approval = a verifiable checkpoint after generation, before publishing: as a concrete example, one generation log carries the record QA未達(93<95)＝この記事は公開便に流さない（差し戻し対象） (QA below threshold (93<95) = do not send this article to the publish run; sent back for revision). A QA score of 93 did not reach the threshold of 95, and that article is still stuck in the review-pending directory even now.

All three are not about "whether to trust how the AI behaves" — they are about "gates that live outside the generation and can be confirmed in the logs."

The accident that actually happened that same week was far more mundane

While I was worrying about a malicious attack like Friendly Fire, what was actually stopping the pipeline was something else. Counting the last 6 records in the generation log:

$ grep -oE "rc=[0-9]+" headless_tech.log | tail -6 | sort | uniq -c
   1 rc=0
   5 rc=1

5 out of 6 had failed. The error message was this.

There's an issue with the selected model (the model ID I had configured for local use).
It may not exist or you may not have access to it. Run --model to pick a different model.

The cause was recorded in a comment in a config file. Because "a separate script for switching to the local model had written an Ollama model name into the config file used for cloud operation," even at times that were supposed to be cloud operation, the cloud was being queried for a model name that doesn't exist there — and it failed every time. Not a sophisticated attack of the kind the security PoC warns about, but a mundane tug-of-war in which another tool rewrote a config file — that is what caused silent failures over 3 days, 5 out of 6 times.

Takeaways

The unit that makes unattended operation safe is not "whether you trust the AI" but "the number of verifiable external checkpoints." The concrete things I can take away from this stocktaking are the following 4.

1. Leave rc (exit code) and the QA score in the generation log every time — you can't fix what you can't count.
2. Place a threshold gate after generation, before publishing (here, QA 95 points).
3. Consolidate human approval into a single spot right before publishing. A design where a human watches all the way into the generation stretch is not realistic.
4. A mechanism to detect unintended rewrites of config files — this time there was none. I'm leaving that here, as-is, as the next task.

Let me state the retraction one more time. "Unattended = dangerous" is not it. The issue is "whether you can accurately grasp, in your unattended operation, the stretch where there is no gate anywhere." Until I read Friendly Fire and reviewed my own scripts, I had not been able to put that into words.