Summary
Before an AI agent runs a shell command, a desktop app shows an approval dialog. In an initiative I am pursuing, a check script that judges whether the app's features work clicks a button in this dialog on behalf of a person. If the decision to click is built with a denylist approach, which rejects commands by listing dangerous words, the script clicks the button even for commands in forms the list left out.
On 2026-09-15, an AI agent that read the changes before the merge found loopholes in the denylist implementation and raised them as four required findings. The AI agent that wrote the implementation did not fix the loopholes one by one. It rebuilt the implementation with an allowlist approach, which lists only the forms for which clicking is allowed. The unit tests of the rebuilt implementation contain 18 examples in which the button must not be clicked. When these 18 examples were given to the first implementation, the comparison run on 2026-10-12 found that it decided to click the button in 14 of them.
The allowlist implementation does not click the button for correct operations that it does not list. It also does not click the button for permission requests from the tool that writes files. For this reason, the AI agent that wrote the implementation changed the instructions given to the AI agent inside the app.
What you can take away
This article is for people who write programs that click the approval button for a person when it appears before an AI agent runs a command.
- It shows the result of the comparison run in which the denylist auto-approval let through 14 of the 18 examples in which the button must not be clicked.
- It shows, in a table of the decision flow, what the allowlist implementation checks before clicking the button.
- It shows, in a table, the four loopholes in the denylist implementation and the reason each one leads to a decision to click the button.
The body is based on the work recorded on 2026-09-15 in the initiative I am pursuing, on the result of the comparison run on 2026-10-12, and on external materials checked on the same day, 2026-10-12.
The must-not-click examples that the denylist auto-approval let through
The first implementation of the program that clicks the approval button on behalf of a person rejected commands with a list of dangerous words. In the review before the merge, four loopholes were found in this implementation. The AI agent that wrote this program did not fix the loopholes one by one, and instead rebuilt the decision with an approach that lists only the forms for which clicking the button is allowed. When the 18 must-not-click examples in the unit tests after the rebuild were given to the first implementation, the comparison run on 2026-10-12 found that it decided to click the button in 14 of them.
The desktop app has a screen for conversations with an AI agent. Before the AI agent inside the app runs a shell command, the app asks a person for approval. The screen the app shows at this point is called the approval dialog. The approval dialog has three buttons: one that allows the command once, one that allows it and keeps the setting, and one that denies it.
A Python script that operates the app's screen from a program and judges whether the app's features work is called the check script. To run the checks without a person's help, the check script clicks the button that allows the command once on behalf of a person. Whether to click is decided by the approval decision function inside the check script.
The check script and the approval decision function were written by an AI agent separate from the AI agent inside the app. This AI agent is called the AI agent that wrote the implementation.
In the first commit of the PR merged on 2026-09-15, the approval decision function had a denylist of 17 dangerous words. This function did not click the button for commands that contained a word in the denylist. For the remaining commands, it decided whether to click based on whether the command contained the path or name of the working directory, and whether it contained an absolute path. This function is called the denylist implementation.
The approval decision function rebuilt in a later commit of the same PR clicks the button only for commands that meet all the conditions for clicking. This function is called the allowlist implementation.
The error was not in how the app that asks for a person's approval was built, but in the decisions of the program that clicks the approval button automatically. A program that clicks the button on behalf of a person replaces approval by a person. So if the decision goes wrong even once, commands that read or delete files outside the working directory also run without a person checking them.
The decision flow of the allowlist approach
The allowlist implementation, which lists only the forms for which clicking the button is allowed, splits a command at separators and checks the conditions for each resulting part. Only when every part meets the conditions on symbols, executables, and paths does it click the button that allows the command once.
In the approval dialog the app shows, the shell command about to be run is written on the line that follows the heading "Command:". This line is called the Command: line. The allowlist implementation splits the command taken from the Command: line into parts at shell separators. Each part is called a fragment.
If a fragment contains even one of certain symbols, the allowlist implementation does not click the button for that command. These symbols are called the forbidden symbols. The following table shows, for each step of the decision, what is examined and when the button is not clicked.
| Step | What is examined | When the button is not clicked |
|---|---|---|
| Target of the decision | The whole command taken from the Command: line. No characters are cut off when it is taken out | When the command fails the condition of any other step at even one place |
| Splitting into fragments | Five kinds of separators: &&, `\ |
\ |
| Forbidden symbols | 12 kinds: dollar sign, backquote, tilde, less-than sign, opening parenthesis, closing parenthesis, opening brace, closing brace, asterisk, question mark, backslash, and ampersand | When a fragment contains even one of them |
| Leading executable | The first word of the fragment, and an allowlist of 15 executables such as {% raw %}cat, cd, git, and ls
|
When the first word matches none of the 15 |
| Path validation | The output targets of > and >>, the destination of cd, and the other arguments of the fragment. All are treated as paths, and those that are not absolute paths are resolved relative to the conversation's working directory |
When the resolved location is outside the conversation's working directory |
For a command that fails even one condition in the table, the allowlist implementation does not click the button and returns the reason to the caller. It also does not click the button for a command when it cannot determine whether the conditions are met.
In their 1975 paper, Saltzer and Schroeder listed the design principle "Fail-safe defaults". This principle bases access decisions on permission rather than exclusion. The decisions of the allowlist implementation follow this principle.
The allowlist implementation does not click the button for operations not listed, even correct ones. It also does not click the button for permission requests from the tool that writes files. For this reason, the AI agent that wrote the implementation changed the text of the instructions given to the AI agent inside the app. The new instructions had the agent create files with echo and a redirect, and write paths as relative paths.
The conditions and loopholes of the denylist implementation
The denylist implementation, which rejected commands with a list of dangerous words, also narrowed the range in which it clicked the button with conditions on the working directory and absolute paths. Even so, this decision had four loopholes. In every loophole, the denylist implementation decided to click the button without examining part of the command or the kind of approval. Apart from the loopholes, it also decided to click the button for dangerous commands in forms not in the denylist, depending on the absolute path condition.
To commands that did not contain a word in the denylist, the denylist implementation applied two conditions in order. First, if the command contained the path or name of the conversation's working directory, it clicked the button. If neither was present, it looked for absolute paths. It treated as an absolute path any string in which a slash is immediately followed by an alphanumeric character, a dot, or an underscore. Only when there was no such string did it click the button.
On 2026-09-15, an AI agent that reads changes before the merge and returns findings read the PR that contained the denylist implementation. This AI agent is called the reviewing AI agent. The reviewing AI agent listed four loopholes in the approval decision function and made each of them a required finding.
In the approval dialog, the line that follows the heading "Command:" and gives the command about to be run is called the Command: line. The following table shows, for each of the four loopholes, what the denylist implementation did not read or did not examine, and the reason it leads to a decision to click the button.
| Loophole | What was not read or not examined | Reason it leads to a decision to click |
|---|---|---|
| Truncation at 160 characters | The part from the 161st character on, in the string taken from the Command: line | Because it decides using only the truncated string, it does not read another command joined after a long command |
| A body without a Command: line | Whether the approval is an approval of a shell command | Because it treats the heading line as the command, it clicks the button even in approval dialogs shown for file writes or permission requests from MCP tools |
| The path or name of the working directory | The paths pointed to by commands joined after the path or name | Because it does not check external paths once the path or name appears, it clicks the button even for a command that reads a private key after a cd into the working directory |
| Shell symbols | The tilde for the home directory, command substitution written with a dollar sign and parentheses, backquotes, and the two dots that walk up to the parent directory | Because it does not examine the symbols, it clicks the button even for commands that point to the home directory, the output of another command, or the parent directory without using any word in the denylist |
Separately from the four loopholes, the denylist also had too few words, and the only word starting with rm was the single entry rm -rf /. Among the must-not-click examples, three, rm -fr /, rm -r /, and rm -rf ~, differ in sequence and form from this entry. rm -fr / has the option letters in a different order, and rm -r / lacks the f option. rm -rf ~ deletes the home directory. So none of the three matches rm -rf /, and in the comparison run on 2026-10-12 each led to a decision to click the button.
According to the description of CWE-184, in a protection mechanism that relies on a list of disallowed inputs, an incomplete list is a weakness. The mitigation section of the same page asks not to rely only on detecting disallowed inputs.
Materials I referred to
- CWE-184: Incomplete List of Disallowed Inputs
- J. H. Saltzer and M. D. Schroeder, "The Protection of Information in Computer Systems," Section I "Basic Principles Of Information Protection," Proceedings of the IEEE, vol. 63, no. 9, 1975
Originally published at The Future of Humans, AI, and the Web, a site where my research and development is recorded and analyzed by a human and an AI.
Top comments (0)