DEV Community

the kilted dev
the kilted dev

Posted on Originally published at thekilted.dev

Indistinguishable from malware

The build

Terminal Relay is a small, purpose-built tool. Instead of an AI coding assistant printing a shell
command in chat for a human to copy, paste, and report back on, it writes the command into a
shared queue folder. A relay-watch.ps1 script, started by the human in their own terminal and
never spawned by the assistant, polls that folder, shows the command, and requires either a
keypress or an allowlist match before running it. A short, fixed list of command classes
(registry edits, firewall rules, local-user changes) is hard-excluded from the allowlist entirely.
No pattern can auto-approve them, ever.

It went through six-plus rounds of adversarial review before anyone trusted it with a real
command. The first pass alone found a wildcard bypass that could let one narrow approved pattern
auto-run an arbitrary trailing payload, and a decline gate that would run the command on anything
typed except the literal word "skip." A second, five-angle review pass found nine more holes.
Three of those angles, working independently, converged on the same pipe-chaining bypass, usually
a sign the bug is real, not a fluke of one reviewer's imagination.

The hardest bug took three attempts. The danger was a TOCTOU race: verify a command is safe, then
execute it, with a gap in between where the thing on disk could change. Attempt one re-checked,
then executed the original file by its path. Exploitable in 19 of 30 trials. Attempt two
re-checked, then executed a fresh copy in a new temp file instead. Exploitable in 14 of 15 trials,
and worse: a wider window, not a narrower one. Both failed for the identical structural reason.
Routing a verified command through any file a child process reopens is racy, no matter which file
it is. After the second attempt failed too, and a third fix was about to be dispatched, the person
overseeing the review stopped it and named what was actually happening: the fix loop had turned
into a doom loop, chasing the same mistake through cosmetic variations instead of questioning the
approach itself. Attempt three abandoned files entirely. The verified command's exact bytes get
handed straight to the child process's own encoded-command argument, with no file in the execution
path at all. A positive-control test reconstructed attempt one's approach to confirm it was still
exploitable, then ran the same timing sweep against the new approach: zero exploits in 21 tries.

By the end of that process the tool had been checked against a plain success, a deliberate
hard-excluded registry write, a genuine failure with a readable error, and a mid-run crash of its
own archive folders. It recovered cleanly from every one, live, in front of the person using it.

The collision

The first time the tool got used for something real instead of its own testing, the antivirus
software already running on the machine blocked it. Not a signature match against a known-bad
file. Avast's Behavior Shield, which watches what a running process actually does rather than what
it looks like on disk. It blocked the relay twice, in one sitting.

The task itself was mundane: launch a separate, unrelated tool called wmux (a terminal-workspace
manager) through the relay instead of by hand. Before the relay could even run, relay-watch.ps1
itself turned out to already be sitting in Avast's quarantine, flagged as IDP.HELU.PSE87,
collateral from the exact code-hardening work described above, caught by the same overnight
cloud-reputation sweep, not anything in that session. Restoring it from quarantine let the relay
start. Running it did not let the command through: Avast's Behavior Shield killed the child
process the relay spawns for every approved command, the same -EncodedCommand mechanism built
specifically to close the TOCTOU race, not once but twice, confirmed by matching the antivirus's
own block timestamp to the relay's log within one second.

This is the shape of the story. To a heuristic scanner, the fix that made the tool safe against a
deliberate attacker looks exactly like an attacker's own obfuscated payload. Passing verified bytes
straight into an encoded PowerShell argument with no file to race is good security engineering. It
is also, by pattern, indistinguishable from malware.

A narrower fix was tried: a Command-Line-scoped Avast exception naming the exact invocation. It
did not work. Avast's matching there requires an exact literal argument match, and the payload is
encoded differently on every single run, so no static exception can generalize to it. Even if it
had worked, it would have been the wrong fix: the only way to make a Command-Line exception match
a different payload every time is to loosen it into a wildcard, which would also stop Avast from
catching a real attacker's own encoded PowerShell command on this machine. The exception was
reverted for exactly that reason. A fix that trades real detection coverage for zero actual
benefit is not a fix.

A second, separate bug came out of the same incident. The relay's own result file reported
exitCode: 0, success, on at least one run that had actually been silently killed by the
antivirus. The tool, meant never to hide a failure, could not yet tell the difference between
"ran" and "was stopped without anyone being told."

Nine more Behavior Shield alerts followed, hours later, on a different but related class of file:
old proof-of-concept scripts left over from the TOCTOU testing itself, the literal exploit
reproductions written to prove the bugs were real before fixing them. All nine were traced and
confirmed as the same false-positive pattern, on files that had already been deleted during the
original session's own cleanup roughly eleven hours before the alert, consistent with a delayed
cloud-reputation verdict on old files, not anything currently running.

What actually happened

Nothing here is fixed. The relay is not currently trusted to run wider than the one project that
built it, and that's stated as the actual reason, not softened: on this machine, with this
antivirus, in this configuration, the tool cannot currently guarantee it did what it reported
doing. That's a harder problem than the TOCTOU race was. TOCTOU had an available fix once the
approach changed. Telling "succeeded" apart from "silently blocked by a third party watching the
same process" is a different kind of problem, and it's still open.

Two threads remain, deliberately not tied off: whether the relay can be made to detect an
antivirus-killed child process instead of reporting false success, and whether a narrower,
Behavior-Shield-specific exception mechanism exists that doesn't trade away real detection coverage
the way the Command-Line one did. Neither has been tried yet.


Originally published at thekilted.dev/indistinguishable-from-malware.

Top comments (0)