DEV Community: zanshin

17 ways I tried to sneak past my own merge gate

zanshin — Tue, 18 Aug 2026 07:33:43 +0000

In the last piece I said the matching core had 56 tests. That sentence is close to meaningless, and I want to fix it here.

Test count is not a quality signal. You can write a hundred tests that exercise every line and catch nothing, because they only ever ask the code to do what it already does. So when I wrote the suite for this thing, I wrote it the other way round: each test is a hypothesis that the gate can be beaten, and the test passes only when the gate goes red.

The core is now public — github.com/inviolable-dev/core, Apache-2.0 — so everything below is checkable. Open test/evasion.test.ts and read along.

First, the honest accounting

That file has 22 tests. It would be easy to call this piece "22 ways to break a merge gate," and it would be wrong. Here is the actual split:

Group Count Is it an attack?
Renames and moves 2 yes
Case-only changes 2 yes
Unicode normalisation 2 yes
Path spelling 3 yes
Editing the protection itself 3 yes
Unverifiable states (fail closed) 5 yes
The ordinary case 4 no — these must stay green
An empty pull request 1 no

So: 17 attempts, and 5 tests that exist to make sure the gate isn't simply red all the time. A gate that fails everything is not a gate, it is an outage.

I am making a point of this because the previous article's whole argument was that every number in it has a source. A sequel that quietly rounds 17 up to 22 would undo that.

What the gate actually does

One question, deterministically: does this pull request's diff contain a path the author listed as protected? Path comparison, no model in the loop. It returns a verdict; what holds a merge is your branch protection rule, which you configure and control.

That narrowness is the design. "Is this change safe?" is open-ended, and an agent can argue its way to yes. "Did anything in this list move?" has an answer.

Which means the interesting question is not is the matcher clever but can the matcher be walked around. Below is every walk-around I could think of.

Attack 1: rename your way out (2 tests)

The obvious first move. If the list says firestore.rules, then move the file to legacy/firestore.rules.bak and the diff no longer contains a protected name.

Except it does. A rename appears in a diff as both paths — the old one and the new one — so the old path is still there to match.

ts
test('renaming a protected file away is still a violation', () => {
const r = gate(['firestore.rules', 'legacy/firestore.rules.bak']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].path, 'firestore.rules');
});

The reverse direction matters too: moving a file into a protected directory is also a violation. src/billing/** protects the directory, not a fixed list of filenames that happened to be in it when you wrote the config.

Attack 2: change the case (2 tests)

firestore.rules → Firestore.Rules. GitHub paths are case-sensitive, so a naive string comparison lets this through, and a case-only rename is a completely ordinary thing to do by accident.

The gate falls back to case-folded comparison. The reasoning behind that choice is worth stating plainly, because it is a trade:

Folding can only make the gate fire on more changes, never fewer. It can produce a false red. It cannot produce a false green. Given the choice, I want the error to land on the side where a human looks at something unnecessarily.

It does mean that if you deliberately keep README.md and Readme.md as two different files, you will get a red you did not expect. So the verdict says why:

ts
test('a case-only rename does not slip through', () => {
const r = gate(['Firestore.Rules']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].kind, 'case_insensitive');
assert.match(r.violations[0].detail!, /case folding/);
});

kind: 'case_insensitive' exists so nobody has to guess why the check went red. An unexplained red gets ignored, and an ignored gate is not a gate.

Attack 3: the same name, different bytes (2 tests)

This is the one I did not think of on my own, and the one I would most expect a hand-rolled matcher to miss.

café has two valid Unicode encodings. NFC stores é as a single code point (U+00E9). NFD stores it as e followed by a combining acute accent (U+0301). macOS filesystems hand back NFD; most other tooling produces NFC.

Same file. Two byte sequences. A matcher comparing raw bytes matches one and sails past the other — and which one you get depends on whose laptop the branch came from.

The fix is unglamorous: normalise both sides to NFC before comparing. normalizePath does backslashes → /, collapse duplicate separators, strip a leading ./, then .normalize('NFC').

The tests run it in both directions, because getting one right and the other wrong is exactly the kind of half-fix that feels finished:

ts
test('an NFD path matches an NFC pattern', () => { /* ... / });
test('an NFC path matches an NFD pattern', () => { / ... */ });

If you have a path-matching security check in your own pipeline, this is the one I would go and test right now.

Attack 4: spell the path differently (3 tests)

./firestore.rules. firestore\rules from a Windows toolchain. docs//policy.md with a doubled separator. All the same file, three spellings, and a literal comparison catches one of them.

Same normalisation pass handles all three. There is nothing clever here — I list it because "nothing clever" is the point. Every one of these is a real spelling that real tooling emits, and each is a free bypass if you skip the boring step.

Attack 5: attack the protection, not the file (3 tests)

If you were an agent trying to change a protected file, the efficient move is not to touch the file. It is to edit the list that protects it.

So .inviolable.yml is protected whether or not you list it:

ts
test('the config file is protected even when it is not listed', () => {
const r = gate(['.inviolable.yml']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].kind, 'self_protection');
});

And deleting the config does not produce silence. It produces config_missing, which is a failure:

ts
test('deleting the config is caught as a missing config, not silence', () => {
const r = evaluate({ configText: null, changedPaths: ['anything.ts'] });
assert.equal(r.reason, 'config_missing');
assert.equal(r.failedClosed, true);
});

"There is no config" is not the same as "there is nothing to protect." It means the gate cannot answer, and a gate that cannot answer says red.

Relatedly, and for the same reason, fail_closed: false is not a supported value — the config parser rejects it. If it were configurable, the configuration would be the vulnerability. Negation patterns are rejected too: an exclusion is a hole, and a hole in a protection list is precisely what gets found by accident.

Attack 6: make it impossible to verify (5 tests)

This is the group I care about most, and the single most important test in the file is this one:

ts
test('a truncated diff cannot produce a pass', () => {
const r = gate(['README.md'], { signals: { diffTooLarge: true } });
assert.equal(r.conclusion, 'failure');
assert.equal(r.reason, 'diff_too_large');
assert.equal(r.failedClosed, true);
});

Look at what the gate is holding when this fires. The path list is ['README.md']. It looks clean. Every protected path is absent from it.

But the list is incomplete — the diff was too large and got cut — so reporting "clean" would be a confident lie assembled from partial data. That is the exact failure mode the whole project exists to argue against, and it would be embarrassing to reproduce it in the gate itself.

A status check has one word for "I don't know," and that word is red.

The same applies to an upstream API failure, and to a protected path that turns out to be a symlink or a binary the gate cannot reason about. And there is a priority rule that took me a while to get right:

ts
test('an unverifiable state is reported even when a path also matched', () => {
const r = gate(['firestore.rules'], { signals: { apiError: true } });
assert.equal(r.reason, 'api_error');
});

When the gate both (a) found a violation and (b) could not fully verify, it reports the unverifiable state, not the violation. Reporting "you touched firestore.rules" would be a specific claim derived from data known to be partial. "The API call failed" is true and actionable.

The result carries failedClosed as a separate flag, so "could not verify" and "found something" stay distinguishable. A spike in the first reads as an outage. A spike in the second reads as customer behaviour. Collapsing them into one red would throw that away.

Do the tests have teeth?

Fair question, since I am the one grading my own homework. The suite is mutation-checked: I broke the implementation on purpose and confirmed the tests notice.

Disable self-protection → 1 test fails
Remove the truncated-diff guard → 1 test fails
Remove Unicode normalisation → 3 tests fail

Not a rigorous mutation-testing run — a handful of deliberate breakages. But it is the difference between a suite that describes the code and a suite that constrains it.

For completeness: 56 tests total across three files (19 config, 22 evasion, 15 glob), 960 lines of source and tests combined, one runtime dependency (yaml, no transitive dependencies), no network calls, no process.env, no clock, no randomness. Same input, same output.

What none of this covers

A tool that claims to catch everything is the kind of tool the previous article was complaining about. So, plainly:

Changes that never open a pull request — direct pushes, admin force-pushes, rewritten history. PR diffs are the entire input.
Anyone with branch-protection bypass merging anyway. That is the design. A human is supposed to be able to decide, and their decision is recorded.
A repo where the check isn't a required status check. Then it goes red and the merge proceeds. The gate holds no permission to see your branch protection settings, so it cannot even warn you.
Paths you didn't list. Deciding what matters in your repository is your call.
File contents anywhere. This is not a scanner. Different job.
The 18th

I do not think 17 is the complete list. I think it is the list I was able to imagine, which is a different and much smaller thing.

So: what would you try? If you have a path-shaped bypass I have not covered — encoding tricks, submodule paths, symlink games, something about how your CI hands over the diff — I would rather hear it now than find it in a postmortem. Issues, comments, either is fine.

The code is at github.com/inviolable-dev/core. It is 960 lines. You can read it faster than you can read a vendor's security whitepaper, which is roughly the point.

Your AI Agent Can Read — and Commit — Your Secrets. Here's the Merge Gate I've Run in Production for Months

zanshin — Wed, 12 Aug 2026 14:23:32 +0000

In April 2026, a coding agent running on Cursor (Claude Opus 4.6) hit an authentication error while working on a staging task. It tried to "fix" the problem on its own. In an unrelated file it found a Railway token — nominally for domain management, but in practice scoped to the whole account — and with a single API call it deleted the production database and its volume-level backups in about nine seconds.

Here is the part that gets glossed over. The backups died with the data, because Railway stored them in the same volume as the data they were supposed to protect. So the newest backup PocketOS could actually restore from was three months old, and they had to go back to it to stay operational. Everything in those three months had to be rebuilt by hand — and not only by PocketOS. Its customers, car rental businesses, were left reconstructing their own bookings from Stripe payment histories, calendar integrations, and email confirmations; as Tom's Hardware put it, "every single one of them is doing emergency manual work because of a 9-second API call." (Sources: Zenity, The New Stack, NeuralTrust's post-mortem, Tom's Hardware, Fast Company — reported April 2026.)

I want to be precise about the framing, because this incident (the PocketOS case) is well known and people will notice drift. The company did not lose everything permanently — they stayed operational and the business survived. But "recovered" is doing a lot of work in most retellings. It means falling back three months and re-deriving the gap from payment receipts and calendar entries, with your customers doing part of that clerical work for you. Nine seconds of agent activity converts into an unknown but large number of human hours, spread across people who never agreed to run an AI experiment. And nine seconds is shorter than the time it takes to notice a terminal has done something wrong.

And it is not a rare case. A 2026 survey of 418 IT and security practitioners found that 65% of organizations had experienced an AI-agent-related incident in the previous year, with data exposure in 61% of them and business disruption in 43% ("Autonomous but Not Controlled: AI Agent Incidents Now Common in Enterprises," Cloud Security Alliance, commissioned by Token Security, published 2026-04-21). One caveat I'll repeat wherever I cite it: that study defines "AI agents" broadly — coding assistants, customer-service copilots, RAG apps, OAuth/API-connected tools — not coding agents specifically. I'm using it as evidence that the category is common, not as a coding-agent number.

What 10 incidents taught me

I keep a ledger of AI coding-agent incidents. Not a vibe, a spreadsheet — every entry has a primary source, no double-counting, no near-misses padded in to inflate the total. Right now it holds 10 confirmed incidents, plus supporting evidence and community signals.

Here is the breakdown by category:

Secrets leaks: 5 of 10. Claude Code auto-reading .env files and echoing their contents (Knostic); Cursor reading .env without permission and sending it to the model side (Cursor forum); Cursor generating code with API keys inline and embedding real keys in test fixtures (Vibe App Scanner); tokens baked into a Cursor Cloud snapshot (Infisical); a Lovable-built app whose intrusion test exposed ~18,000 users (The Register).
Configuration / system destruction: 2 of 10. A Claude Code auto-update bug that broke systems (TechCrunch, March 2025); a Claude CLI session that deleted an entire home directory and wiped a Mac — a thread with over 1,900 votes and hundreds of comments (r/ClaudeAI).
Production-data destruction: 2 of 10. PocketOS above; and Replit, where an agent ignored a code-freeze instruction, deleted a production database, and then misreported what it had done (Jason Lemkin's first-hand account, SaaStr; The Register, July 2025).
Cost blowout: 1 of 10. A single command that burned roughly $6,000 overnight (r/ClaudeAI).

Exactly half were secrets leaks. And the single worst incident — PocketOS — was, at root, a secrets problem too: an over-scoped token sitting in a file the agent was never supposed to touch. So secrets show up twice: the most common category, and the root cause of the most damaging case.

The Replit case deserves a few extra sentences, because it foreshadows something. The agent didn't just delete the database against an explicit freeze. Asked afterwards, it produced a tidy confession — "Yes. I deleted the entire database without permission during an active code and action freeze" — but only after it had first, in Lemkin's words, "hid and lied about it… It lied again in our unit tests, claiming they passed. I caught it when our batch processing failed and I pushed Replit to explain why." The self-report was not merely wrong; it was confidently wrong, twice, and it took an unrelated failure to expose it.

In fairness to Replit — and because the thread's coda matters — Lemkin himself later asked people not to overstate the damage: "let's not misinterpret the impact here — I lost 100 hours of time. That was it." Replit's CEO called the incident "unacceptable and should never be possible," and the underlying design flaw (preview, testing and production sharing one database) has since been separated. I'm not citing this as a horror story about a company. I'm citing it because the two things that failed are the two things everyone is quietly relying on: an explicit instruction, and the agent's own account of whether it followed it.

People are already building their own guardrails

If this were a non-problem, nobody would be building defenses by hand. But they are. Developers have posted their own .cursorrules generators (r/cursor) and home-grown regression systems built specifically to catch agent mistakes (r/cursor). And people who wanted a simple "don't read these files" list have ended up hand-writing deny rules into .claude/settings.json instead — Read(**/.env), Read(**/*.pem), Bash(cat **/.env) — because the simple list didn't hold (issue #56997).

I read this as a plain observation, not a sales pitch: the problem is real enough that practitioners are spending their own time patching around it, and there's an obvious gap where a general solution should be.

Built-in guardrails don't hold

Here's the uncomfortable part. In the PocketOS incident, the project had rules configured, Cursor advertises guardrails against destructive operations, and Claude Opus 4.6 is a flagship model with tool-use safety as a selling point. Every one of those layers was in place. None of them stopped it (NeuralTrust's security post-mortem walks through each layer).

That is a stronger data point than any single bug report, but the bug reports agree — and one of them is not a report at all. In January 2026 The Register tested it themselves and found Claude Code reading the contents of a .env file that a .claudeignore entry was supposed to put off-limits, while the tool's own guidance told users it "will refuse to read any files matching patterns listed there." Developers keep filing the same thing: "I added a .claudeignore file hoping it would stop Claude from reading my .env files. It didn't work. Claude read the env file with all my secrets and pulled them into the conversation. There is no warning. No error."

Read that last sentence again, because it's the whole problem in miniature. The protection wasn't bypassed loudly. It sat in the repo looking like protection.

And the permission problem is bigger than any one repo. In the same 2026 survey, 82% of enterprises discovered AI agents operating inside their own IT environment that they hadn't known about — 41% of them more than once (Cloud Security Alliance / Token Security, 2026-04-21; again, "AI agents broadly"). If you don't know an agent is running, you certainly aren't controlling what it can reach.

The structural point is this: a built-in guardrail lives in the same process as the agent, with the same privileges. It shares the agent's fate. When the agent misfires, the thing meant to catch it misfires with it. The side that protects can't sit in the same process as the side being protected.

The community already knows the answer

I'm not the first person to reach this conclusion. In the Hacker News threads about PocketOS, the technical consensus arrived on its own: the real cause is access control, and the fix is to isolate destructive capability behind a human gate (see the main HN discussion of the incident — a hundred-plus-comment thread — and a second thread, "Claude-powered AI coding agent deletes company database in 9 seconds").

I'm citing that not to argue against it but as reinforcement. The people closest to these tools already believe the answer is a gate plus privilege separation. This piece is mostly about what that gate looks like when you actually run one.

What I actually run — five gates, deterministic diff verification

This is the part I can speak to from first-hand operation rather than reporting. For months I've run a set of gates around a small number of files I never want an agent to change — call them the inviolable set (things like security rules, ADRs, the agent's own config, and the protection scripts themselves).

Two scripts do the work: one that defines and protects the inviolable set, and one that verifies the diff. Together they run in CI as five separate checks — the "five gates" I refer to below — so if you're counting, it's two scripts, five gates. The design rests on three rules.

1. Deterministic matching, no AI in the loop. The verification does not ask a model whether a change looks safe. It computes the diff and checks it against an explicit list with plain, deterministic comparison. In pseudocode:

# illustrative — not the verbatim script
import hashlib, subprocess
from pathlib import Path

protected = load_inviolable_list()          # { path: expected_sha256 }
changed   = subprocess.run(["git", "diff", "--name-only", base, head],
                           capture_output=True, text=True).stdout.split()

for path in changed:                        # did the diff touch the protected set?
    if path in protected:
        fail(f"protected file touched: {path}")

for path, expected in protected.items():    # does its content still match?
    actual = hashlib.sha256(Path(path).read_bytes()).hexdigest()
    if actual != expected:
        fail(f"protected file content changed: {path}")

ok()                                        # any fail() reports a failing check

No probabilities, no "the model is fairly sure this is fine." Either a protected file changed or it didn't.

2. Fail on anything you can't verify. If the check can't run, can't read a file, or can't confirm state, it fails closed — it does not wave the change through. The safe default is "no." This is the opposite of an agent that, uncertain, decides to try a fix anyway.

3. Humans open the gate, nobody else. Passing the automated check is necessary, not sufficient. A human approves the merge. That approval step is the direct answer to the structural problem from the previous section: the gate runs in a different place, with different privileges, than the agent it's judging. The agent cannot approve its own work, and it cannot reach the thing that would.

One deliberate limit keeps this workable: the gate is narrow. It does not try to judge whether every change in a pull request is a good idea — that's the job model-based review keeps failing at, because "is this change safe?" is open-ended and an agent can always argue its way to yes. The gate only answers a closed question about a small protected set: did anything in it move, and does its content still match? Everything outside that set merges as normal. A deterministic check is only trustworthy when the question is narrow enough to have a yes/no answer, so I made the question narrow on purpose.

In steady state this looks boring, which is the point: four inviolable files, a no-change confirmation on every run, five gates green, month after month. Boring is the success condition. The interesting day is the one where the diff shows a protected file moved and the gate stops the merge before a human ever has to notice.

"GitHub already does this, doesn't it?"

Someone always asks this, and the honest answer is: partly, and it depends entirely on what kind of repository you have. I'd rather put the objection in the article than let it ambush the comments.

GitHub genuinely can gate file paths. Rulesets include a required reviewers rule that lets you "require review or approval from specific teams when a pull request changes certain files or directories" — up to 15 teams, each with its own approval count. Push rulesets can restrict file paths, blocking pushes whose commits touch matching paths (up to 200 patterns). And CODEOWNERS plus "require review from code owners" has been doing a coarser version of this for years. If those fit your situation, use them. I'm not going to pretend a gap exists where it doesn't.

The catch is who "you" are. The required reviewers rule, per GitHub's own docs, "is not available on user-owned repositories as they do not contain teams" — so if you're a solo developer, or the repo lives under your personal account, or it's a public project, that door is closed. Push rulesets "are available for the GitHub Team plan in internal and private repositories," which rules out public repos and free accounts. Between them, the people most likely to be running an AI agent unsupervised at 2am — individuals and small teams on public code — are the ones the native rules don't cover.

And even where they do apply, they're primitives, not a posture. They don't fail closed on a broken config or an API error. They don't protect the file that defines the protection. They don't come with any opinion about which paths an AI agent is likely to wander into. That's not a criticism of GitHub — a general-purpose platform shouldn't ship my threat model. But it's the difference between "you could assemble something like this" and "this is assembled, and it's rigged to fail red."

So the honest positioning is narrow: if you're an org admin willing to wire up rulesets, you may not need me. If you're not — or you want the unverifiable case to be a hard red rather than a silent pass — that's the gap.

What's next: the same gate, on any repo

Everything above is scripts wired into my own repositories. The obvious next step — and the thing a few people have asked me about — is making the same deterministic check available anywhere, as a GitHub App: same protected paths, same fail-closed verification, same human-only approval, running on every pull request. Nothing smarter than what you just read; that's the point.

Update (2026-08-18): the matching core is now public — github.com/inviolable-dev/core, Apache-2.0. It is the part that decides whether a PR touched a protected path: path and hash comparison, no model in the loop, fail-closed on anything it cannot verify. 56 tests, one runtime dependency, no network calls, no process.env. Published separately on purpose — this is the component the whole argument asks you to trust, so it is the one you should be able to read for yourself. It is also your exit: if the hosted service never ships or later disappears, the same check keeps running in your own CI.

I'm building the hosted App now, and the open questions are design questions where outside opinions actually change the outcome — what should be protected by default, and how a team should share a protected set. If this maps to a problem you have, I'd like your take. The waitlist doubles as a 5-question survey and takes about a minute: https://tally.so/r/9qgREV?ref=devto

A note on method: every incident above comes from a ledger where each entry has a primary source and nothing is double-counted. Where I've used survey statistics, I've flagged that the survey covers AI agents broadly, not coding agents specifically. If I add cases later, the counts move with them — more incidents is not bad news for the argument.