In the last piece I said the matching core had 56 tests. That sentence is close to meaningless, and I want to fix it here.
Test count is not a quality signal. You can write a hundred tests that exercise every line and catch nothing, because they only ever ask the code to do what it already does. So when I wrote the suite for this thing, I wrote it the other way round: each test is a hypothesis that the gate can be beaten, and the test passes only when the gate goes red.
The core is now public — github.com/inviolable-dev/core, Apache-2.0 — so everything below is checkable. Open test/evasion.test.ts and read along.
First, the honest accounting
That file has 22 tests. It would be easy to call this piece "22 ways to break a merge gate," and it would be wrong. Here is the actual split:
Group Count Is it an attack?
Renames and moves 2 yes
Case-only changes 2 yes
Unicode normalisation 2 yes
Path spelling 3 yes
Editing the protection itself 3 yes
Unverifiable states (fail closed) 5 yes
The ordinary case 4 no — these must stay green
An empty pull request 1 no
So: 17 attempts, and 5 tests that exist to make sure the gate isn't simply red all the time. A gate that fails everything is not a gate, it is an outage.
I am making a point of this because the previous article's whole argument was that every number in it has a source. A sequel that quietly rounds 17 up to 22 would undo that.
What the gate actually does
One question, deterministically: does this pull request's diff contain a path the author listed as protected? Path comparison, no model in the loop. It returns a verdict; what holds a merge is your branch protection rule, which you configure and control.
That narrowness is the design. "Is this change safe?" is open-ended, and an agent can argue its way to yes. "Did anything in this list move?" has an answer.
Which means the interesting question is not is the matcher clever but can the matcher be walked around. Below is every walk-around I could think of.
Attack 1: rename your way out (2 tests)
The obvious first move. If the list says firestore.rules, then move the file to legacy/firestore.rules.bak and the diff no longer contains a protected name.
Except it does. A rename appears in a diff as both paths — the old one and the new one — so the old path is still there to match.
ts
test('renaming a protected file away is still a violation', () => {
const r = gate(['firestore.rules', 'legacy/firestore.rules.bak']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].path, 'firestore.rules');
});
The reverse direction matters too: moving a file into a protected directory is also a violation. src/billing/** protects the directory, not a fixed list of filenames that happened to be in it when you wrote the config.
Attack 2: change the case (2 tests)
firestore.rules → Firestore.Rules. GitHub paths are case-sensitive, so a naive string comparison lets this through, and a case-only rename is a completely ordinary thing to do by accident.
The gate falls back to case-folded comparison. The reasoning behind that choice is worth stating plainly, because it is a trade:
Folding can only make the gate fire on more changes, never fewer. It can produce a false red. It cannot produce a false green. Given the choice, I want the error to land on the side where a human looks at something unnecessarily.
It does mean that if you deliberately keep README.md and Readme.md as two different files, you will get a red you did not expect. So the verdict says why:
ts
test('a case-only rename does not slip through', () => {
const r = gate(['Firestore.Rules']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].kind, 'case_insensitive');
assert.match(r.violations[0].detail!, /case folding/);
});
kind: 'case_insensitive' exists so nobody has to guess why the check went red. An unexplained red gets ignored, and an ignored gate is not a gate.
Attack 3: the same name, different bytes (2 tests)
This is the one I did not think of on my own, and the one I would most expect a hand-rolled matcher to miss.
café has two valid Unicode encodings. NFC stores é as a single code point (U+00E9). NFD stores it as e followed by a combining acute accent (U+0301). macOS filesystems hand back NFD; most other tooling produces NFC.
Same file. Two byte sequences. A matcher comparing raw bytes matches one and sails past the other — and which one you get depends on whose laptop the branch came from.
The fix is unglamorous: normalise both sides to NFC before comparing. normalizePath does backslashes → /, collapse duplicate separators, strip a leading ./, then .normalize('NFC').
The tests run it in both directions, because getting one right and the other wrong is exactly the kind of half-fix that feels finished:
ts
test('an NFD path matches an NFC pattern', () => { /* ... / });
test('an NFC path matches an NFD pattern', () => { / ... */ });
If you have a path-matching security check in your own pipeline, this is the one I would go and test right now.
Attack 4: spell the path differently (3 tests)
./firestore.rules. firestore\rules from a Windows toolchain. docs//policy.md with a doubled separator. All the same file, three spellings, and a literal comparison catches one of them.
Same normalisation pass handles all three. There is nothing clever here — I list it because "nothing clever" is the point. Every one of these is a real spelling that real tooling emits, and each is a free bypass if you skip the boring step.
Attack 5: attack the protection, not the file (3 tests)
If you were an agent trying to change a protected file, the efficient move is not to touch the file. It is to edit the list that protects it.
So .inviolable.yml is protected whether or not you list it:
ts
test('the config file is protected even when it is not listed', () => {
const r = gate(['.inviolable.yml']);
assert.equal(r.conclusion, 'failure');
assert.equal(r.violations[0].kind, 'self_protection');
});
And deleting the config does not produce silence. It produces config_missing, which is a failure:
ts
test('deleting the config is caught as a missing config, not silence', () => {
const r = evaluate({ configText: null, changedPaths: ['anything.ts'] });
assert.equal(r.reason, 'config_missing');
assert.equal(r.failedClosed, true);
});
"There is no config" is not the same as "there is nothing to protect." It means the gate cannot answer, and a gate that cannot answer says red.
Relatedly, and for the same reason, fail_closed: false is not a supported value — the config parser rejects it. If it were configurable, the configuration would be the vulnerability. Negation patterns are rejected too: an exclusion is a hole, and a hole in a protection list is precisely what gets found by accident.
Attack 6: make it impossible to verify (5 tests)
This is the group I care about most, and the single most important test in the file is this one:
ts
test('a truncated diff cannot produce a pass', () => {
const r = gate(['README.md'], { signals: { diffTooLarge: true } });
assert.equal(r.conclusion, 'failure');
assert.equal(r.reason, 'diff_too_large');
assert.equal(r.failedClosed, true);
});
Look at what the gate is holding when this fires. The path list is ['README.md']. It looks clean. Every protected path is absent from it.
But the list is incomplete — the diff was too large and got cut — so reporting "clean" would be a confident lie assembled from partial data. That is the exact failure mode the whole project exists to argue against, and it would be embarrassing to reproduce it in the gate itself.
A status check has one word for "I don't know," and that word is red.
The same applies to an upstream API failure, and to a protected path that turns out to be a symlink or a binary the gate cannot reason about. And there is a priority rule that took me a while to get right:
ts
test('an unverifiable state is reported even when a path also matched', () => {
const r = gate(['firestore.rules'], { signals: { apiError: true } });
assert.equal(r.reason, 'api_error');
});
When the gate both (a) found a violation and (b) could not fully verify, it reports the unverifiable state, not the violation. Reporting "you touched firestore.rules" would be a specific claim derived from data known to be partial. "The API call failed" is true and actionable.
The result carries failedClosed as a separate flag, so "could not verify" and "found something" stay distinguishable. A spike in the first reads as an outage. A spike in the second reads as customer behaviour. Collapsing them into one red would throw that away.
Do the tests have teeth?
Fair question, since I am the one grading my own homework. The suite is mutation-checked: I broke the implementation on purpose and confirmed the tests notice.
Disable self-protection → 1 test fails
Remove the truncated-diff guard → 1 test fails
Remove Unicode normalisation → 3 tests fail
Not a rigorous mutation-testing run — a handful of deliberate breakages. But it is the difference between a suite that describes the code and a suite that constrains it.
For completeness: 56 tests total across three files (19 config, 22 evasion, 15 glob), 960 lines of source and tests combined, one runtime dependency (yaml, no transitive dependencies), no network calls, no process.env, no clock, no randomness. Same input, same output.
What none of this covers
A tool that claims to catch everything is the kind of tool the previous article was complaining about. So, plainly:
Changes that never open a pull request — direct pushes, admin force-pushes, rewritten history. PR diffs are the entire input.
Anyone with branch-protection bypass merging anyway. That is the design. A human is supposed to be able to decide, and their decision is recorded.
A repo where the check isn't a required status check. Then it goes red and the merge proceeds. The gate holds no permission to see your branch protection settings, so it cannot even warn you.
Paths you didn't list. Deciding what matters in your repository is your call.
File contents anywhere. This is not a scanner. Different job.
The 18th
I do not think 17 is the complete list. I think it is the list I was able to imagine, which is a different and much smaller thing.
So: what would you try? If you have a path-shaped bypass I have not covered — encoding tricks, submodule paths, symlink games, something about how your CI hands over the diff — I would rather hear it now than find it in a postmortem. Issues, comments, either is fine.
The code is at github.com/inviolable-dev/core. It is 960 lines. You can read it faster than you can read a vendor's security whitepaper, which is roughly the point.
Top comments (0)