$ git show b200e23:docs/transcripts/README.md | sed -n '67,72p'
The other check was run separately and is not part of the gate: the
transcript was searched for the exact values of the credentials held in the
files on this machine that were found in the snapshots. It does not contain
any of them. That check is worth repeating whenever a transcript is
rebuilt, because the gate cannot know what a random string on this machine
happens to be.
A pattern check covers the shape; a value check covers the instance. The gate
did not catch it, and nothing here claims that it did.
What it cost
The cost belongs here. A post about process that leaves the cost out is the
thing it criticises.
| Tooling | 453 lines: guardrails.mjs 245, size-budget.mjs 54, guardrails.manifest.json 154 |
| Decision records | 17 |
| Commits that touched the mechanism | 29 of 104 |
| Commits whose subject is process work | 45 of the 84 non-merge commits |
$ git log --format=%h b200e23 -- tools/ docs/decisions/ AGENTS.md CLAUDE.md lefthook.yml .cursor/hooks.json .claude/settings.json .github/workflows/guardrails.yml | sort -u | wc -l
29
$ git log --format=%s b200e23 | grep -cE '^(docs|chore|style|build)'
45
$ git log --format=%s b200e23 | grep -c '^Merge'
20
Twenty of the 104 commits are merges, so 84 sit outside the merge group, and 45
of those 84 are process work.
Read quickly, that ratio looks like displacement: forty-five commits of process
work against a product that shipped, could not build the thing, built the
bureaucracy instead.
There are two answers, and neither of them is that the ratio does not matter.
The first is that the ratio is an artefact of this corpus. The product is a PDF
annotator: one workspace, one operator, thirty days. The budgets above exist
so the ratio cannot become the only thing that matters. A file-size budget and a
test budget are arguments against the shape of the work drifting. They do not
measure it.
The second is that this process work was the product, for the purposes of the
paper. The counts are not a tax on the product. They are the other half of it.
So the lesson is narrower than "write more process":
The cost of a safeguard is paid in process, and should be counted there,
in the commit history, where it is visible.
A safeguard that costs nothing is not a safeguard, it is a sentence. The price is
real, so it goes next to the work it protects, where the next person can decide
whether it is worth paying.
What it does not protect
The limits belong here, not in a footnote. Stating them is part of the
mechanism.
It does not make the code good. It does not decide what to build. It does
not catch a wrong change that satisfies every rule. A hash says a file has
not changed; it says nothing about what is in it.
It does not survive the protected files being deleted and their hashes
removed in the same commit. The manifest cannot hash itself without a
bootstrap, so what stands in that gap is a request for review:
# .github/CODEOWNERS: the second lock
# The owner's review is requested for the safeguards. The Guardrails check
# fails the build if one of these is removed or weakened; this is the second
# lock, and the one that survives a check being edited.
/AGENTS.md @effinrich
/CLAUDE.md @effinrich
/lefthook.yml @effinrich
/package.json @effinrich
/.cursor/hooks.json @effinrich
/.claude/settings.json @effinrich
/.github/workflows/ @effinrich
/tools/guardrails.mjs @effinrich
/tools/guardrails.manifest.json @effinrich
/tools/shell-guard.mjs @effinrich
/tools/stop-gate.mjs @effinrich
/tools/size-budget.mjs @effinrich
/docs/decisions/ @effinrichIt requests. That is the honest word for it.
It does not stand in for the branch. The live rules on main:
$ gh api repos/effinrich/h2-fe-assessment/branches/main/protection --jq '{strict: .required_status_checks.strict, checks: [.required_status_checks.checks[].context], admins: .enforce_admins.enabled, force_push: .allow_force_pushes.enabled, deletions: .allow_deletions.enabled, pull_request_reviews: (.required_pull_request_reviews // "absent")}'
{"admins":true,"checks":["Gate","Guardrails"],"deletions":false,"force_push":false,"pull_request_reviews":"absent","strict":true}
$ gh api repos/effinrich/h2-fe-assessment/rulesets/24388801 --jq '{name, bypass_actors, rules: [.rules[] | {type, reviews: .parameters.required_approving_review_count}]}'
{"bypass_actors":[],"name":"main","rules":[{"reviews":null,"type":"deletion"},{"reviews":null,"type":"non_fast_forward"},{"reviews":0,"type":"pull_request"},{"reviews":null,"type":"creation"},{"reviews":null,"type":"required_status_checks"}]}
The pull-request rule requires zero approving reviews, the ruleset names no
bypass actor, and the required checks are Gate and Guardrails: the gate and
the guardrails, which is the mechanism, wired into the branch.
One scope note, because "no bypass actor" is easy to over-read. A second
ruleset, feature-branches, matches every ref and names eighteen bypass
actors: three repository roles and fifteen integrations.
$ gh api repos/effinrich/h2-fe-assessment/rulesets/24519354 --jq '{name, conditions, actor_types: ([.bypass_actors[].actor_type] | unique), roles: [.bypass_actors[] | select(.actor_type=="RepositoryRole") | .actor_id], integrations: ([.bypass_actors[] | select(.actor_type=="Integration")] | length)}'
{"actor_types":["Integration","RepositoryRole"],"conditions":{"ref_name":{"exclude":[],"include":["~ALL"]}},"integrations":15,"name":"feature-branches","roles":[2,4,5]}
Those three role ids are 2, 4 and 5, and their meaning is not published.
Neither GitHub's REST reference nor its OpenAPI description states the
mapping. The open issue
github/rest-api-description#7111
calls it a "critical security feature, currently undocumented" and asks GitHub
to publish it. The reading in use (maintain, write and admin) comes from that
issue and from the Terraform provider's ruleset reference, not from GitHub.
How those two rulesets combine on the trunk is GitHub's evaluation order, not
something the corpus documents, and this post does not assert it. The
checkable claim is narrower: the two checks the mechanism runs are required on
the branch that carries the mechanism.
Before the failure, the same ruleset was different, and the difference is the
whole of F1:
$ git show b200e23:docs/safeguards.md | sed -n '95,98p'
It required a pull request, no force push and no deletion. It did not
require a status check, and it did not require an approval — zero — and
admins, maintainers and writers could all bypass it. So it protected the
shape of a merge and nothing about its content.
The removal satisfied every rule that existed. The mechanism did not replace
the ruleset. It gave the ruleset something to require.
And the environment, which is a different failure. The machine the agents
ran on was compromised:
$ git show b200e23:docs/safeguards.md | sed -n '11,13p'
During this work the machine the agents ran on was compromised by a trojan.
It was found at the end, the credentials it could reach were rotated, and the
hiring team was told on the day the submission went in.
Top comments (0)