I'd bind the green result to the exact tree, not just a branch name. Branches move. A verification record should carry the head SHA, base SHA, and tool/config version, and the merge gate should compare that SHA with the candidate. Otherwise, a valid pass can quietly refer to different code. This is one of the problems I'm addressing with verification receipts in Codapult Guard.
Founder of Reply Intelligence Group. 25 years shipping software for the BBC, Sky, LEGO and Barclays. Now building Slipmark: make AI agents prove what they checked.
Agreed, and that's where it ended up. Slipmark binds each verdict to the SHA-256 of every file it covered, not the branch or even the commit. So a pass can't refer to different code, and an unrelated commit elsewhere doesn't invalidate it.
The config part landed in v0.5.0. The policy is read from the base branch and its hash goes in every report. The agent's own instructions (AGENTS.md, skills, MCP config) are hashed too, so a verdict lapses if what the agent was told changes.
Interested to see how you're doing receipts in Codapult Guard. Are you signing them?
Not currently. Guard's verification receipt is machine-readable evidence, but it's not cryptographically signed.
It records the commit, policy base and fingerprint, approval decision IDs, check statuses, finding counts, contract issues, waiver state, and Guard version. The effective policy revision and fingerprint are also recorded in the run manifest, so the receipt can be traced back to the policy used for that run.
But I don't currently claim that this proves who ran the check or that the artifact hasn't been replaced. That's a separate integrity and provenance problem.
Your per-file hashing approach is interesting, especially because it binds the verdict to exactly what was checked rather than invalidating it on every unrelated commit. Signing receipts and tying them to CI identity is something I'd consider as a separate extension, rather than pretending the current receipt already provides that guarantee.
Founder of Reply Intelligence Group. 25 years shipping software for the BBC, Sky, LEGO and Barclays. Now building Slipmark: make AI agents prove what they checked.
That's an honest line to draw. Slipmark had the same gap until it added signing. An unsigned attestation is only a claim about who produced it. Now attest --sign uses the CI job's own GitHub identity through Sigstore, so the gate can require verdicts signed by a specific workflow on main. Your receipt fields would map well onto an in-toto statement if you go that way. Happy to compare notes.
That makes sense. I'd keep signing outside Guard's core runtime: Guard should produce a deterministic, content-bound receipt, and CI should wrap it in an in-toto attestation using its OIDC identity.
The important part is the verification policy, not just a valid signature. The gate needs to require the expected issuer and trusted workflow identity, then match the attested scope, PR head/base, and policy fingerprint against the change being evaluated. A pull_request workflow can come from the PR's merge commit, so I wouldn't let a workflow modified by that PR become the authority signing its own verdict.
I'd be interested in comparing how Slipmark binds and verifies those claims.
Founder of Reply Intelligence Group. 25 years shipping software for the BBC, Sky, LEGO and Barclays. Now building Slipmark: make AI agents prove what they checked.
Agreed on both points, and that's how the verify side works. A signature counts for nothing until the policy names who it trusts. signatures.trusted lists exact workflow identities and signatures.issuer pins the OIDC issuer (GitHub Actions by default). A verdict signed by anything else is refused as signer_not_trusted.
Your pull_request point is the one that catches people out. On a PR the identity ends in @refs/pull/<n>/merge, which the PR author controls. So the docs say to trust only the workflow at refs/heads/main, run by push, workflow_dispatch or pull_request_target, or a reusable workflow pinned to a protected ref. Since v0.5 the trust list comes from the base branch's policy file, so a PR can't add itself to it.
On scope, Slipmark doesn't match on PR head and base SHAs. The signature covers the attestation, which names the hash of every file it covered, and the gate re-hashes those files in the tree being merged. That's why an unrelated commit leaves a verdict standing.
Details are under Signed verdicts in github.com/founder-ship-it/Slipmar.... If Guard ends up emitting in-toto statements, I'd be keen to see whether a Guard receipt and a Slipmark verdict could sit in the same bundle.
Product leader and systems analyst in fintech and edtech. Founder of AnalystCraft. Writing about requirements, acceptance criteria, agent evaluation, and Test-Driven Product Development.
I'd add an adversarial case for the list of required subjects itself. Keep all recorded file hashes correct, but remove an important path from require, or add a new implementation file that no attestation names. Does the gate still refuse? The file receipt can be sound while the coverage policy is incomplete; I'd keep that policy outside the change being reviewed, then bind its version into the decision too.
Founder of Reply Intelligence Group. 25 years shipping software for the BBC, Sky, LEGO and Barclays. Now building Slipmark: make AI agents prove what they checked.
The new file case is covered if you turn on diff coverage (coverage-base: origin/main). Any changed file withut a current verdict gets refused. It's off by default though, which probably needs rethinking.
The policy one is a real gap. Right now verify reads policy.json from the PR's own checkout, so a PR can loosen the rules it's being checked against. Same for the require list in the workflow.
The fix I'm going with.... read the policy from the base branch, not the PR, and put the policy hash in the report so you can see which rules actually made the decision.
Going into the next release. Thanks for this, exactly the kind of thing I was hoping someone would poke at.
Founder of Reply Intelligence Group. 25 years shipping software for the BBC, Sky, LEGO and Barclays. Now building Slipmark: make AI agents prove what they checked.
The policy is now read from the branch you're merging into, not from the PR. Drop a path from require, loosen the policy, delete it, and the gate still judges you by the rules on main. The change only applies after merge.
Every report now records which policy made the decision, with its hash. That's your "bind its version into the decision" point. Required paths can live in the policy file now too, so they come from base as well.
The one gap left is the workflow file itself, since GitHub runs a PR's workflow from the PR. The docs say to put CODEOWNERS on .github/workflows/ and .slipmark/ for that.
Getting it reviewed turned up three more holes in the fix itself (subfolder runs, a git tag posing as main, a symlinked .slipmark folder), all closed now. So thanks twice over. You're credited in the changelog and the release notes.
We're a place where coders share, stay up-to-date and grow their careers.
I'd bind the green result to the exact tree, not just a branch name. Branches move. A verification record should carry the head SHA, base SHA, and tool/config version, and the merge gate should compare that SHA with the candidate. Otherwise, a valid pass can quietly refer to different code. This is one of the problems I'm addressing with verification receipts in Codapult Guard.
Agreed, and that's where it ended up. Slipmark binds each verdict to the SHA-256 of every file it covered, not the branch or even the commit. So a pass can't refer to different code, and an unrelated commit elsewhere doesn't invalidate it.
The config part landed in v0.5.0. The policy is read from the base branch and its hash goes in every report. The agent's own instructions (AGENTS.md, skills, MCP config) are hashed too, so a verdict lapses if what the agent was told changes.
Interested to see how you're doing receipts in Codapult Guard. Are you signing them?
Not currently. Guard's verification receipt is machine-readable evidence, but it's not cryptographically signed.
It records the commit, policy base and fingerprint, approval decision IDs, check statuses, finding counts, contract issues, waiver state, and Guard version. The effective policy revision and fingerprint are also recorded in the run manifest, so the receipt can be traced back to the policy used for that run.
But I don't currently claim that this proves who ran the check or that the artifact hasn't been replaced. That's a separate integrity and provenance problem.
Your per-file hashing approach is interesting, especially because it binds the verdict to exactly what was checked rather than invalidating it on every unrelated commit. Signing receipts and tying them to CI identity is something I'd consider as a separate extension, rather than pretending the current receipt already provides that guarantee.
That's an honest line to draw. Slipmark had the same gap until it added signing. An unsigned attestation is only a claim about who produced it. Now
attest --signuses the CI job's own GitHub identity through Sigstore, so the gate can require verdicts signed by a specific workflow on main. Your receipt fields would map well onto an in-toto statement if you go that way. Happy to compare notes.That makes sense. I'd keep signing outside Guard's core runtime: Guard should produce a deterministic, content-bound receipt, and CI should wrap it in an in-toto attestation using its OIDC identity.
The important part is the verification policy, not just a valid signature. The gate needs to require the expected issuer and trusted workflow identity, then match the attested scope, PR head/base, and policy fingerprint against the change being evaluated. A
pull_requestworkflow can come from the PR's merge commit, so I wouldn't let a workflow modified by that PR become the authority signing its own verdict.I'd be interested in comparing how Slipmark binds and verifies those claims.
Agreed on both points, and that's how the verify side works. A signature counts for nothing until the policy names who it trusts.
signatures.trustedlists exact workflow identities andsignatures.issuerpins the OIDC issuer (GitHub Actions by default). A verdict signed by anything else is refused assigner_not_trusted.Your pull_request point is the one that catches people out. On a PR the identity ends in
@refs/pull/<n>/merge, which the PR author controls. So the docs say to trust only the workflow atrefs/heads/main, run bypush,workflow_dispatchorpull_request_target, or a reusable workflow pinned to a protected ref. Since v0.5 the trust list comes from the base branch's policy file, so a PR can't add itself to it.On scope, Slipmark doesn't match on PR head and base SHAs. The signature covers the attestation, which names the hash of every file it covered, and the gate re-hashes those files in the tree being merged. That's why an unrelated commit leaves a verdict standing.
Details are under Signed verdicts in github.com/founder-ship-it/Slipmar.... If Guard ends up emitting in-toto statements, I'd be keen to see whether a Guard receipt and a Slipmark verdict could sit in the same bundle.
I'd add an adversarial case for the list of required subjects itself. Keep all recorded file hashes correct, but remove an important path from require, or add a new implementation file that no attestation names. Does the gate still refuse? The file receipt can be sound while the coverage policy is incomplete; I'd keep that policy outside the change being reviewed, then bind its version into the decision too.
Thats a good catch, and you're right !
The new file case is covered if you turn on diff coverage (coverage-base: origin/main). Any changed file withut a current verdict gets refused. It's off by default though, which probably needs rethinking.
The policy one is a real gap. Right now verify reads policy.json from the PR's own checkout, so a PR can loosen the rules it's being checked against. Same for the require list in the workflow.
The fix I'm going with.... read the policy from the base branch, not the PR, and put the policy hash in the report so you can see which rules actually made the decision.
Going into the next release. Thanks for this, exactly the kind of thing I was hoping someone would poke at.
Shipped in v0.5.0: github.com/founder-ship-it/Slipmar...
The policy is now read from the branch you're merging into, not from the PR. Drop a path from require, loosen the policy, delete it, and the gate still judges you by the rules on main. The change only applies after merge.
Every report now records which policy made the decision, with its hash. That's your "bind its version into the decision" point. Required paths can live in the policy file now too, so they come from base as well.
The one gap left is the workflow file itself, since GitHub runs a PR's workflow from the PR. The docs say to put CODEOWNERS on .github/workflows/ and .slipmark/ for that.
Getting it reviewed turned up three more holes in the fix itself (subfolder runs, a git tag posing as main, a symlinked .slipmark folder), all closed now. So thanks twice over. You're credited in the changelog and the release notes.