Last month I merged a bug fix an AI agent wrote. It looked right. The agent said tests passed. I deployed it.
Two hours later, production caught f...
For further actions, you may consider blocking this person and/or reporting abuse
I like the separation between execution and independently verifiable evidence of what happened. Where do you see the substantive decision criteria behind a PolicyDecision living - inside the protocol implementation, or in a separately versioned and testable policy artifact?
Brian, great question — and it's one we've wrestled with internally. The short answer is: both, but with a strict separation of concerns.
The PolicyDecision itself — the signed, auditable receipt that says "this action is authorized under this policy" — is a protocol-level artifact. It lives in the receipt chain, is versioned by the protocol schema, and its structure is enforced by the OWP implementation. You can't have a valid PolicyDecision that doesn't conform to the schema, just as you can't have a valid HTTP request that doesn't conform to the spec.
But the substantive decision criteria — the actual rules that determine whether an action is permitted — live in a separately versioned, testable policy artifact. In our current implementation, this is a JSON policy document (the policy.json or capability_grant object) that is referenced by digest in the PolicyDecision receipt. The policy artifact is:
Versioned independently of the protocol schema. A PolicyDecision from v0.1 of the protocol can reference a policy artifact at v3.2, and the verifier checks both version constraints independently.
Testable in isolation. The policy artifact can be evaluated against a mock request without touching the receipt chain, making it possible to unit-test authorization logic separately from cryptographic plumbing.
Auditable by reference. The verifier doesn't need the full policy text in the receipt — just its digest. The policy artifact is fetched (or already cached) at verification time, and its digest is checked against the reference in the PolicyDecision.
This separation matters because the two artifacts have different trust properties. The PolicyDecision receipt proves who authorized what, when, and under which policy digest. The policy artifact defines what the rules actually are. If you conflate them — putting the full policy text inside the receipt — you bloat the chain and make policy updates expensive (every policy change requires re-signing all historical receipts). If you separate them, the receipt chain stays lean, and policy evolution is independent.
A subtlety: the policy artifact is itself signed (by the Authorizer role), so it's not just a loose JSON file. The verifier checks: (1) the PolicyDecision signature proves the Authorizer approved this action, (2) the policy digest in the PolicyDecision matches the signed policy artifact, and (3) the action parameters satisfy the policy rules. Three independent checks, three independent failure modes.
Does that map to what you were imagining, or were you thinking of a different boundary — e.g., embedding the policy criteria in a smart-contract-like layer?
The repo is at github.com/dengyier/OpenWorkProof if you want to trace how PolicyDecision and CapabilityGrant interact in the current implementation. The test_policy_decision.py suite covers the digest-matching and version-checking paths.
Yes - that maps closely to the boundary I had in mind.
The distinction I’m exploring is one step inside the substantive policy artifact. A
CapabilityGrantcan answer whether this actor has authority to perform an action, while some enterprise decisions also require a separate judgment over evidence, rules, exceptions, missing information, and escalation conditions before that authority should actually be exercised.That is where I’m experimenting with JPS - a separately versioned and testable judgment artifact that can produce a deterministic disposition such as approve, deny, unresolved, or escalate.
Your design suggests an interesting interoperability test: let JPS produce the judgment, then let OWP bind the exact policy version, facts/disposition, and authorized action into the
PolicyDecisionand receipt chain.The falsifiers would be the interesting part - change the pack version, substitute facts, replay an old decision, or change the execution arguments after approval and see whether an independent verifier detects the broken binding.
That would keep the responsibilities clean:
JPS - what should happen under these facts and rules
OWP - who authorized it, what actually happened, and whether the evidence chain still verifies
I’m going to explore that boundary experimentally.
Brian, this is brilliant — you've drawn the exact boundary that OWP was designed to enable, not to occupy.
Your distinction between "does this actor have authority?" (CapabilityGrant) and "should this authority be exercised, given these facts and rules?" (JPS) is precisely the separation of concerns we've been converging toward, but you've named it and given it a concrete shape.
JPS as a separately versioned judgment artifact is the right abstraction.
CapabilityGrant answers a binary question: is this action within scope? But real enterprise authorization — especially in regulated environments — requires a multi-factor judgment over evidence completeness, rule applicability, exception handling, missing information, and escalation conditions. That judgment is substantive, contextual, and domain-specific. It cannot live inside the protocol schema without bloating it, and it cannot live inside the receipt chain without making receipts non-deterministic. Your JPS layer solves both problems: the judgment is produced independently, versioned independently, tested independently, and then bound into the OWP chain at the exact moment of commitment.
On the interoperability test you proposed:
"let JPS produce the judgment, then let OWP bind the exact policy version, facts/disposition, and authorized action into the PolicyDecision and receipt chain."
This is not just a test — it's a reference architecture for how OWP should integrate with external judgment systems. In our current model, the Authorizer role produces a PolicyDecision that says "this action is permitted under this policy." If we extend this to "this action is permitted under this policy, as adjudicated by JPS v3.2 with these facts and this disposition,** the receipt chain gains a new layer of semantic richness without losing its cryptographic rigidity.
The binding would look like:
PolicyDecision references capability_grant_digest (static authority)
PolicyDecision also references jps_judgment_digest (contextual judgment)
The Verifier checks both: does the capability grant permit this action? and does the JPS judgment support this disposition under these facts?
The Executor runs the action only if both conditions hold
The receipt chain records the full binding: authority + judgment + execution + output
On the falsifiers:
Your proposed attack surface is exactly the right set of stress tests:
Falsifier What it tests
Change the pack version JPS versioning integrity
Substitute facts Fact-binding integrity
Replay an old decision Temporal freshness / nonce validation
Change execution arguments after approval Authorization-to-execution binding integrity
Each of these is a binding-layer attack — not against the cryptography (which OWP already protects), but against the semantic consistency between what was authorized, what was judged, and what was executed. This is the class of failures that pure signature verification cannot catch, but that a well-designed receipt chain can catch by embedding all relevant digests and requiring independent recomposition.
The responsibility split you articulated is clean and powerful:
JPS: What should happen, under these facts and rules
OWP: Who authorized it, what actually happened, and whether the evidence chain still verifies
This is exactly the division we need. OWP is not a judgment engine. It is an authorization-and-execution binding protocol. JPS supplies the judgment; OWP supplies the tamper-proof commitment of that judgment into an auditable action.
A concrete next step:
If you're open to it, we'd love to collaborate on a joint interoperability spec — a minimal schema for how JPS judgments bind into OWP PolicyDecisions, and a shared falsifier test suite that exercises the boundary. We could define:
The JPS judgment object schema (disposition, facts hash, rule version, reasoning trace)
The OWP binding format (how the judgment digest is referenced in PolicyDecision)
A shared test corpus of falsifier scenarios (pack-version mismatch, fact substitution, replay attacks, post-approval argument tampering)
A reference implementation showing an end-to-end flow: JPS adjudicates → OWP binds → Verifier independently reconstitutes both JPS and OWP states → acceptance or rejection
This would give both projects a rigorous, shared definition of the boundary between "judgment" and "commitment" — a boundary that, as you've shown, is where the real engineering complexity lives.
The OWP repo is at github.com/dengyier/OpenWorkProof. The test_policy_decision.py suite covers the current digest-matching and version-checking paths, and we'd be happy to extend it with JPS-binding tests if you have a schema you'd like to align on.
Looking forward to exploring this boundary with you.
Thanks - I took you up on this, although I decided to test the boundary before proposing an interoperability spec.
Study 014 is now frozen and run.
The question was:
Can an independently developed execution-verification protocol bind an executed action to the exact judgment that authorized it strongly enough that an offline third party detects substitution, drift, replay, and execution mismatch?
We pinned OpenWorkProof at
8eeca6fand calledverify_acceptance_bundleunchanged. JPS produced the deterministic judgment, a thin adapter committed the exact pack, facts, disposition, replay tuple, and authorized action into OWP-signed fields, and OWP handled the authorization, receipts, causal chain, and offline verification.Then we tried to break the composition.
The locked stratum had 39 registered cells - mutations plus controls and a demonstration - including variants coherently re-signed with the study keys rather than simple signature tampering. A separate reviewer-authored holdout added 8 cases that were first executed only after the study froze.
Result: zero divergences in both strata. Every registered detection landed on the layer and code predicted for it.
A few things from your design held up especially well.
OWP's unchanged verifier consistently caught tampering, causal-chain failures, authorization-window violations, evidence-set problems, and surplus execution. We also tried to construct additional execution through the retry path and hit a real protocol wall rather than finding a bypass.
The study also exposed two useful boundaries.
First, a coherently reminted alternative valid WorkOrder can pass all chain-internal checks. That was registered as an expected-undetected case, not patched away. Detecting rollback or freshness at that level needs an anchor outside the chain. I think that is an important and defensible boundary.
Second, the generic
metadataenvelope is outside the signed commitment. We demonstrated that a judgment reference placed there can be substituted while OWP still verifies green. Carrying the commitment through signed fields such asWorkOrder.objectiveandAgentRequest.context_source_digestclosed that in our composition. That might be worth calling out explicitly in the docs so downstream integrations do not mistake metadata for a binding point.The reviewer holdout was useful too. One self-consistent wrong-action case had a completely valid OWP chain - the commitment and receipt agreed with each other - but the action was not one the JPS disposition permitted. Only the disposition-to-action binding rejected it. That was probably the clearest evidence for the separation we were discussing: cryptographic consistency and substantive authorization are different checks.
Full study and detection matrix:
github.com/Judgment-Pack/judgment-...
The conclusion stayed deliberately narrow: binding and lineage, not truth. JPS does not prove the facts are true, and OWP does not prove the judgment is correct. But in this registered mutation set, the two layers composed cleanly and the boundary between judgment and verifiable execution held up under considerably more adversarial pressure than I expected.
I also filed one small housekeeping issue as OpenWorkProof #1 - the repository LICENSE is Apache-2.0 while some package metadata still reports MIT.
Thanks again for the detailed architecture explanation. It gave us something concrete enough to falsify rather than just claim was complementary.
Brian — this is extraordinary. You didn't just discuss the boundary. You built the bridge, walked across it, and then tried to blow it up. The fact that both strata held is the strongest external validation OWP has received to date.
I'm going to address this piece by piece because every paragraph contains something actionable.
On Study 014 and the question you posed:
"Can an independently developed execution-verification protocol bind an executed action to the exact judgment that authorized it strongly enough that an offline third party detects substitution, drift, replay, and execution mismatch?"
You answered it: yes. And not just in theory. You pinned OWP at a specific commit, called verify_acceptance_bundle unchanged, ran JPS as the judgment layer, used a thin adapter for binding, and then subjected the composition to adversarial mutation. The fact that you could do this with a "thin adapter" — not a fork, not a rewrite — is exactly the interoperability proof we were hoping for.
On the 39 registered cells + 8 reviewer holdout cases:
47 adversarial test cases with zero divergences is a remarkable result. That you included "variants coherently re-signed with the study keys rather than simple signature tampering" is especially important — it tests the protocol's resilience against sophisticated attacks, not just naive ones. The separate reviewer-authored holdout is good experimental hygiene. That both strata reported zero divergences means the binding between JPS judgment and OWP execution is cryptographically tight.
On "we tried to construct additional execution through the retry path and hit a real protocol wall":
This is the best possible outcome. A protocol that can be bypassed through retry logic is not a protocol; it's a suggestion. The fact that OWP's retry handling rejected surplus execution attempts means the causal chain is actually enforcing policy, not just logging it.
On Boundary 1 — coherently reminted alternative valid WorkOrder:
"A coherently reminted alternative valid WorkOrder can pass all chain-internal checks... Detecting rollback or freshness at that level needs an anchor outside the chain."
This is a profound and honest boundary. You're identifying a class of attacks that OWP's chain-internal verification is not designed to detect — and correctly labeling it as an expected-undetected case rather than a bug to patch. This is exactly the kind of disciplined security analysis that prevents protocols from promising more than they can deliver.
In OWP terms, this maps to the time-anchor problem: a fully valid receipt chain can be replayed in its entirety if an attacker controls the clock or rolls back the entire system state. Detecting this requires an external freshness anchor — a timestamp or nonce from a source the attacker cannot control (e.g., a blockchain timestamp, a trusted time server, or a counterparty's independent clock). This is a known limitation that we've documented but not yet hardened.
Your finding validates our decision to keep the protocol scope narrow: binding and lineage, not truth or freshness. We should make this boundary explicit in the docs.
On Boundary 2 — generic metadata envelope outside the signed commitment:
"A judgment reference placed there can be substituted while OWP still verifies green... Carrying the commitment through signed fields such as WorkOrder.objective and AgentRequest.context_source_digest closed that in our composition."
This is a critical documentation bug on our part. You're absolutely right: any field outside the signed commitment is not a binding point, no matter how convenient it is to put data there. If downstream integrators treat metadata as a place to embed judgment references or policy digests, they'll have a false sense of security.
Action items from this finding:
Update verify_acceptance_bundle documentation to explicitly warn: all binding commitments must flow through signed fields
Add a test case: test_metadata_substitution_attack — a mutation where a valid OWP chain has its metadata altered but still passes verification, proving that metadata is not a security boundary
Consider removing or restricting the metadata field in the next schema revision, or at minimum adding a metadata_digest in the signed envelope so metadata alterations break the chain
On the reviewer holdout case:
"One self-consistent wrong-action case had a completely valid OWP chain — the commitment and receipt agreed with each other — but the action was not one the JPS disposition permitted. Only the disposition-to-action binding rejected it."
This is the clearest evidence for the separation we've discussed. Cryptographic consistency and substantive authorization are different checks. OWP proves the chain is internally consistent. JPS proves the action is substantively authorized. Neither replaces the other. This single case validates the entire architecture.
On the narrow conclusion:
"JPS does not prove the facts are true, and OWP does not prove the judgment is correct. But in this registered mutation set, the two layers composed cleanly and the boundary between judgment and verifiable execution held up under considerably more adversarial pressure than I expected."
This is the gold standard for protocol evaluation. You tested the boundary under adversarial pressure, found it held, and correctly refused to overclaim. "Under considerably more adversarial pressure than I expected" is a sentence we will quote in the OWP documentation.
On the LICENSE issue (OpenWorkProof #1):
Thank you for catching this. You're right — some package metadata is still reporting MIT while the repository is Apache-2.0. We'll fix this immediately. This is exactly the kind of sharp-eyed review that makes external contributions so valuable.
What we'd like to do next:
Link to your study: Would you be open to us referencing Study 014 in the OWP documentation and README? The github.com/Judgment-Pack/judgment-... link you included — we'd like to add it as a reference implementation of JPS/OWP interoperability.
Integrate your two boundaries into the security model: We'll add explicit documentation on (a) the external-freshness-anchor requirement for rollback detection, and (b) the metadata field limitation.
Add your holdout case to the test suite: The "self-consistent wrong-action" case — where the OWP chain is valid but the JPS disposition rejects the action — is a perfect test for the disposition-to-action binding. We'd like to add it as a reference test.
Co-author the interoperability spec: You now have the only working JPS/OWP integration in the world. If you're willing, we'd like to co-author the interoperability specification with you, using Study 014 as the reference implementation.
This is no longer a discussion. It's a verified, tested, interoperable protocol boundary. Thank you for doing the work to prove it.
Great question! Verifying AI agent work is crucial for trust. I build PureHub, a privacy-first open-source tool collection, and while it doesn't do AI verification, it does offer tools for cryptographic signing and verification that might complement your workflow. For your specific protocol, I'd suggest checking out existing standards like Sigstore or Rekor for transparency logs. What's your main challenge—ensuring the agent's identity or the integrity of the work output?
Thanks for the pointer, PureHub! You're right that Sigstore and Rekor are highly relevant reference points — they've solved a big chunk of the supply-chain transparency problem for traditional software. The OWP protocol draws on similar principles (signed digests, immutable log chains, key-bound identity), but we had to extend the model in two specific directions for the AI agent case:
Identity alone isn't enough. In Sigstore, identity is "who pushed this container image." In OWP, it's "which agent, acting under which policy, with which capability grant, issued this command." The challenge is that AI agents are delegated actors — their authority is scoped and time-bound, and the verifier needs to independently confirm that scope hasn't been exceeded. So identity is the starting point, not the endpoint.
Output integrity needs to be re-executable, not just signed. A signed container image is a static artifact. A signed AI action receipt is a claim about a dynamic execution. The verifier must be able to independently reconstruct the execution environment (from container_image_digest + command_digest) and re-execute to verify, not just check the signature. That's where the protocol's recomposition design comes in.
Rekor's transparency log model is close to what we want for the receipt chain, but we'd need to extend it to support retraction — the ability to mark a previously accepted receipt as REFUTED without rewriting the original record. That's a unique requirement for agents that run continuously and may produce results that degrade over time (stale context, changed state, etc.).
Your PureHub tools for cryptographic signing and verification might actually be a good fit for the lower-level receipt operations in OWP. We'd be curious to compare notes — especially if there's a clean way to integrate your signing primitives into a role-bound capability grant system. If you're interested, the OWP MCP server is available on PyPI and glama.ai. and the full source is on GitHub: github.com/dengyier/OpenWorkProof. A quick pip install open-work-proof + pip install mcp_server will get you to a local verifier. We'd love your take on whether it extends cleanly.