An offline scope check can catch an inconsistency in a proposed task before dispatch. A runtime gate controls whether a request actually proceeds. Test and effect evidence describe what executed and what happened.
I'm the maintainer of Agent Role Contracts (ARC), an offline declaration checker. Recent conversations around ARC made these boundaries concrete.
Scope and syntax catch different failures
In a maintainer-run synthetic comparison, ARC rejected an out-of-scope declaration that a Godot syntax scanner accepted. The scanner rejected Godot 3 syntax that ARC's scope check accepted. The four-case fixture and pinned inputs are public. ShipRules confirmed the split matched its expectations; its independent run remains pending.
The useful question is what each check adds to the workflow. Passing a scope check says nothing about whether the code parses.
A green result needs execution context
Jackson Ly described a deliberately broken Android test remaining green because Gradle reused a cached result, then clarified that undeclared file inputs can also leave a task UP-TO-DATE. This is his reported example, not an ARC reproduction.
For that failure mode, record the relevant inputs and whether the test actually executed. A broken-then-repaired result is stronger evidence than a green badge alone.
A target can become stale before dispatch
Auten described host-side target reconstruction from a prior element snapshot and acknowledged the stale-ID gap. Its account reports passing a sanitized-schema request to Andrius.
Our proposed comparison changes snapshot freshness first, holding page/site identity constant, then tests identity separately. We have not received runner evidence or an executed comparison. A declared target and the target resolved at execution are separate things to examine.
One small real task exposed a documentation problem
An external evaluator supplied synthetic JSON that I reproduced, then reported a small real runbook edit, about three minutes of setup and a Keep decision. I independently fetched the public one-line edit and reproduced the real-task declaration inputs with matching bundle and task byte hashes: valid declaration PASS → injected out-of-scope declaration FAIL → repaired PASS.
That verifies the declaration result and public edit. Original pre/post execution, timing and comparative value remain the evaluator's report. Actual independent review was not exercised. The helper's negative control changes declaration data in memory; it does not attempt a forbidden write.
The evaluator said most of the initial roughly 15-minute setup went into mapping workflow actors and file paths to the bundle. That feedback points to a simple improvement: explain the mapping before the commands.
- Who writes: a scoped executor with permitted paths in
authority.allowed_write_scopes. - Who reviews: a separate read-only role, connected through the route's reviewers.
- What this task requests: its
inputs.scopepath. - Who actually performs those roles: established by the runtime and review process, beyond the declaration check.
Exact file scopes are useful when only two files should change. A subtree is appropriate only when every path under it should be writable. A valid declaration does not shrink a broader runtime allow-list.
Try the comparison in your existing workflow
What contradiction does the declaration check catch earlier? What does your runtime already stop? What evidence would show that the intended task actually ran?
A small scoped task and its current controls are enough to start. Overlap, setup friction and a decision to remove the extra check are useful results too. The evaluation kit and evidence discussion are public.
Thanks to the contributors for the concrete examples and corrections. These are bounded findings, not a claim of verified runtime enforcement or broad adoption.
Top comments (0)