Fair split. A Stop hook only proves the test command exited 0, so it won't notice a wrong role or a task scope that drifted, and it can't tell that an acceptance check got weaker. My guess for your two cases: the out-of-scope file or role is a preflight or review catch, and the hook stays green. The weakened check is where required CI plus protected test files (CODEOWNERS on the test directory, say) earns its keep, since the hook just runs whatever tests the agent left behind. Happy to look at the fixture, and shiprules-free is public if you want it as the baseline repo.
Founder & CEO of Netsujo. Building trustworthy AI agent systems, with a focus on task boundaries, verification, and production reliability. Creator of Agent Role Contracts (ARC).
Thanks—that’s exactly the split. I’ve pinned shiprules-free main at 8b8302e9551fb8784d7cbe7fa61eeecf127bf247 and opened the reproducible comparison here: github.com/suirindo/agent-role-con...
I’ll publish the pinned source, exact commands/results, overlap, setup cost, and false refusals. I’m keeping weakened-test semantics as a separate follow-up because the free repo does not contain the paid hook/CI files, so I won’t infer those implementations. Until you review or run it, I’ll label this maintainer-executed—not external hands-on evidence.
Nice, that four-case layout makes the split easy to read. Case 2 (scanner green, out of scope) and case 3 (in scope, Godot 3 syntax) are the two I'd expect each layer to own, so seeing them fail on opposite sides is the useful result. 8b8302e is current main, so the pin is right. Agreed on keeping weakened-test semantics separate since the hook and CI files aren't in the free repo. I'll read through the issue and run the cases when I get a chance, and I'll say so there if I do.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Fair split. A Stop hook only proves the test command exited 0, so it won't notice a wrong role or a task scope that drifted, and it can't tell that an acceptance check got weaker. My guess for your two cases: the out-of-scope file or role is a preflight or review catch, and the hook stays green. The weakened check is where required CI plus protected test files (CODEOWNERS on the test directory, say) earns its keep, since the hook just runs whatever tests the agent left behind. Happy to look at the fixture, and shiprules-free is public if you want it as the baseline repo.
Thanks—that’s exactly the split. I’ve pinned shiprules-free main at
8b8302e9551fb8784d7cbe7fa61eeecf127bf247and opened the reproducible comparison here: github.com/suirindo/agent-role-con...The fixture has four cases:
I’ll publish the pinned source, exact commands/results, overlap, setup cost, and false refusals. I’m keeping weakened-test semantics as a separate follow-up because the free repo does not contain the paid hook/CI files, so I won’t infer those implementations. Until you review or run it, I’ll label this maintainer-executed—not external hands-on evidence.
Nice, that four-case layout makes the split easy to read. Case 2 (scanner green, out of scope) and case 3 (in scope, Godot 3 syntax) are the two I'd expect each layer to own, so seeing them fail on opposite sides is the useful result. 8b8302e is current main, so the pin is right. Agreed on keeping weakened-test semantics separate since the hook and CI files aren't in the free repo. I'll read through the issue and run the cases when I get a chance, and I'll say so there if I do.