DEV Community

gracefullight
gracefullight

Posted on

What OMA returns when its test gate fails

#opensource #testing #ai #devtools

oh-my-agent (OMA) can rerun a configured test script when an active workflow
tries to stop. This example exercises that behavior with manual hook calls and
a deliberately broken expiry check. It uses the OMA 15.0.13 source CLI.

The fixture defines an item as expired at now === expiresAt. Its implementation
used >, so the boundary test failed. The test process exited with code 1.
This excerpt comes from the actual output:

not ok 2 - at expiry: expired
# pass 2
# fail 1
Enter fullscreen mode Exit fullscreen mode

The fixture's active Ralph workflow had a test completion gate. A manual
call to OMA's Stop hook reran the package's test script. The response contained
"decision":"block" and this reason:

Stop gate 'test' FAILED (reinforcement 1/5).
Enter fullscreen mode Exit fullscreen mode

The oma hook run process itself exited with code 0. Its JSON decision carries
the block; treating that exit code as a successful test would be incorrect.

The fix changed one comparison:

-  return now > expiresAt;
+  return now >= expiresAt;
Enter fullscreen mode Exit fullscreen mode

Rerunning the tests produced:

# pass 3
# fail 0
Enter fullscreen mode Exit fullscreen mode

The next Stop call ran the same gate and returned empty stdout. OMA removed the
fixture's workflow state and recorded gate.passed, followed by session.ended
with reason completion_gate_passed.

The demo, receipts, and reproduction script contain the full
commands, timestamps, and output. Download reproduce.py and run it against an
OMA source checkout with its CLI dependencies already available:

python3 reproduce.py --oma-source /path/to/oh-my-agent --output /path/to/new-demo-run
Enter fullscreen mode Exit fullscreen mode

Download demo.html from the Gist to watch the 45-second explanation locally. GitHub displays its source rather than playing it on the page.

The script requires Python 3, Git, Node, and Bun. It creates an isolated fixture
and session store; it does not change your project's workflow state.

This is a local fixture with manually invoked hooks, not a recorded autonomous
agent session or a production incident. It demonstrates this configured gate
on these tests. It does not establish general correctness or a comparison with
another tool.

The full harness supplies the CLI and hooks used here. A skills-only install
does not. Start with one scoped task in the
Quick Start,
then inspect its diff and checks. The source is on
GitHub.

Top comments (0)