v0.1.0 · github.com/shaifulshabuj/teststop
Every test suite ever written was written by someone who already knew how the system works.
Real users don't. That's why production still surprises us. We call it test coverage. What we actually have is assumption coverage — the set of things we imagined going wrong, written down, and checked. The things we didn't imagine are still there, waiting.
teststop flips this. Instead of writing tests, you give AI a mandate to think like a real adversarial user — someone who never read the docs, opens the same form in two tabs, retries three times when the page is slow, and pastes emoji into a date field. AI already knows all of these patterns. It just needed the right instruction.
go install github.com/shaifulshabuj/teststop/cmd/teststop@latest
teststop run
No API keys. No config file. claude or copilot CLI must be on your PATH — that's all.
What It Actually Gives You
Zero-configuration adversarial scenarios
teststop run scans your project, detects the language and system type, composes an AI mandate from that context, and hands it to Claude or Copilot in non-interactive mode. You get back structured scenarios: what the adversarial user tries, what chaos they inject, what failure they expose.
teststop run --depth aggressive --output json
Output is JSON by default — machine-readable, agent-parseable, ready to pipe into your CI pipeline.
Tests that reduce over time (not grow)
Most testing tools add work forever. teststop does the opposite. It maintains a confidence score per system area in .teststop/memory.json:
- +0.19 per passing scenario
- -0.30 per failure
- Areas at 0.95+ confidence are retired — teststop stops hammering them
After ~15 clean passes, an area is proven stable and teststop moves on. The test surface shrinks as your system earns its confidence. Commit memory.json to version control — it's the accumulated proof that your system works.
An AI mandate you can read and improve
The mandate that drives AI behavior is mandate/base.md — plain markdown, checked into the repo, readable and editable. It describes the adversarial user archetypes: the one who never reads docs, the one who double-clicks, the one who pastes unexpected input, the one who has a slow connection and a short fuse.
teststop mandate --show # see exactly what AI receives
The mandate is open — file a PR to improve the adversarial patterns for your domain.
Built for AI agent workflows
Exit codes are designed for autonomous agents:
| Code | Meaning | Agent action |
|---|---|---|
0 |
Confidence met | Safe to deploy |
1 |
Below threshold | Review required |
2 |
Critical failure | Do NOT deploy |
teststop run --output json --quiet --no-color
# Exit 0: proceed. Exit 2: stop.
Apple Container sandbox isolation
When you run teststop, AI executes inside an isolated Apple Container VM — not on your host. The mandate is passed as a CLI argument. AI outputs JSON to stdout. Your filesystem is never touched by the AI process.
Works Everywhere
| Languages | Any — Go, Node, Python, Rust, and more auto-detected |
| AI CLI | Claude Code (claude) or GitHub Copilot CLI (copilot) |
| Sandbox | Apple Container (macOS arm64) or direct exec fallback |
| CI |
TESTSTOP_SANDBOX=none for Docker/non-macOS environments |
If you're shipping AI-assisted code and wondering whether your test suite is actually covering what real users do — teststop is the missing piece.
github.com/shaifulshabuj/teststop · go install github.com/shaifulshabuj/teststop/cmd/teststop@latest
Top comments (0)