- Book: LLM Observability Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
Your agent runs the test suite. One test fails. Hundreds pass, and the runner tells the agent about every one of them: a header, progress for each file, the names of passing tests, a summary, and somewhere in there the stack trace it actually needs.
Test reporters were written for a person scrolling a terminal. A coding agent doesn't scroll. It reads what the command printed.
The useful part can get cut. A command that fails reaches Claude Code inline only up to roughly 10,000 characters. Past that, Claude gets "a head-and-tail excerpt of that size", with no file path to the rest. A red test run exits non-zero, so it counts as a failure, and it is the run where the middle matters most.
And whatever fits can stay. Claude Code "sends your full conversation with every request", so the names of passing tests can ride along on later turns until the conversation is compacted or cleared.
You can't change how the agent reads. You can change what the runner prints.
Measure what your agent reads first
Run the suite the way an agent does: piped, not in an interactive terminal, with stderr merged into stdout. Then count.
npx vitest run 2>&1 | wc -c
npx vitest run --reporter=minimal 2>&1 | wc -c
wc -c counts bytes. For plain ASCII output that is also characters, the unit Claude Code's limits use. I'm not printing numbers here, because they depend entirely on your suite: how many tests, how much they log, how deep the stack traces go. Run it on your project, before and after each flag below, and keep the flags that move your count.
A small loop compares several variants in one go:
for cmd in "pytest" "pytest -q" "pytest -q --tb=short"; do
printf '%-24s' "$cmd"
sh -c "$cmd" 2>&1 | wc -c
done
sh -c runs each string as a full command, flags included, in bash or zsh. Run this from your own shell, not inside an agent session. Vitest changes its reporter by itself when it detects an AI coding agent (flag 1), so measuring inside the agent hides the "before".
JavaScript and Python: flags 1 to 4
1. Vitest: --reporter=minimal
Vitest 4.1 added a reporter "to reduce ai agent token usage". The current reporters guide calls it minimal, with agent as an alias. It prints only failed tests and their error messages, and suppresses console logs from passing tests and the summary section. When no reporter is configured and Vitest detects an AI coding agent, it uses this one automatically.
Pass it yourself when your config sets reporters, or when detection doesn't recognise your agent:
npx vitest run --reporter=minimal
If you want a progress signal, --reporter=dot prints "a single dot for each completed test", with details only for failures plus the summary. And the CLI's --silent=passed-only keeps console output from failing tests only.
2. Jest: --silent
If your code logs while tests run, every one of those lines lands in the agent's context. Jest's CLI has one flag for that: --silent will "Prevent tests from printing messages through the console."
npx jest --silent
Jest has a default that bites agents: verbose is false, "or true if there is only one test file to run". The quick rerun of one file after a fix, the run agents do all the time, is the run that reports each test individually.
3. pytest: -q
-q is pytest's "quiet - less verbose - mode". The flag to look for in your scripts and addopts is the opposite one: -v gives each test its own line, and -vv stops truncating assertion text entirely.
4. pytest: --tb=short
The default, --tb=auto, prints "'long' tracebacks for the first and last entry". --tb=short is the "shorter traceback format", and --tb=line is "only one line per failure". Short keeps the file, the line and the failing assertion, which is usually what the agent needs to act. Line is often too little.
pytest -q --tb=short
One more from the same page: if addopts turns on --showlocals, every traceback carries the local variables. --no-showlocals hides them for that run.
Go, Rust, PHP and Ruby: flags 5 to 8
5. Go: take -v off
go test ./... is already quiet: a passing package prints only the final ok summary line. The -v flag logs all tests as they run, and will "Also print all text from Log and Logf calls even if the test succeeds." If a Makefile target or the agent adds -v, every passing test and every t.Log comes back.
If you want a friendlier view, gotestsum wraps go test -json. Its default pkgname format prints one line per package, and dots prints one character per test. Flags for go test go after --:
go test ./...
gotestsum --format dots -- ./...
pkgname grows with packages and dots grows with tests, so measure both on your repo.
6. Cargo: -q, on both sides of --
cargo test -q -- --quiet
The first -q is Cargo's own and hides its log messages. Everything after -- goes to the test harness. There, libtest's --quiet "Displays one character per test instead of one line per test" and is an alias for --format=terse. The default, pretty, is one line per test.
7. PHPUnit: --no-progress
PHPUnit's TextUI docs list --no-progress: "Disable output of test execution progress." Failures and the result summary still print.
vendor/bin/phpunit --no-progress
Two neighbours to avoid in an agent loop: --testdox will "Replace default result output with TestDox format", which reports each test by name, and --no-output will "Disable all output", which takes the failure details with it.
8. RSpec: --format progress
RSpec defaults to the progress formatter: . for a pass, F for a failure, * for pending. The documentation formatter prints the text of every describe and it instead. Check the project's .rspec file: if it has --format documentation, change it to --format progress or delete the line.
Flag 9: stop at the first failure
Every runner above has a version of the same switch. When the agent is fixing one thing, the failures after the first are output it reads before it can act on any of them.
| Runner | Stop early | What the docs say |
|---|---|---|
| Vitest | --bail=1 |
stop after N failed tests |
| Jest | --bail |
N failing suites, default 1 |
| pytest | -x |
stop after first failure |
| go test | -failfast |
no new tests after one fails |
| PHPUnit | --stop-on-failure |
stop on first failure |
| RSpec | --fail-fast |
stop on first failed test |
Sources: Vitest CLI, Jest CLI, pytest, go test, PHPUnit, RSpec.
Cargo stops early at a coarser level: it exits after the first test executable fails, but libtest runs every test inside that executable. Libtest's own --fail-fast exists, and it needs -Z unstable-options.
Put together, the fast loop looks like this:
npx vitest run --reporter=minimal --bail=1
npx jest --silent --bail
pytest -q --tb=short -x
go test -failfast ./...
cargo test -q -- --quiet
vendor/bin/phpunit --no-progress --stop-on-failure
bundle exec rspec --format progress --fail-fast
Stopping early is for the loop, not the finish line. Before the agent says it's done, it should run the whole suite, quietly, and read every failure.
Pin the commands, then watch the meter
Flags help only if the agent uses them. Put the commands in the file your agent reads when a session starts. Claude Code loads CLAUDE.md "into context at session start". AGENTS.md is the open format that Codex, Gemini CLI, Cursor, Aider and others read.
## Tests
- While fixing: `pytest -q --tb=short -x tests/test_orders.py`
- Before saying done: `pytest -q --tb=short`
Claude Code's cost guide lists two heavier options too: a PreToolUse hook that rewrites test commands so only failures come back, and running tests in a subagent so "the verbose output stays in the subagent's context while only a summary returns".
Then you want to see the effect. I build Hermes IDE, so weigh this part with that in mind. Hermes runs Claude Code, Codex and other CLI agents in real terminals, and version 2.3.4 added a context meter that shows how full the conversation is and offers Compact when it gets close to full.
Hermes 2.3.4: Claude Code's /context breakdown after pressing the Context chip, per-session context gauges in the sidebar, and the token total and estimated cost in the status strip.
Run the suite once with default flags and once with the quiet ones, and watch the meter between turns. Compact is there when you need it, in the Commands list above each session.
The action bar above a Claude Code session, with the Commands list open: pinned Context, Model, Plan and Review, then built-in commands such as Compact, Usage and Diff.
Compacting is the fix after the fact, and not a free one: Claude Code's docs note that /compact "reads the conversation it summarizes, so compacting a large context is itself a large request." The flags above are the cheaper fix.
Hermes is free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it here, and the code is on GitHub.
If this was useful
Counting bytes with wc -c is the crude version of a question that gets harder as a team grows: what did the model actually read, and what did that cost? The LLM Observability Pocket Guide compares the tracing and eval tools that answer it across a whole system, not one terminal.
Top comments (1)
Everyone optimizes for speed. Almost nobody optimizes for being able to undo it.