Recent DEV discussions have been circling a real discomfort: when AI starts writing more of the commit history, the human's job becomes review. But review of what? Run the tests. If the tests were written by the same model that wrote the code, you're reviewing a closed loop.
A pairing session between a mid-level developer and a senior engineer set out to break that loop using a surprising stack: a free AI model quota and a free remote server. The project under test was a small internal CLI tool with forty unit tests and a few integration checks. The developer suggested using MonkeyCode's open-source platform, which at the time of writing advertises a free tier of around 10 million tokens and a free server instance for lightweight jobs. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The senior did not object. They only asked three questions.
The Setup Questions
What is the server actually for? The developer wanted a place where PRs could run tests without hogging a laptop. The senior said that was fine, but pushed on state: a free instance is ephemeral. No artifacts survive a restart. That matters if you plan to cache dependencies.
What happens when the token budget is spent? The developer admitted they hadn't monitored that before. The senior noted that a sudden quota cutoff can make a red build look like a test failure when it's really a 429.
How do we know the AI-generated tests are not just agreeing with the code? That was the sharpest question. A model that writes both sides can produce tests that assert the current behavior, even if that behavior is wrong.
First Attempt: Let the Model Write the Full Test Suite
The developer asked MonkeyCode's free model to generate a suite for the CLI tool. The result looked plausible: twenty files, mocked network calls, clean assertions. Then they ran it on the free server.
The suite passed. Too easily. The senior picked one test and changed the CLI's output to a known broken value. The test still passed because it only checked that the process exited with code 0, not the actual output.
That was the first dead end. The second appeared when the full suite hit the free server's memory ceiling during a dependency install. The model had generated tests that import heavy libraries, and the ephemeral instance refused to build.
The Pivot: A Quota-Aware Wrapper
The pair stopped asking for generated tests and started building a harness around what they had. The decision was to treat the free server as a smoke-test runner, not a full CI replacement. The rule: if token usage is above 20 percent of the budget, run everything; below that, run only the fast unit tests.
That rule became a small Bash script, deliberately dependency-free except for pytest:
#!/usr/bin/env bash
# quota-aware-test-runner.sh
# Reads budget and used tokens from environment variables.
budget="${MONKEYCODE_TOKEN_BUDGET:-0}"
used="${MONKEYCODE_TOKEN_USED:-0}"
if [ "$budget" -eq 0 ]; then
echo "No token budget set. Exiting."
exit 1
fi
left=$((budget - used))
percent_left=$((left * 100 / budget))
echo "Tokens remaining: ${percent_left}%"
if [ "$percent_left" -gt 20 ]; then
echo "Running full test suite."
pytest tests -q
else
echo "Running smoke tests only."
pytest tests/smoke -q
fi
The script reads two environment variables, which is enough for any CI system or cron job. The pair wired it into the free server with a simple cron entry that runs after each push:
* * * * * MONKEYCODE_TOKEN_BUDGET=10000000 MONKEYCODE_TOKEN_USED=... /home/user/quota-aware-test-runner.sh >> /tmp/test-runner.log 2>&1
No third-party services. No persistence layer. It works until the server dies, and when it dies, the next run recreates it.
What the Pairing Decided
The senior ended the session with a decision matrix. It became the reference for future AI-assisted test infrastructure:
| Scenario | Use the free server? | Why |
|---|---|---|
| PR smoke tests on small repos | Yes | Short-lived, low memory, easy to recreate |
| Full regression with thousands of tests | No | Ephemeral instances lack stable caching and often time out |
| AI-generated unit test suggestion | Only with human review | Model-written tests can encode current bugs as truth |
| Long-running integration checks | No | Free instances restart and lose state mid-run |
That matrix kept the pair from overreaching. They left with a working smoke-test pipeline, a quota guard, and a clear rule: never trust a green build from an AI-generated test suite unless a human has mutated the code to verify the test actually fails.
Limitations and Who Should Not Use This
This approach has real limits. The free server is not durable; any run that takes longer than the instance lifetime will be lost. Token quotas can change without notice, so the environment variables must be updated from a reliable source. The script itself only skips tests based on a fixed percentage; it does not detect declining model quality or flaky network retries.
Do not copy this pattern if your team needs compliance traces or production-grade SLAs. If your test suite includes sensitive data, an ephemeral instance is a liability. And if you routinely run hundreds of tests that take more than ten minutes, pay for a stable runner instead of stacking free credits.
The Last Takeaway
The pairing's final insight was not about tokens or servers. It was about the review loop. If AI writes the code and AI writes the verification, you are only measuring how consistently the model can fool itself. A cheap, quota-aware harness gives you a real signal, but only when a human occasionally breaks the code on purpose to see if the tests notice.
MonkeyCode's free tier is a reasonable place to experiment with that workflow. Take the script, adapt it to your environment, and run one mutation before you trust the green checkmark.
Top comments (0)