I built my whole product with an AI writing the code, and the scariest failure mode isn't bugs — it's a test suite that passes for the wrong reason. Here are all four steps, and the real moment each one saved me.
Pre-register the pass/fail criteria, frozen in git, before I see any result. If I define "success" after seeing the output, I'll unconsciously pick the definition the output already satisfies. My first project died exactly here: a signal that "passed" was secretly measuring trace length, not failure. Writing the bar down first is the cheapest insurance there is.
Commit the criteria and run the baseline tests. A known-good starting line the agent can't silently move. If a test goes green, I want to know it went green today, not that it was already green.
Ask the agent for a PLAN, not code. Most "the AI wrecked my codebase" stories begin with approving a 400-line diff nobody read. A plan is reviewable in two minutes. This is also where I catch the dangerous "fixes" — once, my agent's fix made the numbers go green by quietly deleting the hard case that exposed a real weakness. I caught it in the plan, not in production.
Review the plan, push back, then approve — manually. The agent never writes code I haven't already read a plan for. Auto-approve is how you wake up to a system that's confidently wrong.
It feels slow. It's the opposite: an hour here saves a week of debugging something that looked correct the whole time. Code's public: github.com/JEONSEWON/Clew-by-Custos
BuildInPublic #AIAgents #ClaudeCode #DevTools
Top comments (0)