I use AI agents to code. Four checks I built to trust them.

I am a solo startup founder. In order to launch TribeROI in a reasonable amount of time, I use AI agents that read the codebase, write code, run tests, and open pull requests. My job is to review and merge the code.

What makes this process work is not the agents, but a set of checks I built so I could trust what they produce.

Everyone worries that AI writes bad code. Bad code was never my problem. The problem is that agents produce more than one person can properly review, and this is when defects surface. A few weeks ago, I hit twelve big defects in a single day, which all passed my tests. I only found them by reading the live system.

The failures fell into three patterns. First, artifacts that exist but nothing runs them. Second, systems that could not find an answer and substituted a plausible default. Third, work that was rebuilt without checking for already existing code that solved the same issue.

The second pattern burned me twice. When the agent cannot determine something, it will write a fallback. While this seems reasonable to the agent, the result is a duplicate resource in production.

Here are the four rules I have now built to give me higher confidence in the agent's code.

1. A rule is a test, or it doesn't exist

Every safety rule I keep has to name the specific automated check that fails when someone breaks it. If I can't name one, it is not a guarantee and it goes on a list titled "things I have to remember to check."

When I held my own rules to that standard, eight of the eleven items on my code review checklist enforced nothing whatsoever. They were boxes you tick that change no outcome. Three of the eight rules I'd labeled "hard requirements" were in the same shape, and a batch of alerts had done nothing since the day I wrote them. I'd been reading that checklist and thinking everything was fine.

An unenforced rule is worse than no rule at all, because you lean on it.

2. Never let "unknown" look like "fine"

This is the highest-value rule of the four and the cheapest to act on. Go find every place your code turns "I could not determine this" into a value, and make it stop instead.

It bites in application code. Mine met a permission role it didn't recognize, fell back to least privilege because that felt like the safe choice, and locked eight paying customers out of the admin screens in their own accounts. Nothing threw. The fallback was doing exactly what I'd told it to do.

The expensive version was in the ops scripts that create-or-update cloud resources:

CHANNEL=$(gcloud beta monitoring channels list --format="value(name)" | head -n1)

The pipe hands the exit status to head, which happily succeeds, so set -e never fires. A transient auth blip becomes an empty string, the empty string reads as "does not exist," and the script cheerfully creates a second copy of a resource that was already there. That was 7 July. I patched that one script, told myself I was done, and on 21 July the same shape turned up somewhere else and orphaned a notification channel that eleven alert policies were pointing at.

The second time taught me more than the first. A fact that every code path depends on belongs in one place they all call:

must_read() {
  if ! MR_OUT=$("$@" 2>"$MR_ERR"); then
    echo "!! READ FAILED, refusing to continue: $*" >&2
    echo "   A failed lookup is not proof the resource is absent." >&2
    exit 1
  fi
  printf '%s\n' "$MR_OUT" | head -n1
}

Written as VAR=$(must_read ...), it propagates the failure, the assignment fails, and set -e ends the run. Two outcomes survive: the real value, or a halt.

3. Test the wiring, not just the artifact

A file that exists reads like proof that it works, and almost nothing checks whether anything runs it. I shipped an alert policy as JSON, declared it in the manifest, and the apply script never called apply_policy on it. The dry-run hid the whole thing, because the validator scans the whole directory, while the applier names each policy by hand, one line per policy. One of them saw the file and the other didn't.

So there's now a test that greps the apply script for its own calls and checks both directions. Every declared policy gets applied, and every call points at something declared. It's ugly and I'm not proud of it, and it's the only thing that proves the file is reached.

The other assertion is the one worth copying:

def test_the_policy_pair_is_not_vacuous() -> None:
    assert ALERT_POLICIES, "no declared policies — the guard would be vacuous"
    assert _applied_policy_patterns(), "no apply_policy calls — same"

A guard that iterates an empty collection passes by looking at nothing. Rename the constant it reads and both sides go empty, so the check stays green while proving absolutely nothing. That's the same failure one level up. If you write guards, assume you already have one of these.

4. An agent's authority is a mechanism, not an instruction

I run a mode where an agent works unattended for hours, and it gets Bash(gcloud:*) outright, because a permission prompt nobody is awake to answer is just a hang with extra steps. That's only safe because the real boundary sits somewhere else: a hook where gcloud is default-deny, letting through reads and a named set of additive commands, and refusing everything else.

A config file can't do this job. In Claude Code, ask beats allow regardless of how specific the allow rule is, so a narrow allow sitting next to a broad ask(gcloud *) is dead code. Prefix matching is fragile too, and gcloud run with two spaces doesn't match Bash(gcloud run *). As a way to permit things, that's a hole. Inside a default-deny hook it's harmless, because anything unrecognized gets refused anyway.

The hook's catch block denies as well. Every other guard I've written fails open, which is the right call almost everywhere. This one is the only thing standing between a broad grant and production, so a crash has to refuse rather than guess.

What I'd build first

Take the second rule. Search your codebase for the fallbacks that fire when something couldn't be determined, and make each one stop instead. It is quick, needs no new tooling, and it is a failure that crops up surprisingly often.

The insight across all these rules is that systems confidently report success when it's clearly not the case. In AI development, agents make this an even higher priority, because they produce code faster than you can read it, so these checks have to scale.

I'd like to hear what other people are building to harden their agentic development workflows, especially if you've hit a class of failure I haven't. Let me know in the comments.