Five Ways My AI Agents Went Wrong (and What I Changed)

Bryan Gelfius — Wed, 07 Oct 2026 18:19:05 +0000

I run a small software company by myself. Claude Code agents play the engineers, QA, marketing and so on. It works better than I expected, but almost every problem I've had came from the same place: I believed something I hadn't checked. An agent's "done," a file the agents read, a doc that looked current.

Here are five times that bit me, and what I do now. At the end there's a small free toolkit with the guardrails, if you'd rather not rebuild them.

1. A persona file said something false, and every agent repeated it

My agents each have a role file. One of them said, flat out, that a certain ad platform bans a certain kind of ad. I never tested that. Nobody did. I traced it back through git log -p --follow and it showed up in the file's first commit as a guess.

After that it got cited in review after review. Each review quoted the one before it, not the platform's policy. It got as far as influencing whether I'd rename something. When I finally read the actual policy, the real rule was much narrower than what my file claimed.

Now anything in a role file about the outside world (platform rules, vendor limits, marketplace requirements) counts as a guess until someone points me to the source. And when I find one that's wrong I fix the role file itself, not just the one review, or it comes back next session.

2. A background agent made up a confirmation

I asked an agent to draft listing copy for one product and told it to leave everything else alone. It said it was done and only mentioned the one file.

I ran git diff anyway. It had edited four other files, and in one of them it had written that I'd confirmed a listing was live. I never said that. It was invented, and it was sitting in a checklist that later sessions would have taken as fact.

So now, every time an agent says it's finished:

git status
git diff --stat

That's the whole repo, not just the files I assigned. Out-of-scope edits get reverted, not cherry-picked. And if agent output says "the user confirmed X," I don't believe it unless X is actually in the conversation.

3. A script I forgot about wiped out my hand-written notes

An agent was refreshing some status data and ran one of my own scripts from an integrations/ folder. The script regenerated the whole output file from scratch. That file also had months of my own analysis in it. One run replaced all of it with a bare machine-generated stub, a 202-line change, and it brought back some stale data I'd specifically marked as unreliable.

I only caught it because git diff --stat showed a big change in a file the task had nothing to do with. I reverted before committing.

The script is append-only now. It adds a new dated block under a known heading and never deletes or replaces a line. If it can't find the heading, it stops instead of falling back to overwriting. I tested it with a dry run and compared it line by line against the real file. The bigger lesson: before you run any script, or let an agent run it, find out which files it writes and whether any of them hold something a script can't recreate.

4. Two sessions, one working tree

I usually have a few Claude Code sessions open on the same clone. They share one index and one working directory. One session ran git add -A and committed, and it swept up a multi-file edit another session had made but not committed yet. The content was right. It just landed under an unrelated commit message, mixed in with other work.

Another time a session stashed someone else's uncommitted edits and switched branches. Nothing was lost, but the files I was editing vanished out from under me.

What I do now:

Commit your own edits in the same turn you make them.
Stage explicit paths, git add path/to/file, never git add -A in a shared clone.
For anything longer than a quick turn, use a worktree: git worktree add ../repo-topic some-branch.

5. My status docs were wrong in both directions

I keep a priorities doc and a design doc. During one push, they said a feature had shipped while gh pr view showed the PR still open. A few hours later it really had merged, so things had just moved. In the same session I thought three tickets were done because the priorities doc said so. I read the code and found no trace of any of them.

So the doc was ahead of reality in one place and behind in another. Neither the doc nor my memory of it counted as evidence.

Now "done" means three things I check myself: a merged PR (gh pr view), merged code that isn't a stub, and a deploy that actually happened.

What these have in common

None of this was the model being dumb. Every time, something got treated as verified when it was only plausible: a persona file, an agent's own summary, a script I'd forgotten, a status doc. The fix is always the same. Check the thing that actually knows the truth (the diff, the PR, the primary source, the real file) before acting.

I apply the same idea to releases. Nothing ships without two separate approvals, and neither one can approve for the other. The last gate in my release playbook looks like this:

QA APPROVED       [date] [who]
PRODUCT APPROVED  [date] [who]

EM GO / NO-GO: GO

If either one is REJECTED, it goes back to the engineers and that step starts over. Anything P0 (data loss, auth bypass, billing error) blocks the release no matter what the schedule says.

The free toolkit

I pulled the role definitions and release gate out of my own setup into a small MIT-licensed repo: an orchestrator, an engineering manager that never writes code, backend and frontend engineers, QA, a security reviewer, an architect, and the release playbook above. There's also a LEARNINGS.md with short versions of what's in this post.

To be clear, it's not a new framework. If you want a full process with lots of roles, BMAD-METHOD is much bigger and probably a better fit. This is for people who want a few opinionated roles and hard release gates. It's v0.1, it came out of my own setup, and I haven't run it end to end anywhere else, so expect rough edges. The role files are prompts, not guarantees. They don't replace reading what the agents produce.

Repo: ageebgee-solutions/claude-agent-sdlc-toolkit. Issues and PRs are welcome, especially if you've been burned in a way I haven't listed.

What have your agents gotten wrong?

I'm Bryan, founder of AgeeBgee Solutions. We build Azure Marketplace apps.

DEV Community: Bryan Gelfius