I Started Version-Controlling My AI Agent's Prompts Like Production Code. The First "Improved" Prompt Broke Three Days of Silent Work.

I started version-controlling my AI agent's prompts like production code. The first "improved" prompt broke three days of silent work.

For months my prompts lived wherever they happened to be written: inline strings in Python scripts, a notes app, one in a README I'd forgotten about, and at least two in my shell history. It worked — right up until it didn't, and when it didn't, I had no way to answer the only question that mattered: what changed?

This is the system I built after that, the failure that forced it, and the parts I'd skip if I were starting over.

The failure: a "better" prompt that wasn't

I run a small support-triage agent on a Raspberry Pi. It reads incoming messages, classifies them, and drafts replies for me to approve. One evening I pasted in a prompt I'd found online — longer, more structured, full of role-setting language. It looked professional. I swapped it in, watched a couple of outputs, thought "seems fine," and went to bed.

Three days later I noticed the drafts had a strange quality: they were polite, thorough, and completely useless for urgent issues. The agent had stopped flagging anything as high-priority. Every ticket got the same calm, apologetic, "we'll look into this shortly" treatment. One customer with a genuinely broken payment waited two days for a human to notice.

The new prompt had a line in it something like "remain calm and reassuring at all times; avoid alarming the customer." The model had interpreted that as "never say anything is urgent." A single clause, buried in paragraph six of a prompt I'd copied without diffing against the old one.

Here's what actually stung: I couldn't tell you what the old prompt said. It was gone — overwritten in the script. No history, no diff, no record of what behavior I'd lost. I spent an evening reconstructing it from memory and a stale backup, and the reconstructed version was measurably worse at classification than the original had been. I never fully got it back.

The system: prompts are code, treat them like code

After that I moved every prompt into a git repo with three rules.

Rule 1: One prompt per file, named for the job.

prompts/
  triage-classify.md
  triage-draft-reply.md
  recon-summarize-findings.md
  digest-morning-brief.md

Each file is pure Markdown with a small YAML header:

---
name: triage-draft-reply
model: gpt-4o-mini
version: 7
last_changed: 2026-08-30
owner_task: reply drafting only — never classifies priority
---

That owner_task line matters more than it looks. Half my early prompt bugs came from one prompt quietly doing two jobs. When the drafting prompt started classifying priority as a side effect, nobody had decided that — it just happened.

Rule 2: Never edit a prompt without a fixture run.

For each prompt I keep 5–10 canned inputs with expected properties of the output. Not unit tests with exact string matches — that's brittle nonsense with LLMs. Instead, cheap property checks:

def check_urgent_flag(output):
    # the angry-payment-failed fixture MUST be classified urgent
    assert "urgent" in output.lower() or "high" in output.lower()

def check_low_priority(output):
    assert "urgent" not in output.lower() and "high" not in output.lower()

FIXTURES = [
    ("fixtures/angry_payment.txt", check_urgent_flag),
    ("fixtures/spam_nigerian_prince.txt", check_low_priority),
    ("fixtures/normal_question.txt", check_no_apology_storm),
]

The third fixture exists specifically because of the failure above. I added an assertion that catches apology-storm output — five consecutive hedging phrases — and it has caught two regressions since.

The whole suite costs me about four cents to run. I run it before every prompt change, in a pre-commit hook, so I literally cannot ship a prompt edit without the fixtures passing.

Rule 3: The commit message says what behavior changed, not what text changed.

git diff tells me the text. The commit message tells me why:

v6->v7: stop the model from apologizing before answering.
Added "no preamble, first sentence = the answer".
Fixture: normal_question now passes apology check.

Three months later, when something feels off, git log prompts/triage-draft-reply.md reads like a behavioral changelog. When I broke things in August, I had nothing.

What the system caught that I didn't expect

The fixtures aren't just regression insurance — they surfaced problems that had been quietly costing me for weeks.

• Model drift. My provider updated a model version and my classification accuracy on the fixture set dropped from 9/10 to 6/10 overnight. Because I had the fixtures, I knew it was the model, not my prompt, and I pinned the old version within an hour. Without them I'd have spent a day "improving" a prompt that was fine.
• A prompt that had quietly doubled in size. Every time I patched an edge case, I added a sentence. Version 4 of the drafting prompt was 700 words of accumulated scar tissue. Reading the diff history made the bloat obvious, and I rewrote it down to 200 words — which fixture-tested better than the long version.
• Two prompts that disagreed with each other. My triage prompt told the agent to be concise; my drafting prompt told it to be thorough. The agent was resolving that contradiction differently on different days. Neither prompt was wrong; the pair was. You only see that when all prompts live in one repo, side by side.

What I'd do differently / what I got wrong

Honest part, because this system isn't free.

I over-engineered the first version. My initial setup had a database of prompt versions, a diff UI, and "rollbacks" as first-class objects. It took a weekend to build and I abandoned it in two weeks. Git already does all of it. The version that survived is a folder of Markdown files and a 60-line test script. If you take one thing from this article: the boring version is the version that lasts.

Fixtures rot. Three of my original ten fixtures encoded assumptions that stopped being true (an old pricing tier, a retired feature). Stale fixtures that fail for the wrong reason train you to ignore failures — the exact failure mode the system exists to prevent. I now review the fixture set monthly, same as I'd review tests.

It doesn't fix prompt quality. Version control tells you what changed and whether behavior regressed. It does not make prompts good. For a long time I mistook "I can see the history of this bad prompt" for "this prompt is good." The improvements came from actually studying which phrasings worked, one experiment at a time.

It's overkill below ~5 prompts. If you have one agent with one prompt, a .txt file and discipline is enough. The system pays for itself when prompts multiply and you can no longer hold them all in your head — which, for me, happened around prompt number seven.

The takeaway

The failure that started this wasn't a model failure. It was an ops failure: I changed production behavior with no diff, no history, and no test. Every developer already knows why that's reckless with code. Prompts are the same thing — executable instructions that shape real behavior — they just don't feel like code because they're written in English.

Put them in git. Write five fixtures each. Make the commit message about behavior. That's the whole system, and it takes an afternoon.

Collecting and refining prompts is the part that actually takes time — I keep every prompt that survived fixture-testing in one place, organized by job, along with the notes on why each phrasing works. All 100 prompts are in The Agent Prompt Vault — $3, lifetime updates. Steal the ones that fit your workflow.