DEV Community

ULNIT
ULNIT

Posted on

I Kept Every Prompt That Broke My Agent in a File Called bad-prompts.md. Entry 60 Changed How I Write Prompts.

Entry 1 is eleven words long:

Agent refunded a customer twice. Added "be careful with refunds." — Aug 4

I started the file because I kept fixing the same class of problem and then forgetting I'd fixed it. A prompt change would hold for two weeks, the behaviour would drift back, and I'd spend an afternoon rediscovering a fix I'd already written down in a git history I never read.

So I made a rule: every time the agent did something wrong in production, it got one entry — the date, one sentence about what happened, and the diff I applied. No essay. Thirty seconds. Committed with the fix, in a file called bad-prompts.md.

Sixty entries later, that file is the most useful document I own, and it isn't because of the fixes. It's because I finally tagged them.

The tag that embarrassed me

Of 60 entries, 41 traced back to four mistakes in how I wrote the instructions. Six were infrastructure problems wearing a prompt costume. Two were the model genuinely not being able to do the task. The remaining eleven were me changing two things at once and not knowing which one worked.

Not one of those 41 was "the model is dumb." That's the part I sat with for a while.

Pattern 1: adjectives where I needed a test (14 entries)

"Be careful with refunds" is not an instruction. It's a mood.

Here's the actual before and after from entry 23:

Before:
Be careful and accurate when handling refunds. Only refund when appropriate.

After:
Before issuing a refund, read order.created_at.
- Older than 30 days: do not refund. Call escalate_to_human(reason="outside_window").
- 30 days or newer AND status == "paid": refund, then reply with the refund id.
- Anything else: escalate_to_human(reason="unclear_eligibility").
Enter fullscreen mode Exit fullscreen mode

The second version is longer and it's the only one that ever worked. "Appropriate" was a word I understood and the agent had to guess at. A testable rule names the input, the comparison, and the action for each branch. If I can't write it as a comparison, I don't understand my own policy yet — I just have a feeling about it.

Pattern 2: prohibitions with no substitute (11 entries)

"Never be salesy." "Don't hallucinate." "Avoid overpromising." "Don't mention competitors."

Every one of those leaves the space of acceptable behaviour undefined, so the model picks the nearest fluent thing. I told it not to mention competitor pricing and it started writing sentences that implied competitor pricing was worse — which is arguably worse, because now it's a claim I have to defend.

The fix is mechanical: every "don't" gets an "instead."

Before: Never mention competitor pricing.
After:  If asked about a competitor's price, reply with our price and this line only:
        "I can only speak to our own pricing." Then stop. Do not compare.
Enter fullscreen mode Exit fullscreen mode

Eleven entries, and I'd guess eight of them would have been prevented by that one habit.

Pattern 3: two rules that are both true (9 entries)

My refund policy said 30 days. My retention rule said "offer a discount to anyone who tries to cancel." A customer on day 45 cancelled, and the agent — following both rules correctly — offered a retention discount and, inside that message, promised a refund that policy forbade.

Neither rule was wrong. They collided on a Tuesday.

Two rules that can both apply to the same message is not an edge case, it's a schedule. So the prompt now has a section I didn't have for the first fifty entries:

Precedence:
1. Safety and escalation rules override everything below.
2. Refund window overrides retention offers.
3. If two rules still conflict on the same request, escalate_to_human and stop.
Enter fullscreen mode Exit fullscreen mode

Nine entries were the same mistake in nine costumes. The precedence block is four lines and it closed all of them.

Pattern 4: examples that taught the wrong shape (8 entries)

I had five "good reply" examples. Four were two sentences. One was a paragraph with a bulleted list. The agent wrote paragraph-with-bullets almost every time.

Models copy structure far more reliably than they follow intent. My examples disagreed about shape and the longest one won, every time, including on replies where a list made no sense. If you have one outlier example, you have one example and four pieces of noise.

What I do now: examples matched in length, one per output shape, and when I add a new shape I add two so neither is an outlier.

The two failures inside the failure log

The first is mine, and it's embarrassing in a small way. For three weeks I ran a section in the system prompt called "Voice" — six lines about warmth, sounding human, not sounding like a bot. It was the part of the prompt I was proudest of. It read like writing.

It shows up in exactly two entries, and both are cases where the agent softened a message it should have delivered plainly. One was a delivery delay. "Warm" turned your order is late into three sentences that never used the word late, and the customer replied asking whether it had shipped. I kept the section for three weeks after I knew, because deleting it felt like admitting the writing was decoration.

Two entries out of sixty is a 3% failure rate for a section whose absence I never noticed. Removing it cost nothing.

The second one changed my process more than any single fix. Entry 44 said the agent ignored a new escalation rule. I spent two hours rewriting that rule, tested it, deployed it, and it worked. Entry 45, two days later: same failure.

The rule was fine both times. The prompt was loaded from a config file that was only read at process start, and my "deploy" restarted a worker that didn't own the prompt. The rewrite didn't fix anything — the restart that came along with it did. I credited the wording. For two weeks afterwards I believed a specific phrasing was load-bearing, and I wrote other prompts to imitate it.

That's the failure mode a fix-log is supposed to catch and mine didn't, because I was recording what I changed and not how I knew it worked. Every entry now has a line for that, and if I can't write it, I'm not allowed to close the entry.

What the file looks like now

## 2026-10-14 — refunded day-45 order
What happened:   promised refund outside window inside retention offer
Suspected cause: no precedence between retention + refund rules
Change:          added Precedence block (diff 4c9f1a)
How I know:      replay of ticket 8812 → escalates, no refund tool call
Tag:             collision
Enter fullscreen mode Exit fullscreen mode

And five questions I run before any prompt deploy:

  1. Does every rule name an input and an action?
  2. Does every "don't" have an "instead"?
  3. Can two rules both apply to one message? Which wins?
  4. Are my examples the same shape and roughly the same length?
  5. What specific check proves this change worked?

Sixty entries in, the honest summary is that the model was rarely the problem. I was writing wishes and calling them instructions, and bad-prompts.md is just the receipts — one line per time I found out the difference.


All 100 prompts are in The Agent Prompt Vault — $3, lifetime updates. Steal the ones that fit your workflow.

If you keep a file like this, I'd genuinely like to know which pattern dominates yours. Mine was adjectives, and I didn't see it until I counted.

Top comments (0)