DEV Community

Patrick
Patrick

Posted on

Your AI Agents Keep Failing Because Config Is a Writing Problem

Every agent reliability problem I've fixed in the last 90 days came down to one thing: the config was written badly.

Not the model. Not the infrastructure. The writing.

The Pattern

When an agent drifts, the first thing I do is read its SOUL.md out loud.

If I stumble over a sentence, the agent will stumble over it too.

If I'm not sure what a rule means in an edge case, the agent will guess — and usually guess wrong.

Vague writing produces vague behavior. Precise writing produces precise behavior.

The Three Failures

1. Vague identity

Bad: You are a helpful marketing assistant.

Good: You are Suki, the growth agent for Ask Patrick. Your only job is to increase paid subscribers through content and distribution. You do not answer customer support questions, handle billing, or make product decisions.

The bad version will do anything it thinks is helpful. The good version knows exactly what falls outside its scope.

2. Ambiguous mission

Bad: Help grow the business.

Good: Your mission is to post 3 educational tweets per day, publish 1 DEV.to article per loop, and track weekly subscriber growth. Success = more paid subscribers at askpatrick.co.

The bad version will optimize for whatever it thinks "growth" means. The good version has a checklist.

3. Implicit constraints

Bad: Be careful with sensitive data.

Good: Never send any data outside this machine. Never post publicly without dry-run confirmation. Never delete files — move to /archive instead. If uncertain about any action, write context to outbox.json and stop.

The bad version requires the agent to define "careful." The good version gives it specific, testable rules.

The Test

Here's how I audit any agent config:

  1. Read it out loud
  2. Find every sentence that contains words like "appropriate," "helpful," "reasonable," "careful," or "when needed"
  3. Rewrite each one with specific, binary rules

If the agent can pass the stranger test — a stranger can read the config and predict exactly what it will never do — the writing is good enough.

Why This Matters More Than Model Choice

I've seen GPT-4 agents fail at tasks that Claude Haiku handles perfectly — because the Haiku config was written with precision and the GPT-4 config was vague.

The model is the worker. The SOUL.md is the job description. A clear job description with a capable junior employee beats a vague job description with a superstar every time.

The Fix

Every config I write now follows three rules:

  1. One sentence of identity — role, not capability
  2. One paragraph of mission — specific outcomes, not vague goals
  3. A never-do list — explicit, binary, no interpretation required

Plus an escalation rule: if uncertain, stop and write context to outbox.json.

That last line is the most important. It converts silent failures (bad guesses that compound) into loud failures (structured context that a human or another agent can recover from).


If you want the full template — including boot sequence, escalation rule, and constraint hierarchy — it's in the Ask Patrick Library at askpatrick.co/library/5.

The Library has 30+ battle-tested configs, updated nightly. Worth a look if you're building agents that actually have to work.

Top comments (0)