DEV Community

Cover image for AI slop isn't a model problem. It's what you get when nothing checks the output before it ships.
Alex Mantello
Alex Mantello

Posted on Edited on

AI slop isn't a model problem. It's what you get when nothing checks the output before it ships.

Claude is good enough now that most slop isn't the model's fault. It's the workflow: one prompt, no facts, no checks, straight to publish.

I've spent the last few months building the boring part that comes after generation, around Claude Code. These are the five things that actually cut slop for us.

1. Facts live in a file, not in the prompt.
Every workflow reads one facts file: what we sell, prices, what we don't do. The rule in the system prompt is one line: if it isn't in the file, say you don't know. Most "the AI invented a feature" problems disappeared that day.

2. A linter for prose.
Before a human sees anything, a deterministic checker rejects the draft and tells the agent why:

  • emoji, unfilled placeholders like [CLIENT NAME]
  • "delve", "seamless", "game-changer", "unlock" and friends
  • wrong length for the platform, wrong language, missing link The agent fixes it and resubmits. No model call needed for any of this, and it catches more than you'd expect.

3. A second look by a model, then a person.
What passes the linter gets a quick model review against the brand rules, then lands in an approval queue. Nothing leaves (post, email, client file) until someone clicks Approve. Rejections need a reason, and the agent sees it next time.

4. Evals per workflow.
Every workflow ships with test cases: an input, plus what a good output must and must not contain. A model grades them before a workflow goes live. Our internal set is 345 cases. When we change a prompt, we rerun them instead of eyeballing two examples.

5. Small toolboxes.
Agents that run unattended get an allowlist, not a denylist:

"allow": ["Bash(ourcli:*)", "Read(./work/**)", "Edit(./work/**)"],
"deny":  ["Bash(sudo:*)", "Bash(curl:*)", "Bash(npm install:*)", "Bash(pip install:*)"]
Enter fullscreen mode Exit fullscreen mode

with --permission-mode dontAsk. Less clever, much easier to trust.

None of this is fancy. It's the same thing you'd do for a junior hire: give them the facts, a style guide, a reviewer and limited access.

Disclosure: this is how Zen Staff works, the app I build. It started as my fork of T3 Code in July. It runs the official Claude Code (or Codex) on your own account, it's free and open source (MIT). Windows and Linux for now, no Mac app yet.

Curious what others do. What's the one check that removed the most slop from your agents' output?

Top comments (1)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.