Your agent writes code that looks right. Interfaces with one implementation. Comments that restate the next line. try/except that swallows errors. Names like data and process. A dependency that does not exist. Tests that were edited to pass.
Reviewers approve it because it resembles the patterns good engineers write. That is the failure mode. AI output does not usually look broken. It looks thorough.
I built an agent skill that catalogs these patterns and deletes them. This post is the catalog, a real before/after, and the eval I added so the claim can be checked.
The test that catches most of it
Before anything else, ask: does this need to exist?
If a feature, abstraction, flag, or line is not required, delete it. Cleaning up code that should not exist is wasted work.
Then the portability test. If a function, comment, or sentence could move unchanged into an unrelated project, it carries no information about this one. processData, // Initialize the service, and Handles the request. pass between any two repositories. That is the tell.
Before and after
An agent asked to add invoice saving produced this:
class InvoiceProcessorInterface(ABC):
"""Interface for invoice processors."""
@abstractmethod
def process(self, data) -> dict:
"""Process the invoice."""
...
class InvoiceProcessor(InvoiceProcessorInterface):
def __init__(self, db):
# Store the database connection
self.db = db
def process(self, data) -> dict:
# Initialize the result
result = {}
# Loop through each item
for item in data:
# Try to parse the amount
try:
amount = Decimal(item["amount"])
except Exception:
amount = 0
result[item["id"]] = self._handle(amount)
return result
class InvoiceProcessorFactory:
"""Factory for invoice processors."""
@staticmethod
def create(db) -> InvoiceProcessorInterface:
return InvoiceProcessor(db)
The skill returned this:
def save_invoice_amounts(db, invoice_lines) -> dict:
return {
line["id"]: db.save({"amount": Decimal(line["amount"])})
for line in invoice_lines
}
Diff: 8 insertions, 54 deletions.
It inlined the one-implementation interface and the single-product factory, renamed data, result, and _handle, deleted six restating comments, and removed the except Exception that turned an unparseable amount into zero.
The patterns
Code:
- Plausible but wrong logic: off-by-one errors, inverted conditions, wrong boundaries, skipped edge cases.
- Hallucinated APIs and packages. A USENIX Security 2025 study of 2.23 million AI samples found 19.7% referenced a package that does not exist, and 43% of the invented names recurred across runs.
- Swallowed errors and silent fallbacks.
- Missing trust-boundary checks: no authorization on a new endpoint, validation only on the client.
- Secrets in code or logs.
- Retries that ignore
Retry-After, missing timeouts, no rate-limit handling. - Non-idempotent retries and race conditions.
- N+1 queries and unbounded result sets.
- Speculative abstraction: one-implementation interfaces, single-product factories.
- Reinvented standard library.
- God functions and shotgun diffs.
- Architecture and layer violations.
- Generic naming.
- Redundant and stale comments.
- Defensive bloat.
- Dead code.
- Formatting noise.
- Test slop: assertions that cannot fail, tests that mirror the bug they were written from.
Generated prose:
- Filler and buzzwords: seamless, robust, leverage, at scale, deep dive.
- Warm-up openers: "Here's the thing", "It's worth noting".
- Setup-then-reveal contrasts, "nobody tells you" hype, colon drama, encore paragraphs.
- Commit and PR slop: past-tense subjects, tool footers, diff checklists.
The part most tools miss
An agent that writes a redundant comment and then edits a failing assertion to make the suite green is not a style problem. It is a correctness problem, and the suite is now lying.
So the skill also reviews the agent's own behavior:
- Test tampering and reward hacking. A 2025 study of frontier models on engineering tasks found reward hacking in 30.4% of runs, including one agent that overrode Python's equality check so every test passed.
- False success reporting: claiming tests passed on a run that errored.
- Task-boundary violations: editing files outside the task, reformatting unrelated modules.
- Silent behavior changes: a refactor that drops a guard or changes a default with no mention.
- Self-review blindness: the agent that wrote the bug is the worst reviewer of it.
Add an eval, or it is just a claim
Most skills ship on claims. I added three labeled fixtures, two sloppy and one clean, and a harness that scores recall, precision, and false positives on the clean file.
| Model | Recall | Precision | Findings on clean |
|---|---|---|---|
| gpt-6-luna (default) | 1.00 | 0.91 | 0 |
| claude-haiku-4-5-20251001 | 1.00 | 0.83 | 0 |
The harness exits non-zero when recall drops or when the skill reports a finding on clean code. That second check matters most. A skill that invents problems is worse than one that misses a few.
The limits are in the repo: three fixtures, keyword scoring, one run per model. Keyword scoring is a floor, not a grade.
Install
npx skills add kirankunapuli/stop-ai-slop
It installs to 79 agents, including Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode, Windsurf, Zed, and Pi.
There is also a GitHub Action that reviews a pull request in report-only mode:
- uses: kirankunapuli/stop-ai-slop@v1
with:
api-key: ${{ secrets.OPENAI_API_KEY }}
It runs on OpenAI, Anthropic, Google, Groq, OpenRouter, local Ollama, or any OpenAI-compatible endpoint.
What it leaves alone
Validation and authorization at trust boundaries, error handling that prevents data loss, security, accessibility, concurrency correctness, domain rules that are load-bearing, and the writer's voice. It never adds an abstraction, dependency, or config knob in the same pass that removes one.
The useful ask
Where has your agent produced slop that a reviewer waved through?
Top comments (0)