Every few weeks there's a thread about some LLM "failing" at a task. Half the replies are someone else running the same task, getting a solid result, and just not saying what prompt they actually used.
This isn't a defense of any particular model. It's a pattern worth naming: most "the AI got it wrong" reports are actually underspecified-request reports.
The gap that always existed
We've always had this problem with underspecified tickets. "Fix the login bug" without repro steps, expected behavior, or environment details gets you a fix — maybe not the right one. Nobody's surprised when that ticket produces a mismatched result. We just say the ticket needed more detail.
LLMs get a different reaction to the same failure mode. Instead of "the prompt needed more detail," it's "the model isn't good enough yet." Sometimes that's true. Often it's the same underspecified-ticket problem wearing a new interface.
What actually changes the output
Not length. Specificity. Compare:
"Write a function to validate user input"
vs.
"Write a function that validates a signup form:
email (RFC 5322, not just regex-lite), password
(min 12 chars, 1 number, 1 symbol), and username
(alphanumeric + underscore, 3-20 chars). Return a
list of specific error messages, not just true/false.
Python, no external deps beyond stdlib."
Same model. Wildly different output quality. Nothing about the model changed between those two requests — the constraint surface did.
The unglamorous actual skill
This is closer to writing a good spec or a good code review comment than it is to any "prompt engineering" secret technique. Explicit constraints. Explicit definition of done. Explicit scope boundaries (what NOT to do, sometimes matters more than what to do).
Most of us already know how to do this — we do it for Jira tickets, PR descriptions, code review comments. The skill transfers almost directly. It just doesn't feel like it should, because talking to an LLM still feels conversational instead of like writing a spec.
The actual debug step
Before filing a "the model can't do this" bug report against reality: would a new hire, handed the literal text of your prompt and nothing else, have produced something meaningfully better?
If not — that's not a model limitation. That's a spec problem, and it's fixable in the next five minutes, same as tightening a vague ticket.
Top comments (0)