Most teams treat prompt work as a writing problem. You phrase the request, read the answer, adjust a few words, and move on when it looks right. That loop feels productive, and it is why so many prompts fall apart the moment they meet real inputs. The prompts that hold up in production are the ones that were measured, not the ones that were polished.
Why The Same Question Gets Two Different Answers
A language model is not executing your instructions, it is predicting what text should follow them. Every word in the prompt is a lever on a probability distribution rather than a statement of intent. Two phrasings that mean the same thing to a person can sit in very different parts of that distribution, which is why the same question asked two ways produces answers that differ in accuracy, format and length.
This is also why "the model got worse" is usually the wrong diagnosis. Nothing changed in the model. The inputs changed, and the prompt was only ever tuned against the handful of inputs you happened to try while writing it.
The Techniques That Actually Move Accuracy
A short list does most of the work. A system prompt that states the role, the constraints, and what to do when the model is unsure. Two or three examples of the exact output you want, because a demonstrated format beats a described one. Structured output, so the response is a schema you can parse instead of prose you have to interpret. And an explicit reasoning step for anything that contains more than one decision.
Prompt chaining is the one people skip. Splitting a task into two smaller prompts, each with a narrow job, almost always beats one long prompt trying to do everything, and it gives you a place to check the intermediate result before it poisons the next step.
Prompts For Agents Are A Different Job
For a single question, the prompt shapes one answer. For an agent, the prompt shapes every decision the agent makes for the rest of the run: which tool to call, when to stop, what to do with an error, how much to ask the user.
That raises the cost of vagueness. A tool description that never says when not to use the tool will get the tool used constantly. A system prompt with no stopping condition produces an agent that keeps going. Most agent behavior that looks like a model problem turns out to be an unwritten rule in the prompt.
Build A Test Set Before You Optimize
The cheapest useful thing you can build is a file of twenty real inputs paired with the outputs you would accept. Run it before and after every prompt change. It takes an afternoon to put together and it converts prompt work from opinion into a number that moves.
Once that exists, the cost conversation gets easier too, because you can finally see what shortening the prompt or dropping an example actually costs you in quality. The longer version of all of this, including the token budget side, is in our prompt engineering guide.
The Takeaway
Prompt engineering earns the second word only when there is a measurement attached to it. Write the prompt, then build the small test set that tells you whether the next edit helped. Everything else is rewriting.
Top comments (0)