An LLM asked to generate the same kind of thing repeatedly — a follow-up question, a product description, a code review comment — will often reach for the same phrasing across calls that have nothing to do with each other. That's not a bug in the usual sense. It's a predictable outcome of how three separate things interact, and treating it as one problem with one fix usually leaves two of the three causes untouched.
I ran into this directly while building MockEvalio's interview follow-up system, and fixing it took three separate changes, not one. The reasoning generalizes past that specific case.
Cause one: the prompt is teaching the model its own habit
If a prompt includes example phrasings to show the model what a "good" output looks like, the model doesn't just learn the shape of a good answer — it learns those specific phrasings as plausible things to say. MockEvalio's interviewer-persona prompts did exactly this: the instructions for one persona included, as an example, "What would you sacrifice to ship this faster?" That's a reasonable illustration for a human reading the prompt. For the model, it's now part of the pattern it's drawing from every time it generates a follow-up in that persona's voice — and a small, fixed set of examples is a strong attractor when you're calling the same prompt hundreds of times.
The fix here isn't "write better examples." It's recognizing that any literal phrasing in a prompt is a phrasing the model is now more likely to reproduce, and either avoiding fixed examples entirely in favor of described patterns ("rotate across these categories of question," not "here are three sample questions"), or accepting that repetition of your examples specifically is the cost of including them.
Cause two: temperature is quietly doing less than you think
Sampling temperature controls how sharply a language model favors its single most probable next token. Lower temperature makes the model's most likely continuation dominate more strongly; higher temperature flattens that distribution and gives less-probable-but-still-reasonable continuations more of a chance to surface. MockEvalio's follow-up generation ran at 0.4 — low enough that the model's single most likely phrasing for a given prompt shape would win consistently, call after call.
Temperature is a real lever, but it's a blunt one. It doesn't target the specific phrase you're tired of seeing — it reduces the model's confidence across everything it generates for that prompt, including good outputs you weren't trying to change. Raise it enough to meaningfully diversify output, and you're also raising the odds of an answer that's technically less coherent or slightly off-topic. It's a dial, not a fix aimed at a specific symptom.
Cause three: how often you're actually asking matters as much as what you get back
The least obvious cause isn't about generation quality at all — it's about exposure. MockEvalio's original logic asked for an AI follow-up on every single strong answer, unconditionally. That means the exact prompt shape most likely to produce a repeated phrase was firing at 100% frequency, across every user, every session. The model's tendency to converge on certain phrasings didn't change — but the number of times a person could actually notice it went way up, because the same narrow code path kept firing.
Reducing how often that path executes — MockEvalio moved to a 35% probability for strong answers, pulling a genuinely different question the rest of the time — doesn't make any individual generation less repetitive. It reduces how often you're exposed to whichever repetition is still there. That's a legitimate fix for the symptom a user experiences, and a completely different kind of fix from the other two.
Why you need more than one of these
Each lever has a real, separate failure mode if used alone:
- Explicit negative constraints ("don't say X") are reactive and fragile — you can only ban a phrase after you've already noticed it repeating, and there's no guarantee the model's next favorite phrasing isn't just as narrow. - Temperature increases add variety without adding control — you're as likely to diversify into a worse answer as a better one. - Frequency gating doesn't touch quality at all — it only changes how often a person sees whatever the underlying generation tends to produce.
None of the three, alone, addresses what the other two do. Combined, they cover more of the actual problem surface — a structural cause (how often does this exact prompt fire), a sampling cause (how sharply does the model favor its top candidate), and a prompt-content cause (is the prompt itself demonstrating the pattern you don't want repeated) — because repetition in LLM output isn't generally one problem wearing different clothes. It's three different mechanisms that happen to produce the same symptom.
Limitations
This reasoning is grounded in one real, fairly small case: a single commit, with configuration values (0.35 probability, 0.72 temperature) that — as far as the repository shows — were never empirically validated against measured output diversity after the fact. The three-cause framing itself draws on well-established, generally known behavior of temperature/sampling and prompt design, not a controlled experiment. Treat this as a diagnostic framework worth checking against your own system, not a benchmarked result.
Practical recommendation
When an LLM-generated feature feels repetitive, check all three independently before assuming a fix in one place solved it: does the prompt contain literal example phrasings the model could be echoing, is the sampling temperature low enough to sharply favor one continuation, and how often does the exact same prompt shape actually fire in production. A fix that only touches one of the three will usually look like it worked — because it changed something real — while leaving most of the actual cause in place.
Top comments (0)