If you run abliterated (uncensored) local models, you may have seen this: you ask for a short piece of writing, and instead of finishing, the model starts saying the same sentences again. And again. And again.
I measured that loop on my own machine, changing one setting at a time. Setup: qwen3.5-abliterated:9b on a Mac mini, in Ollama and llama.cpp, same prompt every run (a ~1,000-character essay on Japan's four seasons). One run per setting — see the limitations at the end.
The symptom: with no repetition settings, the model produced 16,291 characters instead of ~1,000, and 76% of its sentences were repeats. The most-repeated sentence appeared 13 times. A light repeat_penalty of 1.1 barely helped: 17,344 characters, still 76%.
Disclosure
Based on my own local experiments (cost: time and electricity only). The final section's two links (Novita AI, Vast.ai) are affiliate links — I may earn a commission at no additional cost to you. I have not paid for or used either service, so nothing here is a first-hand review; they are described only from their official pages, as options when local hardware is not enough. A longer paid measured note from the same experiments exists (end of this article).
Ollama: the range mattered, not the value
| Setting | Repeated | Most repeats | Ending |
|---|---|---|---|
| No penalty | 76% | 13× | Cut off |
| repeat_penalty 1.1 only | 76% | 11× | Cut off |
| 1.15 + repeat_last_n 1024 | 52% | 6× | Natural end |
| 1.3 + repeat_last_n 1024 ("rp3") | 6% | 2× | Concluded |
The pattern points at repeat_last_n — the window the penalty looks at. Ollama's default is 64 tokens; a sentence-level loop returns hundreds of tokens later, outside that window, so the penalty never sees it. Widening the window worked; raising the value alone did not.
My current Ollama setting: repeat_penalty 1.3 + repeat_last_n 1024. Caveat: at 1.3 the Japanese near the end got slightly unnatural. If naturalness matters more, start at 1.15 with repeat_last_n 1024.
llama.cpp: the loop was in the reasoning
llama-server returns reasoning and answer separately, so I measured them separately:
| Condition | Reasoning repetition | Answer repetition |
|---|---|---|
| No DRY | 64% (max 9×) | 0% |
| DRY multiplier 0.8 | 0% | 0% |
| DRY multiplier 0.4 | 5% | 0% |
The biggest finding: the loop lived in the reasoning, not the answer. Ollama's ollama run mixes both into one stream, so the 76% → 6% above was mostly a reasoning loop. DRY 0.8 erased the loop but caused heavy overuse of Japanese quotation marks (52.3 per 1,000 characters vs 1.3 baseline). At 0.4, repetition stayed low (5%) and the overuse dropped to about a third (15.0). My current setting: DRY multiplier ~0.4, base 1.75, dry_penalty_last_n 8192, ctx 16384.
Honest limitations
- Every setting was run once; sampling randomness is not separated from the settings' effects. Treat these as measured indications, not proven constants.
- All numbers come from one model and one prompt. Yours may behave differently.
- Stronger settings cost some naturalness. The values above are a balance point, not a free win.
When local isn't enough
A Mac mini was enough for a 9B model. For much larger models, the realistic options are renting GPU time or using a hosted API. Two services that publish official information about this (starting points for your own comparison, not personal recommendations — I haven't used them):
- Novita AI — hosted model APIs and GPU services. https://novita.ai/?ref=m2zhmmz&utm_source=affiliate
- Vast.ai — a marketplace for renting GPU compute. https://cloud.vast.ai/?ref_id=870806
Check current pricing and terms on the official sites before paying — I can't verify those from experience.
The full measured note
This article is the summary. The fuller measured note (complete settings, exact configurations, side-effect counts, caveats) is here: https://shinkalabs.gumroad.com/l/rnjmph
If you try rp3 or DRY 0.4 on a different model, I'd like to hear whether the loop behaved the same way.
Top comments (0)