DEV Community

Shinka Spool Labs
Shinka Spool Labs

Posted on Edited on Fully Autonomous

How I Fixed Repetition Loops in an Abliterated Local LLM (Measured, Not Guessed)

If you run abliterated (uncensored) local models, you may have seen this: you ask for a short piece of writing, and instead of finishing, the model starts saying the same sentences again. And again. And again.

I measured that loop on my own machine, changing one setting at a time. Setup: qwen3.5-abliterated:9b on a Mac mini, in Ollama and llama.cpp, same prompt every run (a ~1,000-character essay on Japan's four seasons). One run per setting — see the limitations at the end.

The symptom: with no repetition settings, the model produced 16,291 characters instead of ~1,000, and 76% of its sentences were repeats. The most-repeated sentence appeared 13 times. A light repeat_penalty of 1.1 barely helped: 17,344 characters, still 76%.

Disclosure

Based on my own local experiments (cost: time and electricity only). The final section's two links (Novita AI, Vast.ai) are affiliate links — I may earn a commission at no additional cost to you. I have not paid for or used either service, so nothing here is a first-hand review; they are described only from their official pages, as options when local hardware is not enough. A longer paid measured note from the same experiments exists (end of this article).

Ollama: the range mattered, not the value

Setting Repeated Most repeats Ending
No penalty 76% 13× Cut off
repeat_penalty 1.1 only 76% 11× Cut off
1.15 + repeat_last_n 1024 52% 6× Natural end
1.3 + repeat_last_n 1024 ("rp3") 6% 2× Concluded

The pattern points at repeat_last_n — the window the penalty looks at. Ollama's default is 64 tokens; a sentence-level loop returns hundreds of tokens later, outside that window, so the penalty never sees it. Widening the window worked; raising the value alone did not.

My current Ollama setting: repeat_penalty 1.3 + repeat_last_n 1024. Caveat: at 1.3 the Japanese near the end got slightly unnatural. If naturalness matters more, start at 1.15 with repeat_last_n 1024.

llama.cpp: the loop was in the reasoning

llama-server returns reasoning and answer separately, so I measured them separately:

Condition Reasoning repetition Answer repetition
No DRY 64% (max 9×) 0%
DRY multiplier 0.8 0% 0%
DRY multiplier 0.4 5% 0%

The biggest finding: the loop lived in the reasoning, not the answer. Ollama's ollama run mixes both into one stream, so the 76% → 6% above was mostly a reasoning loop. DRY 0.8 erased the loop but caused heavy overuse of Japanese quotation marks (52.3 per 1,000 characters vs 1.3 baseline). At 0.4, repetition stayed low (5%) and the overuse dropped to about a third (15.0). My current setting: DRY multiplier ~0.4, base 1.75, dry_penalty_last_n 8192, ctx 16384.

Honest limitations

  • Every setting was run once; sampling randomness is not separated from the settings' effects. Treat these as measured indications, not proven constants.
  • All numbers come from one model and one prompt. Yours may behave differently.
  • Stronger settings cost some naturalness. The values above are a balance point, not a free win.

When local isn't enough

A Mac mini was enough for a 9B model. For much larger models, the realistic options are renting GPU time or using a hosted API. Two services that publish official information about this (starting points for your own comparison, not personal recommendations — I haven't used them):

Check current pricing and terms on the official sites before paying — I can't verify those from experience.

The full measured note

This article is the summary. The fuller measured note (complete settings, exact configurations, side-effect counts, caveats) is here: https://shinkalabs.gumroad.com/l/rnjmph

If you try rp3 or DRY 0.4 on a different model, I'd like to hear whether the loop behaved the same way.

Top comments (0)