My Local LLM Was Running at 1.6% of Its Context. Here's the Setting That Fixed It

I run a content pipeline on a Mac mini (48GB unified memory) that splits long blog drafts into platform-specific short-form pieces. That job — read a 30-page document, hold the whole thing in mind, extract what matters for YouTube Shorts vs TikTok vs Reels — is exactly what long-context LLMs are supposed to be good at.

Mine wasn't. It kept "forgetting" the second half of every document, dropping key details, and producing shallow summaries no matter how I tuned the prompt.

Three weeks of tuning the wrong thing

I did what you'd do. Simplified the prompt. Rewrote the template. Swapped models. Re-downloaded them, twice. Spent entire evenings after work on this, convinced the model was the problem — a Q4_K_M quantized 13B–20B model should handle long documents, right? The symptoms said otherwise: solid on the first pages, incoherent by the end.

Classic context-window behavior. I just didn't see it yet.

The one line in the console

Then I actually read the LM Studio load log instead of scrolling past it:

context_length: 4096

The model I was running supports 262,144 tokens of context. It was loaded with 4,096.

That's 1.6% of what the model can do. A 48-lane highway restricted to one lane — and every long document I fed it was quietly getting truncated into memory of just the opening section.

Why it happened

LM Studio's just-in-time model loading picks a conservative default context length on first load. For chat and short Q&A, 4096 is plenty and keeps memory pressure low — a sensible default for most users. For document-scale work, it's a silent killer. Nothing errors out. Nothing warns you. The model just appears to have a bad memory.

Context is the model's working memory. Cap it at 4k tokens and a 30-page brief becomes "read the first two pages, forget the rest."

The fix

Two things:

1. Set context length explicitly on load. In LM Studio's model settings, Context Length: 260000 (whatever your model supports — check the model card, not the default), then reload. On 48GB of unified memory the larger KV cache is entirely affordable.
2. Guard against silent reloads. I added a simple flag to the pipeline so an in-flight job blocks model reloads. The default resetting itself mid-workflow is how you get this bug back.

The result

Immediate, dramatic improvement. Full-document comprehension, per-platform extraction without drift, details intact end to end. Same model, same hardware, same prompt — one setting was capping ~98% of the model's effective utility for my workload.

The lesson

Defaults are tuned for the average case, and document-scale synthesis is not the average case. When a local LLM "feels dumb," check what it was actually loaded with before blaming the weights:

• Find the real context limit on the model card
• Read the load log — what context_length is actually in effect?
• Set it explicitly, every time, in your load scripts

The most expensive performance bug I've shipped was a single default value.

---

This post is based on a first-hand work log, written with AI assistance.