My Free Coding Model Kept Failing on Big Repos. The Fix Was a Context Budget, Not a Better Model.

A while ago I hit a wall that a lot of developers hit with free or low-cost coding models: the model would nail small, self-contained questions, then completely fall apart the moment I pointed it at a real repository. Wrong files edited, hallucinated imports, confident answers about functions that don't exist.

My first instinct was the obvious one — the free model just isn't smart enough. But before switching models, I tried something cheaper: I measured what I was actually sending. It turned out my "repo-aware" prompts were 60–90% irrelevant code. The model wasn't failing on the task. It was drowning in my context.

This article is the workflow that came out of that: treat your prompt's context window as a budget, spend it deliberately, and verify the spend with a trivially small check. It works with any model, and it's especially useful when you're on free model access where you can't just throw a giant context window at the problem.

Why context stuffing backfires

Two failure modes show up constantly:

1. Attention dilution. When the relevant function is 400 lines inside a 25,000-token prompt, models routinely anchor on the wrong file. More context is not monotonically better — past a point it actively degrades accuracy.
2. Truncation roulette. If your prompt silently exceeds the model's real limit, something gets cut — and you don't control what. I've seen the task description itself get truncated while three irrelevant test fixtures survived.

The fix is boring and effective: decide how many tokens you can spend, score your files by likely relevance, and pack only what fits — with a receipt showing what you included and why.

The artifact: a greedy context packer

Here's a small, runnable Python script (standard library only) that implements the budget idea. It scans a repo, scores each file by keyword overlap with your task description, and greedily packs files under a token budget. The token count uses a crude characters-per-token estimate — deliberately, so there's zero setup. Swap in a real tokenizer for your model if you have one.

#!/usr/bin/env python3
"""context_budget.py — pack a repo into a prompt under a token budget.

Usage:
    python context_budget.py ./src "fix the retry logic in the payment client" 6000
"""
import os, re, sys

CHARS_PER_TOKEN = 4  # crude estimate; replace with a real tokenizer if available
SKIP_DIRS = {".git", "node_modules", "__pycache__", "dist", ".venv", "vendor"}

def tokens(text: str) -> int:
    return max(1, len(text) // CHARS_PER_TOKEN)

def keywords(task: str) -> set:
    return {w.lower() for w in re.findall(r"[a-zA-Z_]{3,}", task)}

def iter_files(root: str):
    for dirpath, dirnames, filenames in os.walk(root):
        dirnames[:] = [d for d in dirnames if d not in SKIP_DIRS]
        for f in filenames:
            if f.endswith((".py", ".ts", ".js", ".cpp", ".h", ".go", ".rs", ".java")):
                yield os.path.join(dirpath, f)

def score(path: str, content: str, kws: set) -> int:
    haystack = (path + " " + content[:4000]).lower()
    return sum(haystack.count(k) for k in kws)

def main():
    root, task, budget = sys.argv[1], sys.argv[2], int(sys.argv[3])
    kws = keywords(task)
    candidates = []
    for path in iter_files(root):
        try:
            content = open(path, encoding="utf-8", errors="ignore").read()
        except OSError:
            continue
        candidates.append((score(path, content, kws), tokens(content), path, content))

    # Highest relevance first; break ties by smaller files (more per budget)
    candidates.sort(key=lambda c: (-c[0], c[1]))

    spent, packed = 0, []
    for s, t, path, content in candidates:
        if s == 0:
            continue  # zero relevance: never pack
        if spent + t > budget:
            continue  # doesn't fit; try the next one
        spent += t
        packed.append((path, t, s))

    print(f"# Task: {task}")
    print(f"# Budget: {budget} tokens | Spent: {spent} | Files: {len(packed)}\n")
    for path, t, s in packed:
        print(f"## {path}  ({t} tokens, relevance {s})")
    print("\n---PROMPT BELOW---\n")
    for path, _, _ in packed:
        print(f"### File: {path}")
        print(open(path, encoding="utf-8", errors="ignore").read())
        print()

if __name__ == "__main__":
    main()

The output doubles as the prompt and the receipt: the header tells you exactly how much of the budget you spent and which files earned their place. Save those receipts. When a model answer goes wrong, the first debugging question becomes "was the right file even in the context?" — which you can now answer in seconds instead of guessing.

The workflow around the script

The script alone isn't the workflow. This is the loop I run:

1. Measure before blaming. Before concluding a model is weak on your repo, run the packer and look at what you would have sent. If the relevant file wasn't in your old prompts, you never tested the model — you tested your context hygiene.
2. Write the task description like a search query. The scoring is keyword-driven, so "fix the bug" selects nothing, while "fix retry backoff in the payment client timeout path" selects the right neighborhood. This is a feature: it forces you to articulate the task precisely, which also improves the prompt itself.
3. Set the budget from the model, not from your repo. Find your model's real context limit from its official docs, reserve ~30% for the task description, instructions, and the model's own output, and use the rest as the packing budget.
4. Verify the spend with a canary question. Before asking for the actual fix, ask: "Which file contains the retry logic, and what is the function signature?" If the model answers correctly, your context spend was right and you proceed. If not, no amount of clever prompting will save the session — repack first. This 20-second check has saved me more time than any prompt-engineering trick.
5. Iterate on failure, not on vibes. When an answer is wrong, check the receipt: missing file (raise budget or improve task keywords), right file but wrong edit (that's genuinely the model), or truncation (lower budget — you're over the real limit).

Where free model access and a free server fit

This loop is iteration-heavy by design — packing strategies, budget sizes, canary questions — and that iteration is exactly what gets awkward on metered APIs. This is where I've found MonkeyCode's free model access genuinely useful: I can run step 4's canary checks and step 5's failure iterations against free models without watching a bill, which makes it practical to tune the context instead of giving up on the model after one bad answer. Their free server option also matters here in a concrete way: the packer script and the receipt logs live there as a tiny always-on utility, so the whole loop runs from any machine without me babysitting a local setup.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Two honest notes: the free access is what makes the iteration cheap — it doesn't change the models' context limits, so the budgeting discipline above still does the real work. And I have no verified numbers on quotas or duration for the free tier, so treat both as "generous enough for a tuning loop," not as infrastructure to build a product on.

Limitations, and who should skip this

• The relevance heuristic is naive. Keyword overlap misses semantic relevance — a task about "authentication" won't match a file that only says verify_credentials. If you hit this often, upgrade the scorer to embedding similarity; the budget logic stays identical.
• Some tasks genuinely need wide context. Cross-cutting refactors, API migrations touching dozens of files, or "why is this system slow" questions can't be packed into 6,000 tokens. For those, a budget forces bad tradeoffs — use a larger-context model or decompose the task instead.
• The token estimate is crude. chars / 4 is wrong for code-heavy files in some languages. Fine for relative budgeting, but verify against your model's real tokenizer before trusting the ceiling.
• It doesn't fix weak models. If the receipt shows the right file was in context and the canary passed, and the edit is still wrong — that's the model. Budgeting removes context as an excuse; it doesn't remove the model's actual ceiling.
• Skip this entirely if your repo fits comfortably in your model's context and your answers are already good. Don't add machinery to a problem you don't have.

Wrapping up

The mental shift that mattered: stop treating the context window as a bucket and start treating it as a budget with a receipt. Once I could see what I was spending, most of my "the free model isn't good enough" cases turned into "I never showed it the right 800 lines."

If you want to try the loop, the script above runs anywhere Python does — and a free model tier plus a free server (MonkeyCode's or any equivalent) is honestly all you need to tune it against your own repos. Measure first, blame the model second.

Example outputs and scores in this article are illustrative; run the packer against your own repository and verify context limits against your model's official documentation.