Teaching a 7B model on a CPU to finish what it starts

Here is a complete run of my agent. I gave it one sentence: "What is the tallest mountain in Africa, and how tall is it in meters and in feet?" It searched DuckDuckGo, opened the Wikipedia article on Mount Kilimanjaro, recorded the answer, checked each part of the question against what it had found, and wrote this:

The tallest mountain in Africa is Mount Kilimanjaro [1], and its elevation is 5,895 meters (19,341 feet) above sea level [1].

1. Turn 3, from https://en.wikipedia.org/wiki/Mount_Kilimanjaro: Mount Kilimanjaro is the highest mountain in Africa, with an elevation of 5,895 m (19,341 ft) above sea level.

Four turns, about twelve minutes. Not impressive for a frontier model in the cloud. But this ran on an Intel i5-9500 (six cores, no graphics card) with Qwen2.5-7B-Instruct, quantized to 4 bits and served by LM Studio. Nothing left the machine except the searches and page visits themselves.

Early versions of this agent never finished anything. They searched the same thing over and over, recorded nothing, wandered off topic and ran until the turn limit. This article is about what changed between then and now: 30 versions, each tested on live runs and measured. The project is open source, at github.com/SnowFXXX/LocalAgentBrowser, and installs with pip install localagentbrowser.

The setup

The agent is a single Python file. Each turn the model sees the objective, what it has found so far, a short journal of its recent actions and the current page, and answers with one JSON action:

{
  "thought_process": "The article's first part gives the height in meters and feet. I should record it.",
  "action": "DISCOVER",
  "parameter": "Mount Kilimanjaro is the highest mountain in Africa, with an elevation of 5,895 m (19,341 ft)."
}

There are five actions: SEARCH, NAVIGATE, READ_MORE (the next part of a long page), DISCOVER (record a finding) and FINISH. LM Studio constrains the reply to a JSON schema, and Playwright carries the action out in a visible Chromium window you can watch.

That is the easy part. Everything below is about keeping a 7B model on track.

Lesson 1: Enforce rules in code, not in the prompt

The prompt said "don't repeat searches". The model repeated searches. A small model reads a rule and then does the most likely thing anyway, especially many turns later in a long prompt.

So every rule that can be checked, is checked in code before anything happens: repeated searches, opening the page it's already on, visiting the same URL more than three times, empty findings, findings that duplicate earlier ones (recognised by word pairs, so rewording or reordering lines doesn't fool it). A refused action comes back to the model with the reason.

Two things I learned the hard way:

• Check the checks. In one early version, the duplicate-URL check ignored the query string, so every search after the first counted as a duplicate. 34 of 35 turns in that run were blocked. The tests that now cover these rules run in about a second with a fake model and fake browser.
• When the model misbehaves the same way every time, replay the prompt. The model kept sending DISCOVER with an empty parameter. Replaying the exact prompt showed why: it thought DISCOVER was a command that would extract the information for it. One sentence ("you must write the information itself into the parameter") fixed it in every replay.

Lesson 2: On a CPU, reading the prompt is the whole cost

On this machine the model reads about 8 tokens per second, and reading the prompt is about 90% of every model call. A 4,000-token prompt costs minutes before the model writes a single word.

But LM Studio (like other llama.cpp-based servers) skips any part of a prompt that matches the start of the previous one. So the agent doesn't rebuild its prompt each turn. It keeps one growing conversation and appends only what changed: a one-line result, the next part of a page, a new page. It starts a fresh prompt only when the conversation nears the context limit.

Change	Effect
Reorder the prompt so consecutive turns share a longer identical start	About 25% faster per turn
Continue the conversation instead of rebuilding it	Same-page turns from 150–220 s to 35–120 s
Correct a refused action inside the same conversation ("cheap retry")	25–45 s instead of a wasted turn of 150–200 s
Shorten old findings in fresh prompts; use 75% of the context before rebuilding	Fresh prompts from 234 s to 162 s on average

The cheap retry matters more than it looks. A small model gets things wrong often. If every refusal costs a whole turn, the agent spends most of its time being refused. If it costs a short appended note, mistakes become cheap.

Lesson 3: A small model doesn't know when it's done

This was the real wall. The agent would find good material and then keep going, or drift to loosely related pages, or leave a page without recording what was on it. Three additions got it to finish for the first time:

• A progress check. After four turns without a new finding, it is asked to compare its findings with the objective, then finish or name what's missing and go find it. Without this, it never finished.
• An objective check every eight turns, even while findings keep coming, because findings on loosely related topics kept resetting the progress check.
• Confirm before leaving. Leaving a page with nothing recorded from it triggers one short question: record something, or repeat the action to leave. It costs one cheap retry when leaving really is right.

With those, version 1.27 finished a documentation research task on its own: 13 turns, about 36 minutes, with the commands and file layout it was asked for.

Then the opposite problem appeared. Asked the mountain question, the agent recorded the full answer on turn 3, wrote in its own reasoning that "the objective has been met", and kept reading the 33-part article for interesting details. The fix was a note after every new finding: if your findings now cover every part of the objective, finish now. Next run: finished on turn 4.

Lesson 4: Ask for evidence, not a yes or no

"Do your findings satisfy the objective?" A 7B model answers yes far too easily. On a task to research how spiritual traditions view cats and then form an opinion, it finished after one blog post that named no traditions, with a summary instead of an opinion.

What works better is making it show its work. Before finishing, it must split the objective into parts and write each part followed by the finding that covers it, or "not covered". It's also told how many separate sources its findings come from. The next run named six traditions from two sources and stated an opinion.

The model still finds loopholes. On a comparison task, its own check read "Available Libraries: Not covered. Written at FINISH.", and it finished anyway: it had stretched a phrase meant for its own opinion to cover information it hadn't found. Now a finish whose own reasoning says "not covered" is sent back to work on that part, at most twice per run, so something that truly isn't on the web can't block it forever.

Lesson 5: Make writing the answer a separate job

At first, the agent's FINISH text was the result. Small models write much better when writing is their only job, so the final result is now a separate model call that sees only the objective and the numbered findings, with a short numbered list of rules.

Each rule came from a real failure:

• When that step continued the browsing conversation (to reuse the cache), it used page text the agent had read but never recorded, and cited findings that didn't say what it claimed. Fix: a fresh prompt with only the findings. It turned out about as fast.
• On a how-to (a sourdough starter), it invented the first water amount, "45 grams (about 1/4 cup)", where 45 g was really the daily feeding. It had copied the guess from the agent's own draft. Fix: no draft, no outside knowledge, say when a detail wasn't found. The report now also flags any number in the final result that appears in none of the findings.
• Asked for three Photoshop alternatives, it listed four and added an opinion nobody asked for. Both rules were there, buried in one long paragraph. Fix: rules as a short numbered list. Next run: exactly three, no opinion.

Lesson 6: Test different kinds of tasks

Until version 1.29, every test used the same objective. That's how you overfit a set of prompts. Six different kinds of task each broke something new:

Kind of task	Outcome
Documentation research	Finished in 13 turns (~36 min) with commands, roles and file layout
Opinion from several sources	Six traditions from two sources and an opinion, 18 turns (37 min)
Factual question	Correct, cited answer in 4 turns (~12 min)
Comparison and recommendation	Cited comparison and a recommendation, 16 turns (33 min); one part not covered
List of items	Exactly the three items asked for, all cited, 9 turns (24 min)
Step-by-step how-to	A clear plan in 7 turns (19 min), with one invented amount, which led to the number check

Times are model time on the CPU. A GPU would cut them dramatically.

What's still hard

• Reading without recording. It sometimes reads a long page part by part without recording the relevant parts, even with an outline of the page's headings in front of it. This is the main weakness now.
• Very strict objectives. Told "do not finish until everything is fully validated", it keeps searching for details the web may not contain.
• Judgement. Results depend a lot on the first pages a search turns up, and a 7B model sometimes cites the wrong finding number, so check claims against the listed findings.

What I'd tell anyone building agents on small models

1. Put every checkable rule in code, and tell the model why it was refused.
2. Make mistakes cheap. Correct them inside the same conversation.
3. On slow hardware, design for the prompt cache: append, don't rebuild.
4. Don't ask "are you done?". Ask it to show, part by part, what covers each piece of the objective.
5. Separate gathering from writing, and give the writer only the evidence.
6. Test on different kinds of tasks early, and measure every change.

Try it

You need LM Studio with a model loaded (I use Qwen2.5-7B-Instruct) and Python 3.10 or later. The agent is on PyPI:

pip install localagentbrowser
python -m playwright install chromium
localagentbrowser

Type an objective and watch it work. When it's done, read research_report.md. The changelog has every version with the measurements behind each decision. Results with other models and other kinds of tasks are very welcome as issues.

I built this with Claude (Anthropic) as a coding partner from version 1.13 onward, working in Claude Code; those commits list Claude as co-author.