Same Local Model, 3 Coding Agents: pi 12/12, Codex 12/12, Claude Code 7/12 on a 6GB GPU

Claude Code told me the web page was done. The folder was empty.

That was the worst moment of a small test: one local model on a 6GB laptop GPU, plugged into three coding agents. Same model, same tasks. Only the agent changed.

Setup

• RTX 3060 Laptop 6GB + 64GB RAM, WSL2 Ubuntu 24.04
• llama.cpp 9655061: llama-server -c 32768 -ngl 99 -fa on -ctk q8_0 -ctv q8_0 --jinja; experts on CPU (-ncmoe 36 for LFM, -ncmoe 40 for Qwen). Fully offline.
• Models: LFM2.5-24B-A2B Q4_0 (3 runs per task), Qwen3.6-35B-A3B UD-Q4_K_XL (1 run, control)
• Tasks:
  • A: fix two bugs in calc.py until pytest passes
  • B: single-file timer page, checked by a script against 8 items
  • Pelican: Generate an SVG of a pelican riding a bicycle
• 900 s cap per run

Results

Agent	Passed	LFM bug-fix median	First prompt
pi	12/12	47 s	~1,400 tokens
Codex	12/12	85 s	~6,400 tokens
Claude Code	7/12	132 s	~15,800 tokens

The prompt is the problem

Every request starts with the agent's system prompt and tool definitions. First prompt eval in the llama-server log:

Agent	Tokens	Time
pi	1,442	5,666 ms
Codex	6,429	18,567 ms
Claude Code	~15,800	48,837 ms

With a 32K context, Claude Code fills about half of it before doing anything, and reading that prompt takes about 8.6× longer than pi's.

pi

12/12 on both models, zero tool errors in the logs, fastest bug fix (47 s median). I also trimmed its prompt to about 750 tokens with only read/edit/bash enabled. Still 12/12, same speed. Harder tasks might separate the two; these didn't.

Codex

12/12, but LFM made 12 tool-argument errors, mostly integers sent as strings:

failed to parse function arguments: invalid type: string "96787", expected i32

It retries and recovers. The cost is time: 85 s median for the bug fix, 599 s on the first run.

Claude Code

• LFM passed the bug fix once in three runs. The other two died with:

  Prompt is too long · automatic compaction failed: summarization produced empty response

• Timer page: 3/3, but an 881 s median, and one run hit the 900 s cap.
• Qwen: 0/3. For the page and the pelican it replied that the task was complete. The working folder had no files.

An error tells you something failed. A false "done" only shows up when you check the folder yourself.

Pelicans

Called directly, LFM's pelican floats above the bike; inside an agent it actually sits on it. Qwen's drawings are more complete. Claude Code + Qwen produced no SVG.

Takeaways

• Small local model, small context: pick the agent with the smallest prompt. Here that was pi, then Codex.
• If you must use Claude Code locally, give it more context (Ollama's docs suggest 64K+) and look at Unsloth's notes on trimming its prompt (--bare). I didn't test those here.
• Always ls after an agent says it's done.

Limits

• VRAM was sampled once per second, so I only saw resident usage (~2,641 MiB LFM, ~3,111 MiB Qwen), not peaks.
• Tool-error counts come from log keyword matching and are approximate.
• Qwen got one run per task.
• Three small tasks, not a real project.

Full data and configs: https://kunbuilds.com/en/posts/agents-local-model-6gb/

Sources

• Ollama, Claude Code integration: https://docs.ollama.com/integrations/claude-code
• Unsloth, Claude Code with local LLMs: https://unsloth.ai/docs/basics/claude-code

Comments

[Mihai Perdum]

The empty folder with a confident "done" is the most useful finding here. For the Qwen 0/3 runs, did the llama-server log show a tool call arriving at all, or only a plain text reply? If the model emits its call in a format the server's parser doesn't pick up, it can come back as normal content with finish_reason stop, and the agent reads that as finished. From the outside that looks the same as the model claiming work it never did, but the fix is different. Curious which one it was.

[Paul Irolla]

Same local model, same tasks, 12/12 with pi and Codex, 7/12 with Claude Code. The system prompt decided that gap, so the harness belongs in the result, not in a footnote. Report it next to the model every time, or the pass counts do not compare across tools.