The 8-bit model loaded. It generated for more than half an hour. It also failed before its agent session finished.
The 4-bit version completed the same experiment, ran faster, and built the thing I liked most: a surprisingly complete cellular automata workbench called Lattice.
That changed the question I ask about large local models. “Can my laptop load this?” is easy to answer and not especially useful. The better question is whether the model can stay alive through a real agentic loop and produce something worth keeping.
On my 48 GB M4 Pro, 4-bit Qwen3.8-27B did.
I tested complete agent sessions, not isolated prompts
My first attempt at this comparison was wrong.
I asked the 4-bit, 6-bit, and 8-bit models to emit an entire web artifact in one long completion. That produced numbers, but it removed the part that made our earlier Local LLM Choice experiments interesting: the model was supposed to work through an agent harness. It needed to create a file, inspect it, run it, notice problems, and revise its own work.
“I feel like we’ve not done this right” was the most useful conclusion from that first pass. We withdrew it and reran the experiment properly.
For the replacement, every quantisation started Pi in a genuinely empty directory. Each received the same broad Billie-domain prompt, the same sampling settings, and up to 25 Pi iterations. There were no retries and no repairs after the run. The model had to choose its own project and carry it through the same working loop.
You can open the public comparison to use all three preserved artifacts and inspect their run receipts and manual evaluations.
The useful result is whether the agent finishes
Here is what happened:
| Quantisation | Model files | Aggregate completion rate | Wall time | Peak MLX allocation | Outcome |
|---|---|---|---|---|---|
| 4-bit | 15.1 GB | 12.47 tok/s | 30m 20s | 33.98 GB | Completed: Lattice |
| 6-bit | 21.9 GB | 9.73 tok/s | 25m 32s | 36.10 GB | Completed: The Long Now |
| 8-bit | 28.6 GB | 6.88 tok/s before failure | 32m 10s | Not available | Partial: Metal OOM |
These are whole-session measurements, not raw decode benchmarks. Pi’s working history grew through the run, and the aggregate completion rate includes prompt processing. The figures are useful for comparing these three matched sessions, but they should not be treated as universal model speeds.
Model-file size is not the same as memory use either. The 8-bit server exited before it could produce a comparable final MLX allocation receipt, so I have not invented one for the table.
Four-bit was surprisingly good
Lattice is not just a Game of Life grid with a play button. It has draw and erase tools, play and single-step controls, randomisation, undo, speed and grid settings, symmetry modes, named patterns, live population and generation figures, and a small history chart.
The 4-bit model chose the idea and built this complete cellular automata workbench through Pi on my laptop.
The broad prompt did not ask for a cellular automata tool. The model chose that idea, decided what the workbench should contain, built it, opened it, exercised parts of it, and made changes over several rounds.
I’m ridiculously impressed by it.
It is also real enough to have real defects. Some library patterns do not behave as described. A few die or remain static even though the model’s tests later called them correct. Four-way symmetry can clip a stamp near the edge of the rectangular board. The browser smoke test showed that the controls ran without throwing errors; it did not prove the automata were correct.
That distinction matters. Lattice is impressive because it is a coherent piece of software I can use and criticise, not because it passed a flattering demo.
Six-bit won the scorecard; four-bit won me
The 6-bit model built The Long Now, an interactive 5,000-year timeline with logarithmic, hybrid, and linear views. During the run it measured its first scale mapping, found that the maths did not support its own explanation, and replaced it with a hybrid scale.
That inspect-and-repair behaviour is exactly why using Pi mattered. The result earned 4/5 for interest, 3/5 for execution, and 4/5 for taste in the formal review. It also contains overlapping labels and several simplified or wrong historical claims.
The 4-bit artifact scored 4/5 for interest, 3/5 for execution, and 3/5 for taste. I still preferred it. A structured scorecard can help describe an artifact without replacing the judgement of the person actually using it.
The 8-bit artifact is a preserved partial result. It had already built a substantial timeline and repaired some behaviour when the fourteenth provider call failed with a Metal out-of-memory error. The page renders, but its date calculations have central bugs and its labels crowd together.
All three models loaded. Only two completed the agent run.
What I would run on this machine
I would start with 4-bit Qwen3.8-27B on this 48 GB Mac.
In this experiment it was about 28% faster than 6-bit by the whole-session aggregate measure. It left more headroom for the harness and everything else on the laptop, and it made my favourite artifact. That is a much more useful combination than choosing the largest quantisation that will technically initialise.
Six-bit is completely viable when I want to trade some speed and memory headroom for the more considered behaviour it showed in The Long Now. I would not start another long 8-bit Pi session on 48 GB without shortening the working history or narrowing the task.
This was one run per quantisation, not a leaderboard. I did not measure battery use, temperature, or repeated-run variance, and another seed or project could reverse the quality order.
But the line has moved. A local model at 4-bit chose its own idea, used an agent loop to build it on my laptop, and left me with software I wanted to keep playing with.
It did this at 4-bit. Damn, yeah.
Want to talk about something I’ve written or built? Get in touch.
This article was adapted with AI assistance from an original article on billiem.uk. The original article was reviewed before publication.
Top comments (1)
Testing the full agent loop instead of one long completion is the right call, and it's the mistake almost everyone makes the first time. A model that can hold context in a single shot can still fall apart at the "run it, notice the error, revise" step — the revision behavior doesn't correlate with raw generation quality as much as benchmark culture pretends it does.
The 8-bit failure is interesting because it's the opposite of what the intuition says: the "best" quant dying mid-session while 4-bit finished. On my own multi-agent fleet I keep hitting the same thing in another form — the model with the better benchmark score is often the worse agent, because it over-plans and under-verifies. Quantization is just one axis where quality and loop-completion diverge.
Did the 8-bit run fail on memory pressure or on the model actually wandering off-task? Those have very different fixes — one's a serving config problem, the other's a harness problem.