DEV Community

lbobylev
lbobylev

Posted on

Bonsai 27B 2-bit on a MacBook M1

#ai #llm #programming

Bonsai 27B looks like an attractive model for local use: 27B parameters, 2-bit quantization, and an 8.5 GB download. On paper, it offers a large model that can fit on a regular MacBook Pro M1 with 16 GB of RAM.
I tested how it works in practice. I compared Bonsai with Qwen3 14B, GPT-OSS 20B, and Gemma 4 12B in LM Studio. The tests were simple but practical: exact answers, JSON, code, logic, tool calls, and one long task with calculations.

Models

Name Model Parameters Format Quantization / size
Bonsai prism-ml/bonsai-27b 27B MLX 2-bit, 8.5 GB
Qwen3 qwen/qwen3-14b 14B MLX 4-bit, 8.3 GB
GPT-OSS openai/gpt-oss-20b 20B GGUF / MLX 12.10 GB
Gemma google/gemma-4-12b 12B MLX 4-bit, 6.8 GB

Method

I ran the tests with Promptfoo. The first nine tests cover math, code generation, factual knowledge, JSON output, instruction following, logic, avoiding made-up facts, and tool calling. The tenth test is a fixed task with data filtering, several calculations, and a strict JSON response.

Speed is based on the combined time of the two runs. Eval tokens/s includes evaluation tokens only, not tokens used by the LLM graders.

Test Set

The test set has ten checks. The first nine test basic skills and output format. The last one tests whether a model can follow a long, fixed set of rules.

  1. Basic arithmetic reasoning tests rounding down: Peter has 17 apples and gives away one third. The expected answer is 12.
  2. Python coding asks for only a gt2(n) Python function that checks n > 2.
  3. General knowledge checks the exact answer for the capital of Australia: canberra.
  4. Strict JSON output requires valid JSON without Markdown, with status: "ok" and value: 42.
  5. Exact bullet count asks for an explanation of a transformer in exactly three Markdown bullet points. It must mention windings, magnetic induction, and voltage.
  6. Logical reasoning checks a simple conclusion about a digital sensor. The answer must be exactly yes.
  7. Hallucination resistance checks that the model does not invent a publication date for a fictional paper.
  8. Technical explanation asks for a correct explanation of concurrency and parallelism in no more than 80 words.
  9. Tool calling requires a get_weather call for Rome or Roma with unit: "celsius". The test checks the tool call, function name, and arguments.
  10. Deterministic rule-following benchmark checks filtering, calculations of net, total, penalties, and score, selecting scores from 400 to 500, calculating a checksum, and returning valid JSON. The expected records are E, A, H, and G, with scores 488, 471, 471, and 447, and checksum 1877.

The last test checks the records and checksum, but not the requested order. Both lists are sorted before comparison.

Results

Model Passed Failed tests Time Tests/min Eval tokens/s
Bonsai 27B 9/10 Deterministic benchmark 9m 52s 1.01 16.3
Qwen3 14B 9/10 Deterministic benchmark 5m 14s 1.91 19.8
GPT-OSS 20B 8/10 Tool calling, deterministic benchmark 2m 56s 3.41 26.5
Gemma 4 12B 7/10 Math, tool calling, deterministic benchmark 48s 12.50 44.2

Bonsai and Qwen passed all nine basic tests. On the hard benchmark, Bonsai returned an empty list. Qwen returned the right records but made a one-point error in one calculation and in the checksum. GPT-OSS failed tool calling and produced invalid text for the benchmark. Gemma made mistakes in math, tool calling, and the benchmark calculations.

Extra Test: Python Instead of Manual Calculations

I changed the hard task: instead of returning the calculated JSON result, the model had to write a Python program to do the calculation. This run used no thinking mode. Speed, token count, and time below are shown exactly as reported by LM Studio.

Model Result LM Studio speed Tokens LM Studio time
Bonsai 27B Passed 10.52 tok/sec 643 15.12s
Qwen3 14B Passed 13.10 tok/sec 634 13.92s
Gemma 4 12B Passed 13.90 tok/sec 693 7.31s
GPT-OSS 20B Failed - - -

In this version, Bonsai, Qwen3, and Gemma solved the task, while GPT-OSS failed again. For hard calculations, it is more practical to ask a model to write code that can be checked instead of doing all the math in its answer.

Conclusion

Bonsai 27B 2-bit is competitive on short tasks. It scored 9/10 and handled math, code, structured output, logic, avoiding hallucinations, and tool calls. This shows that a low-bit 27B model can still be useful on devices with limited memory.
However, Bonsai was not the most practical choice in this comparison. With nearly the same file size, Qwen3 14B 4-bit was about 1.9 times faster and came much closer to the correct answer on the long calculation task. Bonsai is still an interesting compact 27B model for chat, short reasoning, and code. For tasks with strict calculations, rules, and output format, Qwen3 14B 4-bit looks more reliable.

Links

Top comments (0)