## Bonsai 27B: A 27‑billion‑parameter model on a phone

Bonsai 27B: A 27‑billion‑parameter model on a phone

Bonsai 27B is a new language model with 27 billion parameters.

It can run directly on a smartphone without cloud servers.

The model fits in the device’s memory and uses efficient quantization to stay fast.

This makes advanced AI usable offline, preserving privacy and reducing latency.

Developers can load the model with a few lines of code.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = prismml/bonsai-27b
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map=auto,          # runs on phone GPU/CPU
    torch_dtype=auto          # uses low‑bit quantization
)

output = model.generate(tokenizer.encode(Explain bonsai, return_tensors=pt))
print(tokenizer.decode(output[0]))

The snippet works on Android or iOS with a compatible PyTorch runtime.

Running a 27B model on a phone opens new possibilities.

Chatbots, translators, and coding assistants can work anywhere.

No internet connection is needed, and user data stays on the device.

The approach shows that large AI models are no longer limited to data‑center servers.

Source: Bonsai 27B – PrismML