Artificial Intelligence

Bonsai 27B Runs on a Phone—the First 27B-Class Model Small Enough for an iPhone

PrismML compressed a 27B vision-language model to ~1.125 bits per weight, fitting it on an iPhone and running chat, vision, and tools locally.

A 27-billion-parameter model that fits in your pocket used to be a fantasy. PrismML’s Bonsai 27B changes that. The model is compressed so aggressively, to about 1.125 bits per weight in its 1-bit build, that it runs on a modern iPhone without memory offloading, while a normal 27B model would not even load on many laptops. It is, by the project’s own framing, the first 27B-class model to run on a phone.

The release, documented in PrismML’s Bonsai-demo repository and accompanying whitepapers, is the family’s largest generation and its first vision-language models. Bonsai 27B can take photos, screenshots, and PDFs and answer questions about them; call tools in the OpenAI style with full round-trips; reason through problems with adjustable effort; and hold 256,000-token conversations. The kicker is the footprint: the 1-bit Bonsai-27B packs to roughly 1.125 bits per weight and fits on a phone, while the higher-quality Ternary-Bonsai-27B, at about 1.7 bits per weight, is the demo’s default and runs accelerated on consumer hardware. The code and weights are published in PrismML’s Bonsai-demo repository.

For context on why that is unusual, a standard 27B model in 16-bit precision needs about 47.73 GiB just for weights. A typical “4-bit” quantized 27B still needs roughly 15.7 GiB. Bonsai-27B at 1-bit needs about 3.53 GiB of weights; the ternary default needs about 6.66 GiB. That is the difference between a model that demands a datacenter GPU and one that fits in the memory of a flagship phone.

How they compressed a 27B model that far

Bonsai does not rely on ordinary post-training quantization, the familiar “4-bit” trick that shaves a model down but hits quality cliffs. Instead it pushes to extreme low-bit representations: a 1-bit format where each weight carries almost no precision, and a ternary format that uses three states per weight. The 1-bit build is published as Q1_0 in llama.cpp’s GGUF format and has been fully merged upstream, so it runs out of the box on CPUs, Metal, CUDA, and Vulkan. The ternary build uses a Q2_0 format that now runs on mainline CPU and Metal, with Vulkan in review.

The engineering matters because extreme compression usually destroys a model’s ability to reason and follow instructions. PrismML pairs the low-bit weights with a hybrid attention architecture that keeps the key-value cache small for its size, which is why the 27B supports up to 262,144 tokens of context. The FP16 KV cache costs about 64 KiB per token, so 100,000 tokens of context fits on many consumer devices even without quantization; with the optional 4-bit KV cache, that drops to about 18 KiB per token, roughly 1.8 GiB at 100K context.

Memory tables in the repo lay out the trade. Bonsai-27B (1-bit) needs about 10.8 GiB at 100K context; Ternary-Bonsai-27B needs about 13.7 GiB, falling to about 9.2 GiB with the 4-bit KV cache. By contrast the 16-bit reference needs 55.2 GiB at 100K. A phone cannot hold 55 GiB; it can hold 3.5. That single fact is what makes on-device 27B plausible. The repo also supports speculative decoding, pairing the 27B with a drafter for roughly 1.8-2x faster decode on code and reasoning, and the 4-bit KV cache as an off-by-default option.

Why a phone-sized 27B actually matters

The most obvious use case is privacy and offline use. Because Bonsai runs locally through llama.cpp or MLX, nothing has to leave the device. There is no API call, no cloud bill, and no transcript of your prompts sitting on someone else’s server. That matters for the same reasons earlier on-device work did: Frontierbeat covered how Google’s Gemma 4 runs a full AI agent on your phone with no data leaving the device, and Bonsai pushes the parameter count an order of magnitude higher while still fitting in memory. Running large models on consumer hardware is a broader trend Frontierbeat has tracked as Apple’s foundation models shipped for on-device use.

Then there is capability. At 27B, this is not a toy chatbot. The model is a thinking model with adjustable reasoning effort, native tool calling over the OpenAI API shape, MCP server support, and vision. The demo wires it into Open WebUI so a phone or laptop can host a ChatGPT-like interface locally, complete with a code interpreter and agentic tools. For developers, that means a capable agent that lives entirely on hardware you own, which is a different trust posture from renting inference from a frontier lab.

The comparison against current models is stark. Most “runs on your phone” claims involve 1B- to 8B-class models, and even those often need offloading to slower storage. Bonsai’s 27B in 1-bit is in a different league: it brings reasoning, vision, and tool use to a device class that previously topped out at small models. Against a cloud 27B, it trades some quality for independence and cost; against a phone-sized 8B, it brings far more capacity to the same memory budget. PrismML also ships the family in 8B, 4B, and 1.7B sizes, and a 1-bit Bonsai-8B and smaller, so the same toolchain scales down to weaker devices.

Innovation here is less a single trick than a systems result: extreme low-bit weights, a cache-friendly architecture, and mainline-compatible inference engines combined so a 27B model becomes something a phone can hold. The project is open, with GGUF and MLX builds, prebuilt binaries across Mac, Linux, Windows, and even iOS, and a setup script that does the whole install in two commands. Whether the 1-bit build’s reasoning holds up under real tasks remains for the community to measure, but the engineering milestone is real: a 27B-class model that fits, and runs, on a phone.

FAQ

What is Bonsai 27B?

It is a vision-language language model from PrismML compressed to about 1.125 bits per weight in its 1-bit build, small enough to run on a modern iPhone without memory offloading. It supports chat, vision input, agentic tool calling, and 256k-token context, and runs locally via llama.cpp and MLX.

How small is it compared with a normal 27B model?

A standard 16-bit 27B needs about 47.7 GiB for weights; a typical 4-bit build needs about 15.7 GiB. The 1-bit Bonsai-27B needs about 3.5 GiB, and the ternary default about 6.7 GiB, a roughly 13x reduction versus 16-bit that puts it inside phone memory.

Does running it on a phone mean it is low quality?

The 1-bit build trades some accuracy for size, which is why PrismML ships the higher-quality ternary default. But the model still offers reasoning, vision, and tool use at a parameter count far above typical phone-sized models, and the quality under real tasks is something the community is now measuring.