World’s most popular open source modelQwen3.8 27B, the base model of Bonsai 2 27B. Qwen/Qwen3.8-27B on Hugging Face compressed 9x to run on machines with 8 to 16 GB of memory

Bonsai 2 27B is a language model from Prism ML with ternary weights that are only 5.95 GB, ideal for running in systems with limited accelerator memory.

3D scene from the Ternary Bonsai 2 WebGPU Space.

Install

Until support for these files is upstreamed to llama.cpp, this page helps you get started with Bonsai locally. It builds on Prism ML's Bonsai-demo, the original setup for these models.

Run this in a terminal on an Apple Silicon Mac or on Linux. On Windows, use WSL 2.

curl -fsSL https://huggingface.co/spaces/osolmaz/bonsai-installer/resolve/main/install.sh | sh

The script installs the bonsai command in ~/.local/bin and runs it. The installer looks at the memory and the GPU of the machine, and then offers a few settings with the memory each one needs. You can take one or choose each setting yourself.

┌ Bonsai Installer 0.2.1
│
│ This machine: 16 GB memory · runs on Metal
│   Bonsai fits comfortably in 9.6 GB here and should not use more than 14.4 GB.
│
┃ How should Bonsai run?
┃ Each choice shows the memory it needs, including the context cache.
┃ > Recommended    PTQ1_0, 32K context, images on  needs about 8.7 GB
┃   Long context   PTQ1_0, 64K context, images on  needs about 10.7 GB, tight
┃   Two sessions   PTQ1_0, 16K context × 2 sessions, images on  needs about 8.8 GB
┃   Custom         choose each setting

The choices bonsai computes for a Mac with 16 GB of memory.

The first run then downloads 6 to 8 GB, checks every file, and starts the server. Once the model has loaded, the server answers at these addresses until you press Ctrl-C.

Chat in the browser: http://127.0.0.1:8080
OpenAI-compatible API: http://127.0.0.1:8080/v1, model bonsai-2-27b
Coding agent: localpi, opened in the browser for you

Later runs reuse the saved settings and start in a few seconds. The README lists every option, and these are the ones you are most likely to need.

Command	Effect
`bonsai setup`	Shows the choices again
`bonsai --ctx 16384`	Sets the context for each session, in tokens
`bonsai --yes`	Starts without questions, for scripts
`bonsai start`	Starts the server in the background and returns when it is ready
`bonsai check`	Sends one test request to the running server
`bonsai stop`	Stops a server started in another terminal

Or ask your coding agent

Paste this into Claude Code, Codex, or another coding agent to have it install Bonsai and start the server for you.

Install Bonsai 2 27B with Bonsai Installer and start a local server here.

Attention agent: read this file before running anything:
https://huggingface.co/spaces/osolmaz/bonsai-installer/raw/main/docs/AGENT_ENTRYPOINT.md

Follow it exactly. Tell me the download size before it starts, ask me only
for missing choices such as a longer context or a second session, and report
the endpoint and the result of `bonsai check`.

Memory use

The installer chooses the weight file and the context size from the memory it finds. To check those choices we loaded the model in six setups and measured how much memory the model process held after one short request. Five setups ran on a Mac and one ran in a Linux virtual machine with 8 GB of memory.

Linux CPU, PTQ1_0, 8K, text only 6.37 GiB
Mac, PTQ1_0, 8K, text only 6.49 GiB
Mac, PTQ1_0, 16K, text only 7.01 GiB
Mac, PTQ1_0, 32K, with vision 8.68 GiB
Mac, PQ2_0, 32K, with vision 9.83 GiB
Mac, PQ2_0, 64K, with vision 11.86 GiB

Weight file
Context cache, image encoder, and buffers

Resident memory of the model process, measured on 2026-09-24 with runtime prism-b10709-9a9394a. The Mac rows come from an Apple M5 Pro with 64 GB, and the Linux row from Ubuntu 24.04 in a Docker virtual machine with 7.7 GiB. The dashed lines mark 8 and 16 GiB.

On an 8 GB machine the installer uses PTQ1_0 with an 8K context and leaves out the image encoder. In the Linux virtual machine that setup took 6.37 of its 7.7 GiB, which is enough on a dedicated box and tight next to a browser. With 16 GB the installer adds the image encoder and a 32K context, and that took 8.68 GiB on the Mac.

Speed depends far more on the hardware than on the weight file. The Mac decoded about 25 tokens per second on its GPU, while the CPU build in the Linux machine managed 1.7, which is fine for batch jobs and slow for chat. Prism ML reports 91 tokens per second on an RTX 4090 with CUDA. We have not tested CUDA yet.

Installer steps

The installer downloads two things and does the rest on your machine. The server listens only on 127.0.0.1, so prompts and files never leave it.

Runtime

The Prism ML llama.cpp release from GitHub, built for Metal, CUDA 12.8, or CPU
Model

One weight file and the image encoder from Hugging Face at a pinned revision

Your machine

Choose

Settings that fit the memory of the machine, or your own
Check

Each file must match its SHA-256 hash before it is used
Start

llama-server starts in router mode with no model loaded
Load

POST /models/load loads the model, and the installer waits for loaded
Use

Chat in the browser, call the API, or start localpi

Router mode is what lets Pi and localpi load and unload models through the server. The installer writes a preset file for the chosen weight file, so every load uses the same context size and flags. When your Hugging Face cache already holds a file with the right hash, the installer hard-links it instead of downloading it again.

Limits

The installer runs on Apple Silicon Macs and on Linux, and on Windows through WSL 2. For AMD and Intel GPUs, use Prism ML's Bonsai-demo scripts.
On Linux the CPU build needs libgomp1, and the CUDA build needs the NVIDIA driver with the CUDA 12 runtime. The installer names any missing library before it starts.
Stock llama.cpp, Ollama, and LM Studio cannot load these weight files yet, which is why the installer uses the Prism ML build.
The model thinks before it answers, so give it a large output limit such as max_tokens: 16384. For shorter replies, send reasoning_effort: "medium".
Prism ML keeps a list of known issues with workarounds.

Credits

Onur Solmaz (Hugging Face) built Bonsai Installer and this page.
Joshua Lochner (Xenova, Hugging Face) made the 3D scene at the top of this page and the in-browser WebGPU demo.
Pasha Khosravi, Brian (@bri-prism), Sahin Lale, and the Prism ML team made Bonsai 2 27B, the Prism llama.cpp fork with its ternary kernels, and Bonsai-demo.

Ternary Bonsai 2 WebGPU Kernels

This Space runs the model in the browser on WebGPU. The 3D scene at the top of this page is adapted from it, without the model loading.

Open the Space

Bonsai 2 27B

Prism ML built the model from Qwen3.8-27B with ternary weights at 1.72 bits per weight, and published it under Apache 2.0.

Model card Prism ML

Prism ML llama.cpp

The llama.cpp fork that carries the ternary kernels. The installer pins release prism-b10735-842b188, the one Bonsai-demo uses.

Repository Upstream llama.cpp

Hugging Face

Hosts the model files and this page.

This Space

localpi

A build of the Pi coding agent for small local models, which finds the llama.cpp server on its own.

npm package

Pinned versions, hashes, and test notes