Bonsai 2 is a new “model” out of PrismML. It is not a new model — it is a new ternary quantization method applied to an existing one, Qwen 3.8 27B. Here I compare four different ways to use an 8 GB card on Bob-Bench Wide Alpha, and ask where Bonsai 2 is actually the right thing to deploy.
Want a copy without ads, an account, or tracking? Grab the torrent (51.4 MB).
It carries a webseed,
so it downloads even when nobody else is online — and every person who
seeds makes it cheaper for me to keep serving. Please seed it if you can.
magnet link
The comparison this video is actually about — four configurations that all target the same commodity 8 GB GPU:
Every row is the same 86 Bob-Bench Wide Alpha tasks on a 4090 — same harness, same task definitions, same grading, same timeout. The only things that change are the model, the quant, and the memory budget it was held to.
| Budget | Model | Score | Wall clock |
|---|---|---|---|
| Full 4090, 256K ctx | Ternary Bonsai 2 27B | 74.5% | 13 h |
| Full 4090 | Qwen 3.6 35B-A3B IQ4 | 90.6% | 5.4 h |
| Strict 8 GB VRAM, no spare RAM | Ternary Bonsai 2 27B | 71.2% | 11 h |
| Strict 8 GB VRAM, no spare RAM | Qwen 3.5 9B Q4, 128K ctx | 66.1% | ~11 h |
| 8 GB VRAM + ~10–12 GB system RAM | Qwen 3.6 35B-A3B | 80.6% | — |
Bonsai 2 wins exactly one row — strict 8 GB VRAM with no system RAM to spare — and it wins it by about 5 points over the most boring vanilla setup imaginable. Give the same machine 10 GB of spare system RAM and the 35B-A3B takes it back by 9.
The whitepaper’s headline is that Ternary Bonsai 2 27B averages 83.9 across a 20-benchmark suite, retaining 98.2% of the Qwen3.8-27B FP16 baseline’s 85.4, at 5.93 GB instead of 53.8 GB. Those are their numbers on their suite, and I did not measure them. What I measured is different and narrower: agentic coding, one harness, wall clock included. On that, against a Q4 quant of essentially the same model, it is significantly lobotomized — 74.5% versus 90.6%, and it took more than twice as long to get there.
Both things can be true. A 98% aggregate can coexist with a large drop on long-horizon tool-driven work, because the aggregate is dominated by benchmarks that compress well. The whitepaper is fairly direct about this too: on Terminal-Bench 2.1 and SWE-bench Verified it reports 52.8 and 60.8 against 69.7 and 80.6 for Qwen3.8-27B — roughly three quarters of full-precision performance, not 98%.
Four execution environments, and the question is which one this is for.
If you want a fast, genuinely intelligent coding setup on the cheap: buy a 1070 Ti. 8 GB of VRAM, about $70–75 plus shipping. Drop it in whatever computer you have lying around — the power budget is unlikely to trouble a supply you already own — and run the 35B-A3B on it. It will blow Bonsai 2 out of the water, for less than a tank of gas.
Two objections, and the second is the one I actually care about. The hype has run well ahead of the capability. And Bonsai 2 is not a new model — it is a quantization method for Qwen 3.8 27B. The ternary work is genuinely clever and those folks are smarter than I am, but shipping someone else’s model under your own name is like buying a car, painting your name on the side, and calling it the Bob Sedan. Same complaint applies to Ornith.
Bonsai 2 needs PrismML’s fork of llama.cpp — it will not run on upstream. Build it first:
git clone https://github.com/PrismML-Eng/llama.cpp prism-llama.cpp cd prism-llama.cpp && git checkout b10685 cmake -B build -DGGML_CUDA=ON && cmake --build build -j --config Release # binary lands at build/bin/llama-server
Fits on the card by itself — 48K context, Q4 KV cache.
/path/to/prism-llama.cpp/build/bin/llama-server \ -hf prism-ml/Ternary-Bonsai-2-27B-gguf:Ternary-Bonsai-2-27B-PTQ1_0.gguf \ --host 0.0.0.0 --port 8080 \ -ngl 99 \ -fa on \ -c 49152 \ --cache-type-k q4_0 --cache-type-v q4_0 \ -ub 2048 \ --parallel 1 \ --cache-ram 0 \ --no-mmap \ --jinja \ --temp 0.7 --top-k 20 --top-p 0.8 --min-p 0.0
The 256K-context, Q8 KV cache configuration — the 74.5% row above.
/path/to/prism-llama.cpp/build/bin/llama-server \ -hf prism-ml/Ternary-Bonsai-2-27B-gguf:Ternary-Bonsai-2-27B-PTQ1_0.gguf \ --host 0.0.0.0 --port 8080 \ -ngl 99 \ -fa on \ -c 262144 \ --cache-type-k q8_0 --cache-type-v q8_0 \ -ub 2048 \ --parallel 1 \ --cache-ram 0 \ --no-mmap \ --jinja \ --temp 0.7 --top-k 20 --top-p 0.8 --min-p 0.0
--n-cpu-moe is what makes the 35B-A3B fit in 8 GB of VRAM.Auto-captions, lightly corrected for proper nouns. Timestamps jump to that point on YouTube.
0:00Bonsai 2 is a new model from PrismML. Oh, sorry, no, it's a Qwen 3.8 quantization method. I got the marketing and reality confused again. Bonsai 2 is an impressive ternary quantization
0:11method, but it's not impressing me with my testing so far as a choice as a model for a software engineering agent. I've done an apples to apples comparison of Qwen 3.8 bonsai versus other models of the same RAM and VRAM budgets.
0:24I ran it through my benchmark 3 times, once at Bonsai Q1, which is awful, the 1st version of Bonsai.
0:31But first, a little bit about Bob Bench and my evaluation framework, each point on the wall clock versus benchmark graph is 86 tasks on a 4090 with the same harness, the same task definitions, the same grading, same timeout.
0:45I try to keep as much, the same as I can to get a true apples to apples comparison.
0:50Bonsai 2 ran at 2 different memory budgets. One used a full 4090 card, a full 256 K context, and a higher quant on the KV cache.
1:01So it consumed tons of VRAM. This is a useful comparison against the fuller model quant. So I have the apples to apples Q4 comparison versus Qwen 3.6.
1:11And again, this is the previous Qwen, still a valuable comparison.
1:14And Bonsai 2 gets a 75% in 13 hours. And Qwen 3.6 got a 90 in way less time. So this shows us that Bonsai 2, when you compare it to a Q4 model quant
1:29of basically the same model is significantly lobotomized. Okay, this is not surprising, but most people don't seem to get this.
1:35There's a 98% number that keeps getting thrown around where people are saying Bonsai 2 retains 98% of the intelligence of the full model quant. I didn't measure that.
1:47So maybe it's in the 8 gigabyte VRAM only range where Bonsai 2 is good. You have no system RAM to spare. So in this setup, Bonsai did do a little bit worse than
1:59my full VRAM setup for it.
2:02It got a 71% in 11 hours, and it beat out Qwen 3.5 9B by 4 points in roughly the same amount of time, golf clap.
2:13Where it gets really ugly for Bonsai 2 is if you have a little bit of extra system RAM to sprinkle on top. All right, so you're not just VRAM constrained, but you have a system where you can run the model with 8 gigs of VRAM,
2:26and then you also have maybe 10 or 12 gigabytes of system RAM to use for inference, all right? And it gets utterly dominated by Qwen 3.6, 35B A3B.
2:37It's much faster, and it's much more intelligent because you can run it at a larger model quant. So if you have some extra system RAM to spare, switch to the Qwen 35B and you get
2:5010 extra points on Bob Bench.
2:52So let's be clear about where it is good. Within the strict 8 gigabyte VRAM and no system RAM available regime, Bonsai 2 is the strongest model I have tested fully on Bob
3:04Bench wide alpha, but it's narrowly beating out a reasonable contender. Boring ass Qwen 3.5 9B at a healthy Q4 model Quant and 128 context.
3:15It's like the most boring vanilla 8 gigabyte setup you can imagine. And. And Bonsai 2 is just narrowly beating out in a very similar amount of time.
3:26So when would you deploy this? Let's consider 4 different execution environments. One is a severely constrained resource like a phone. Two is an extremely old gaming PC,
3:38and you've got a GPU in it, but you're also running Windows on it, and it's your main daily driver. And so you don't have a lot of system RAM available. Third is a dedicated AI rig with a crappy graphics card.
3:50So it's a system with 16 or 32 gigabytes of system memory, and you've got like a 1070 TI or a 2080 in it. And it's like a very budget solution for running AI
4:02at home, but it is dedicated to the task. And then the 4th class and the final execution environment here is a really high-end setup where you have maybe a 5090 or a dual GPU kind of card and you can run whatever you want on it.
4:15And clearly on this last 4th option, we don't want Bonsai 2, because you can just run a larger model quant and get way better performance faster. So that's not what it's for.
4:27Is it three? And here, the correct model to deploy is extremely cut and dried. It's a Qwen 3.6, 35B A3B variant for sure.
4:37either an obliterated one. If you need to do reverse engineering or red team work. Um, and no matter what, the 27B Bonsai 2 is going to be utterly dominated by Qwen 35B
4:49on this, like, low end dedicated rig.
4:52Now, it gets a little bit murky on our 2nd configuration. Because this PC, you know, it's doing other stuff and maybe we don't have that much RAM available and the right model to deploy here.
5:05If you want to use it to drive your coding sessions is going to depend on the amount of system RAM that you have available. Okay? And if the answer is zero, then, you
5:16know, maybe Bonsai 2 is your choice. Uh, but it's going to be really close compared to the Qwen 3.5 9B. So, I don't know.
5:29It's like a maybe. Like maybe Bonsai 2 is a useful option to consider there, but probably not, because if you have 10 gigs of RAM on that PC.
5:38And a lot of setups do, you're going to have better options.
5:42So it leaves us at one. It leaves us with the phone. And I'm left with the question, why do I want my edge compute driving the
5:53model that I'm using for my hardest coding problems? It doesn't make any sense. So I don't get why we want Bonsai 2? Because any situation that I care about for coding,
6:07any kind of reasonable environment that I could recommend to somebody, it's not the right choice. So maybe it's for chatbots that run on your phone. It's certainly not a good model for coding.
6:19If you want a cheap coding setup. All right, a really intelligent, a really fast setup, buy a 1070 TI, it's got 8 gigabytes of VRAM, and the thing costs like $70.
6:33I saw one for 75 bucks today, plus shipping, throw it in any computer that you have lying around. The power budget is probably not going to be too much of a constraint with the power supply that you already have, and get the
6:4535 BA3B on there. It'll blow Bonsai 2 out of the water. And this setup is going to cost you less than a fucking tank of gas.
6:54So, I'm really not a fan of Bonsai 2 for a couple reasons. One is that the hype for the model is getting so far ahead of the capabilities. And the 2nd is that I don't think that
7:06people should go off and make a minor change. Or in their case, this is a major change. It's a very interesting quantization method. This ternary thing.
7:13It's super cool. Those guys are way smarter than I am And I am impressed with what they've done, but they have not created a new model.
7:21Bonsai 2 is not a new model. It's a quantization method for Qwen 3.8 27B. And I take issue with that. And I think it's more or less the same thing with Ornith.
7:33Uh, it's like, you know, buying a car and then painting your name on the side and then being like, this is the, you know, the Bob sedan. It's like no. Anyways, guys, there's a discord, join us there,
7:47and thank you for watching, subscribe if you would like to, and I'll see you in the next video.