Meta’s Muse Spark Open is pitched as America’s next top model: an American open-weights model at the frontier. On 2× rented B200s it scores about the same on Bob-Bench Wide as a 27B model that runs on one gaming GPU, and loses to Qwen 3.8 Flash Next. Then both models built a skateboarding game. Play them below.
Want a copy without ads, an account, or tracking? Grab the torrent (76.2 MB).
It includes a webseed, so it downloads even when nobody else is seeding. Please seed it if you can.
Magnet link
86 private software-engineering tasks, same harness for every configuration. Score is the mean task score. Full = tasks scored 10/10, zero = tasks scored 0, TO = timeouts. VRAM is peak usage in MiB.
| Configuration | Score | Full | Zero | TO | VRAM MiB | Context | Hardware |
|---|---|---|---|---|---|---|---|
| Qwen 3.8 Flash Next IQ3_XXS +MTP (Unbiased Strata) | 92.9% | 75 | 2 | 2 | 24,000 | 192K | RTX 4090 |
| Muse Spark Open 816B-A42B UD-IQ3_XXS | 90.8% | 71 | 3 | 0 | 333,246 | 128K | 2× B200 |
| Qwen 3.6 27B IQ4_XS | 90.6% | 68 | 2 | 4 | 19,132 | 128K | RTX 4090 |
| ThinkingCap 3.6 27B Q4_K_M +MTP | 89.3% | 62 | 3 | 4 | 22,038 | 192K | RTX 4090 |
Muse Spark Open ran with reasoning_effort: low, temperature 0.6,
top-k 20, top-p 0.95, min-p 0, max 16,384 output tokens per request. In the video run it did
569 tok/s prompt processing and 41 tok/s decode; Qwen 3.8 Flash Next does about 2,500 and 93.
Same prompt to both models in the pi harness, with a headless browser to test their own work. Published unedited. Click a game to give it the keyboard, or open it full page.
→ push · hold SPACE to charge an ollie · tap ←/→ in the air to flip · Z kickflip · R restart
←/→ steer · ↑/W accelerate · ↓/S brake · SPACE ollie · R restart
Build a polished, playable skateboard game in the current directory. You own this game directory only. Use HTML/CSS/JavaScript (Canvas or WebGL is fine), with no backend dependency. Make it fun: a controllable skater, acceleration/braking, jumping/ollies, ramps or obstacles, collision/bail and restart, a score or trick-combo system, and clear on-screen controls. Give it a distinctive visual style. Support keyboard controls and an usable smaller-screen layout. Do not just draw a static mockup. Read the LookingGlass SKILL.md and use that headless-browser skill to debug your implementation. Use your SKATE_PORT environment variable for the local HTTP server. Your BROWSE_STATE_FILE environment variable provides an isolated LookingGlass browser; preserve it in subprocesses. Start the server, play through the controls and a restart, inspect console errors, take screenshots, and use your read tool to actually inspect the screenshot images. Fix problems you observe and leave the playable server running. Save screenshots and a concise TESTING.md describing what you tested and any limitations in this directory. You have image input enabled; do not claim screenshots are unavailable without attempting to read one. The attached image is a vision check: first state its two shapes/colors/left-right positions and its exact printed text. Then implement the game; do not wait for further approval. Do not touch other projects, modify model serving, kill other sessions, or publish anything remotely. Do not run rm -rf. Deliver the local URL when ready.
The prompt came with a small vision-check image (a red square, a blue circle and the text “RAMP 47”). Both models described it correctly.
llama.cpp commit 335916ad3f0cab876a7a3cf16faaaaea28bf9a96, Unsloth
UD-IQ3_XXS (8 shards), everything in VRAM across two B200s, no spill into system RAM.
llama-server -m Muse-Spark-Open-816B-A42B-UD-IQ3_XXS-00001-of-00008.gguf \ -ngl 999 --split-mode layer --tensor-split 1,1 \ -c 131072 -ctk q8_0 -ctv q8_0 -fa on \ -np 1 -b 2048 -ub 512 -t 16 --fit off \ --host 127.0.0.1 --port 8080
From the final-cut captions. Names and technical terms have been corrected; timestamp links open the corresponding point on YouTube.
0:00Were you looking for America's next top model? Meta's Muse Spark might be for you. On this channel, we look at AI models that you can run on your own infrastructure, and typically that means the GPUs and the Apple Silicon Macs that you already have at home.
0:13Today it doesn't. We're going to look at something that was run on 2 rented B200 GPUs. Now, Meta's Spark Open is basically Lennie from Of Mice and Men. It's a giant at 842 billion parameters. It's mentally handicapped, getting about the same score as a model literally 30 times smaller than it on Bob-Bench.
0:34And if you've read the book recently, you'll know that Lennie absolutely adores small fluffy animals. And so does Spark. All right, the end of last year, we're like, all right, we are quite excited about the progress. We haven't seen the models yet.
0:48You sure this is gonna work?
0:51Now, Bob-Bench Wide is a specific comparable test set, okay, it measures a model configuration on the same 86 tasks on the same hardware, a single 4090. And let's take a look at a few specific dots on this plot. I know it's complicated, but even if you're a consumer of Meta's AI offerings, I know that you're smart enough to follow along.
1:13The highest score in Bob-Bench goes down to Qwen 3.8 Flash Next, running on a fork of Strata. On specifically, the IQ3_XXS quant. It got a 92.9% in almost 5 hours. This ran upstairs at my house. On a GPU that is relatively pricey at around $2,300 and that GPU has been increasing in value exponentially since I bought it.
1:38Another spot on the frontier here goes to Qwen 3.6 ThinkingCap, which is a 27-billion-parameter model coming in at just over 4 hours. It got an 89%, which is worse than Spark Open. Good job, Lennie. You beat up a model 30 times smaller than you by one point.
1:53So Spark Open. We set it up on stream today. According to the model card, it needs an incredible 700 gigabytes of VRAM to run. You could literally buy a house with that kind of hardware. Now, regardless, we rented 2 B200s, and we downloaded the IQ3_XXS quant.
2:13Now, if you're good at math, you'll remember that that's the same quant that we used earlier for 3.8 Flash Next, okay? Again, 3.8 Flash Next is like 4 or 5 times smaller than Lennie. Anyways, we fired this thing up on llama.cpp with low reasoning.
2:31The entire mess fits into VRAM this way on 2 B200s. You get 128K context with a Q8 KV cache. And this was rented for less than the starting average wage at In-N-Out Burger. I paid $16 per hour on RunPod for this run.
2:48So the llama commit is here. We ran the Unsloth IQ3_XXS quant, Q8 KV cache, like I said, 128K context, sampling from the model card. No spilling the system memory. We were getting 569 tokens per second prompt processing, and 41 tokens per second decode.
3:07So, again, let's compare that to Flash Next, all right? 2500 prompt processing, and 93 tokens per second. So it was much faster for Flash Next. And this configuration for Lennie used a staggering 324 gigabytes of VRAM.
3:25So moral of the story is, if you want to run on tremendously expensive hardware and get the same score as Qwen 27B, this could be America's next top model for that use case. Otherwise, I'm not too interested in it.
3:41There is a couple problems with comparing it this way, and one is that the hardware needed to run Lennie is just impossible to come by. And then the other is that we ran it on light reasoning or low reasoning as opposed to medium for Qwen 3.8 Flash Next, but the difference in score is phenomenal here.
4:01And because it's so expensive to test, and I can't run it at home, it's not likely for me to go back and revisit this with different sampling parameters or with different reasoning. It's just not impressing me enough to look like it's worth it, especially when it's 842 billion parameters, and, you know, hundreds of gigabytes to download a different quant.
4:26So, um, in any case, we'll settle this like men do with a skateboarding game. We'll start here with the version of the 2D skate game that was created by Qwen 3.8 Flash Next. Check it out. I'm not going to read the I'm not going to read the instructions.
4:45Okay, we'll see if that makes any difference. Here we go. Go off the ramp. It looks like we need the ollie. Okay, so it looks like the controls for ollie are. Yeah, so you have to kind of front load it with a spring, which is kind of cool.
5:08Ouch. Okay, don't hit the green box. I think I'm getting it. Can we clear this gap? No, you have to, you have to kind of race the jump. So it seems like the mechanic of this game and this version of it is all about how you ollie.
5:31So yeah, hold the charge space. You can tap left or right in midair to spin. Oh, and you can kickflip. Ooh, changes color after you kickflip. All right, we kickflip twice and got a big air with a bonus. All right, I think I'm I think I'm seeing how it works here.
6:01I'm going too fast. So if you hold If you hold D to get a bunch of speed, it's pretty difficult to get through the map. So it seems like slowing down and braking would really help. Can you just do a stationary kickflip? Slightly better at skateboarding than I am.
6:24So, I mean, this seems to be reasonably, whoa, front flip. So you'll notice that the character did a backflip, but then got points for a front flip. So Qwen 3.8 Flash Next. Should have figured that out. Let's try that again.
6:42Okay, so that was a double backflip, and then we fell into the gap. So if you do a backflip, you get points for doing a front flip. All right. That's enough. That's enough. So that's how Qwen 3.8 Flash does it.
6:58And let's see, Neon Grind. So this is implemented by Meta's Muse Spark Open. You guys think it's going to be good? Whoa, bailed. We got a 112.97-66-7876. There's our guy. Um, Oh, whoa, you can really get a lot of velocity on those turns.
7:31Holy cow. If you get into the air, you can really spin and do a front flip like 70 times. That's crazy. Dude, this is way worse than I thought it was going to be. Okay, so W and S allow you to accelerate. And then as soon as you press space to ollie.
7:48You are doing a flip. Okay, this is terrible. Holy crap. This is way worse than I thought it would do. The character has like no definition. It's just like a air hockey puck. And by the way guys, this is the first time I'm seeing this.
8:12Not at all in the same league as this game, which has some had some gameplay, had sort of a loop to it, had some mechanics, The scoring seemed pretty reasonable. The front flip was not very reasonable. So that's actually a backflip.
8:32But, you know, I can't really complain too much. And also, the control scheme is just a lot more a lot more sane and comfortable here in terms of using A and D to go left and right as opposed to W and S. And we'll decide who the real winner is based on the results of these 2D skateboarding games.
8:53In conclusion, I'm not going to spend any more time on this model until I get 2 B200s at home. What I've seen isn't promising. It's possible that in higher reasoning modes, it could act differently, but it's not a very verbose model.
9:06It gives up really early, even when it's successful and completes the task fast. It does it with very few tokens used. So even though it ran slowly on these 2 behemoth GPUs. Not that interesting. All right, guys, thank you for listening.
9:21Appreciate you watching the short video on Lennie, and I will see you in the next one.