Meta Muse Glimmer 30B Runs in 14GB RAM via Unsloth

Every few months something lands in the local AI space that makes me do a genuine double-take—not the performed enthusiasm kind, but the kind where I actually stop and re-read the numbers because they don't look right. Muse Glimmer 30B running autonomously in 14 gigabytes of ordinary system RAM is that moment. And I'm holding two feelings about it simultaneously: genuinely thrilled that we're here, and clear-eyed enough to know this isn't the whole story.

Meta shipped Muse Glimmer from their Super Intelligence Labs on August 10th, Apache 2.0, open weights. It's a 30 billion parameter dense model—not a mixture of experts—with a 1.8 billion parameter perception encoder bolted on for image understanding. 52 layers, 131,000 token context window. Meta built it for a 24GB graphics card, which means it was already aimed at serious prosumer hardware, not a MacBook Air.

Then Unsloth happened.

The compression ladder nobody asked for (but absolutely needed)

The reason this model even fits in a conversation about everyday hardware is Unsloth's dynamic quantization work. Here's what the ladder looks like in raw numbers: 16-bit precision comes in at 55.7GB. Drop to 8-bit and you're at 29.6GB. 4-bit lands at 15.9GB. And at the bottom rung—2-bit—12.4GB on disk, loading into approximately 14GB of system RAM.

Two-bit quantization is normally the part of this conversation where people shake their heads. Compress every layer equally to 2 bits and you get looping gibberish—Unsloth's own writeup says other attempts at this compression level "either failed to load outright or produced looping gibberish." That's the baseline. That's what 2-bit used to mean.

Unsloth's fix is conceptually elegant: stop treating every layer as equally compressible. Their dynamic quantization inspects each layer individually and assigns precision based on how much structural weight that layer carries. The embeddings and the outermost attention blocks—the parts that hold the model's grammatical skeleton together—stay high-precision. The middle layers, which are doing more rote computation, get crushed down. Cloud Codes breaks this down as a "ladder" approach where the model's critical architecture is preserved while the expendable middle is sacrificed for memory.

Does it actually work? There's a published benchmark—though importantly, it was run on a DeepSeek build rather than Glimmer itself, so treat it as a proxy for Unsloth's method rather than a Glimmer-specific score. On the Aider Polyglot coding benchmark, full precision scores 71.6%. The 4-bit dynamic build drops to 69.7%. The 2-bit dynamic build lands at 65.8%. That's a 5.8 point fall from the top—real, measurable, not catastrophic.

The comparison that actually matters, though, is 2-bit against 2-bit. A competing quantization at the same approximate file size scored 56.6 on the same benchmark. Same bit budget. Nine points of gap. Purely from the decision about which layers to protect.

So the 2-bit tax isn't "model becomes useless." It's "model loses about six points on a coding benchmark compared to full precision, and nine points compared to dumber compression." Whether that's acceptable depends entirely on your alternative—which, for most people on most hardware, is "the model doesn't run at all."

One more thing worth flagging before you sprint to download it: the file size is not the memory requirement. The key-value cache grows with your context. The perception encoder needs its own slice. All of these sit beside the weights in RAM. In Cloud Codes' demo, the context counter sat at 1,400 out of 131,000 available tokens. That context window is enormous. The room you have to actually fill it is not.

What it actually does with 100 tool calls

Meta didn't build this as a chat model, and the architecture decision to prioritize tool calling shows up immediately in the benchmarks. On MCP Atlas—the benchmark for calling tools through a protocol—Glimmer scores 75.5. Gemma 4 gets 54.2. Qwen 3 gets 62.5. That's not a marginal lead; that's Glimmer operating in a different register.

The tradeoff is raw coding performance. On SWE-bench Verified, Qwen wins 77.2 to 76. On Terminal Bench, Qwen wins 60.7 to 51.7. These are Meta's own numbers, on Meta's own page. Cloud Codes puts it plainly: "Glimmer does not win on raw coding at this size. What it wins is tool calling. And those are two different jobs."

If your work is one clever function, run Qwen. If your work is 40 steps with a file system and a browser in the middle, that MCP Atlas gap is the one that matters.

The demo Cloud Codes runs illustrates exactly that distinction. Using Unsloth's desktop app with the UDQ_2K_XL build—the 2-bit one, the 14GB one—the task is deliberately unfriendly: find a real, reproducible bug in the Unsloth repository itself, with no private credentials, no paid API, no GPU. The model searches the issue tracker, assembles a shortlist, then does something that actually distinguishes an agent from a search box: it reasons about which bugs it can actually prove. Its own on-screen note during the run: "Better candidates look for a bug with code logic, not user interface because a UI bug cannot be reproduced from a terminal."

It catches itself mid-run when curl gets blocked. It notices the block, reasons around it via a search tool to pull the raw file instead. It settles on a version mismatch, writes a regression test with an explicit assertion, and states that the test must fail before the fix counts. That's a pull request writeup, not a summary. Over 100 tool calls, nothing leaving the machine.

As Cloud Codes describes it: "That recovery is the capability Meta put on the box happening at two-bit precision."

The speed problem (it's real, it's hardware-dependent)

Here's where the honest accounting gets uncomfortable. An independent hands-on test on an M4 Pro MacBook Pro with 24GB unified memory clocked text decoding at 10.13 tokens per second via llama.cpp. Call that roughly seven words a second—the pace you'd read out loud. Survivable for a chat model. For an agent that's assembling a huge prompt, running hidden reasoning, making multiple model calls, and stuffing tool outputs back into context, tasks in that test ran four to seven minutes each.

The speculative decoder is supposed to fix this—it drafts likely next tokens in parallel so the main model can skip ahead. On that Mac, it made things worse: 6.73 tokens per second with the drafter switched on, about 33% slower than running without it. The accept rate tells the story—only 211 of 648 drafts were kept, which means the drafter was mostly generating tokens that got thrown away, adding latency instead of cutting it.

Vision inference is the sharpest pain point. At 2.61 tokens per second, a UI screenshot took over four minutes to process.

To be fair about the scope: that independent test ran the 17GB quantization, not the 2-bit build. There's no published tokens-per-second figure for the 14GB build—Cloud Codes is explicit about this. Unsloth's claims are about tool-call capability and memory footprint, not throughput. Those are genuinely different claims.

The part that keeps me up at night

I cover this space. I watch local agents get more capable every few months. And there's a number in Cloud Codes' breakdown that I cannot be breezy about.

Meta's own benchmark table shows the base model's prompt injection attack success rate—meaning, roughly, how often a malicious instruction embedded in content the agent reads can redirect what it does. One in four attempts succeeds on the base model. One in four.

I want to be precise here: this figure comes from Meta's benchmarks for the base model, not specifically for the Unsloth 2-bit build. Whether compression changes that surface is an open question nobody has published an answer to. But even as a base-model figure, it's the kind of number that should make anyone pause before handing a local agent shell access to a real repository.

An agent that autonomously calls 100 tools, reads live web content, writes regression tests, and can generate pull requests is genuinely impressive. An agent doing all of that while occasionally being redirectable by content it encounters in the wild is a different kind of impressive. The architecture that makes this model useful—deep file system access, real tool calls, extended autonomous workflows—is exactly what makes a successful redirect consequential. You might not notice until the diff is already there.

Cloud Codes notes that fine-tuning on your own workflow data can help close behavioral gaps like this. Tool descriptions, correct function calls, permission handling—that's trainable. The 1-byte file copy bug the demo uncovered (each copied file missing its final newline, confirmed by SHA-256 comparison) is the same category of problem: discipline, not knowledge, and discipline responds to examples.

But "you can train it to behave better" and "you should run it unsandboxed on your production repo today" are not the same sentence.

Where this actually lands

If I have 12 to 14 gigabytes free and I want a coding agent that never touches an external API, this is the most capable option I've seen at this memory footprint. The local model compression story has been moving fast, and Muse Glimmer 2-bit is a genuine milestone in that arc. The benchmark losses from quantization are real and measurable but not fatal. The tool-calling lead over same-size competitors is real too.

What I'd actually do: run it in a sandboxed environment. Point it at a non-production repository. Watch what it does for a week before giving it access to anything that matters. The 100-tool-call autonomous bug fix is impressive enough to take seriously. The security surface is real enough to take seriously too. Both of those things are true, and running this model well means holding both at once.

The model is ready. My shell is not—not yet, not without walls around it.

---

Yuki Okonkwo is Buzzrag's AI & Machine Learning correspondent. She covers the people building the systems and the systems they're letting loose on the rest of us.