How a 26B AI Model Now Runs in 2GB of RAM on a Mac

Apple published a research paper in 2023 called LLM in a Flash that proved, mathematically, that a Mac could run an AI model twice the size of its available RAM. The trick: leave most of the model parked on the SSD and only pull the pieces you need into memory on the fly. The math worked. The paper was public. The hardware was already sitting in people's backpacks.

And then Apple shipped nothing.

Three years later, a 26-billion-parameter model that fills 15GB on disk is running on a standard MacBook using roughly 2GB of active memory — a sevenfold reduction. The Stack's breakdown of how this happened is worth sitting with, because the story isn't really about one clever trick. It's about two completely separate developments colliding in a way nobody quite planned.

Step one: the windowing trick

The core idea from LLM in a Flash is elegant. When you run inference (generate text), you don't need the entire model in memory at once. You need the specific slice that handles the next token. So: load that slice, run the math, then drop it immediately. Your RAM becomes a revolving door instead of a parking lot.

The problem Apple didn't fully solve in 2023 was speed. As The Stack puts it, "if you just park a model on your drive and try to read the weights you need into memory for every single word it predicts, the physical drive simply can't push data to the processor fast enough." Simply fitting the model on the drive was step one. Making it readable fast enough to be usable was the actual hard part.

That's where an open-source project called Turbo Fieldfare comes in. Written specifically for Apple's chips and Metal graphics layer, it finally built what Apple's paper described. The result: a 26B model running locally on a standard MacBook at usable speeds. But here's the arithmetic problem — Apple's paper promised a 2x improvement. Turbo Fieldfare delivers 7x. So where did the other five come from?

Step two: models learned to sleep

The missing multiplier didn't come from better engineering of the drive-streaming trick. It came from the models themselves changing shape.

Newer architectures like Gemma 4 use something called Mixture of Experts (MoE) — the neural network isn't one solid block of math anymore, it's split into specialist sections. A router looks at your prompt and picks only a handful of those sections for each word it generates. According to Turbo Fieldfare's system design documentation, the model activates just 8 of its 128 expert sections at a time, meaning roughly 85% of the network is idle at any given moment.

An idle section is exactly what you can afford to leave parked on the drive. The Stack frames this cleanly: "the missing multiple belongs entirely to the model's learning how to sleep on the job."

Apple's hardware trick was sitting there waiting. The open-source model architecture finally gave it something worth doing. This is the collision — two independent timelines (Apple's 2023 paper, and the post-2023 MoE architecture wave) intersecting in a way that neither team designed for.

Why this only works on a Mac

Here's where the hardware story gets specific. On a standard PC, the CPU and GPU have separate memory pools. When AI is running on the GPU and needs to pull a fresh model fragment off the drive, that data has to be physically copied into the GPU's own memory. The Stack calls this "a toll booth," and if you're yanking hundreds of tiny fragments per second, the traffic backs up fast.

Apple Silicon sidesteps this entirely with unified memory — the CPU and GPU share the same RAM pool. No copy step. The moment Turbo Fieldfare pulls a fragment off the SSD, the GPU can read it right where it sits. That zero-copy handoff is what makes the juggling act survivable.

This is also why oMLX's performance gains on Apple Silicon aren't incidental — the unified memory architecture creates a fundamentally different performance ceiling than what x86 machines can reach, and tools built specifically for that substrate can exploit it in ways that general-purpose runners can't.

The bottleneck moved, it didn't disappear

Here's the part that gets glossed over in the headline number: the SSD is now the active engine driving inference, not just a storage shelf. Every token generated requires fresh reads from the drive. Memory bandwidth — not GPU core count, not Neural Engine TOPS — becomes the dominant performance variable.

2026 LLM inference benchmarks published by VRLATech confirm what The Stack's breakdown argues: memory bandwidth is the single strongest predictor of how fast a machine generates text when the model lives on the drive. That number is printed right on Apple's product pages and spans nearly an order of magnitude from base-model chips up to something like the M5 Max. It's also, reliably, the spec buyers skip past on their way to checking total RAM.

The drive speed variability compounds this in a genuinely weird way. Tom's Hardware reported that some MacBook Air M3 units at the 256GB storage tier shipped with an extra NAND chip on the logic board, allowing parallel reads that made those units drastically faster than predecessors at the same nominal capacity. Two laptops, same model, same price, different SSD performance — and with this workflow, that difference matters in a way it never did before.

Then there's thermals. The fanless MacBook Air is the machine most people shopping for local AI are considering. The Stack is blunt about the gap: "a short test and a long session are two completely different measurements." Burst benchmarks look great. Sustained inference over an hour triggers thermal throttling, and the chip slows itself down to avoid cooking the internals. The machine you most likely want to run this on is the one least capable of holding its headline speed.

The 7x has fine print

The 7x figure is real, but it's measured against a specific baseline: loading the full Q4KM-quantized model (the compressed, 4-bit version) into RAM all at once. Change that baseline to the model's full BF16 precision — what you'd need without quantization — and the gap becomes so wide it's almost meaningless as a comparison.

Also worth being clear about: your SSD still holds the entire model. The 15GB file isn't going anywhere. Active RAM usage drops to ~2GB, but disk footprint stays fixed. The savings are in what your machine needs to hold in working memory at once, not in what it needs to store.

And critically, this trick only works on architectures where most of the network can stay idle. Point Turbo Fieldfare at a dense model like Qwen3.6-27B — where every part of the network fires for every token — and the efficiency vanishes. Testing published by AI Productivity found that Qwen3.6-27B requires around 17GB even at Q4KM compression, ballooning to roughly 56GB at full BF16 precision. In a dense architecture, nothing is ever idle, so nothing is safe to leave on the drive. TurboQuant-style compression can shrink what you load, but it can't manufacture the sparsity that MoE provides natively.

The memory saving is a property of how the model was built, not a setting you can toggle on.

Two timelines, one lucky collision

What The Stack's breakdown ultimately surfaces is that this efficiency gain isn't the product of a single breakthrough — it's a coincidence of timing. Apple proved the drive-streaming math in 2023, when models were still solid blocks that had to be fully active. The MoE wave arrived afterward, creating architectures with massive idle sections. Turbo Fieldfare finally connected the two.

"The memory saving belongs entirely to how the model was constructed," The Stack concludes, "while the speed it runs at relies on Apple's unified memory — which is the part you still have to buy."

That's the honest accounting. MoE models that sleep on the job travel with you regardless of your hardware. The zero-copy handoff that makes the streaming fast enough to be usable is Apple's contribution, and it's baked into the silicon.

Which raises the obvious question hanging over all of this: what happens when the next generation of frontier models goes back to being a dense brick of math where nothing sleeps? The 7x goes away. The disk-streaming trick still works, but you're back to Apple's original 2x. Whether MoE remains the dominant architecture or gets superseded by something denser is a question the model developers — not the hardware engineers — will answer.

---

Yuki Okonkwo is Buzzrag's AI & Machine Learning correspondent.