What malloc Actually Does (It's Not Magic)

Every time your code calls malloc(128) and gets back a pointer, something genuinely complicated just happened silently on your behalf. You probably didn't notice. That's the point.

Dave from Dave's Garage recently walked through exactly what's happening behind that deceptively simple interface—building a memory allocator from scratch to show how quickly "simple" becomes "terrifying." It's the kind of explainer that makes you feel briefly bad about every line of C you've ever written. In the best way.

The interface is a lie (a useful one)

The pitch for malloc is almost offensively clean. As Dave puts it: "You say, 'I need 37 bytes.' And it hands you back an address. That's it. No ceremony, no explanation, just a pointer to what looks like fresh empty space."

What's actually behind that address is a memory allocator playing a very involved game of real estate management between your code and the operating system. The OS hands out memory in big chunks—typically 4KB pages—and has zero interest in being called every time you need 19 bytes for a string. So malloc sits in the middle: it requests big regions from the OS and carves them up for you.

The simplest version of this is a bump allocator, and watching Dave build one is genuinely funny because it's so small. You have a pointer into a big array. You return the current pointer. You move the pointer forward. Done. It's fast—"barely more than a shrug" on a modern CPU—and it's not actually useless. Compilers use arena allocators during parsing. Web servers allocate a whole arena per request and throw it away when the request completes. Some OS kernels use region-style allocation at boot because they don't have a full heap manager yet.

But here's what gets me about the bump allocator: you call free() on it and... nothing happens. Dave cheerfully describes adding a no-op free function as "reinventing the memory leak officially." What's wild is that even that has legitimate uses—if you're going to throw away the entire arena at once anyway, freeing individual objects is just overhead with extra landmine potential. The bump allocator isn't bad because it's simple. It's bad because once you've given out a pointer, it has absolutely no plan for what comes next.

The metadata you never see

The moment you need real free() behavior—objects dying in arbitrary order, memory getting reused—the bump allocator falls apart and you need something with memory. Specifically, the allocator needs to track which parts of the heap are used and which aren't.

Where does it store that tracking data? Right before the pointer it just gave you.

Every allocation you make actually reserves space for a hidden header: block size, whether it's free, pointer to the next block. The address malloc returns points past that header into your "usable" memory. To you it looks clean. But there's a luggage tag stapled to the front of every allocation that you were never supposed to notice.

This is the detail that made me do a full stop. Every pointer you've ever gotten from malloc is lying to you about where your memory actually starts. The real beginning is a few bytes back, full of bookkeeping you're not supposed to touch. When you pass a garbage pointer to free(), the allocator walks backward from it and tries to read those bytes as a valid header. Dave's comparison: "you don't want to go swimming after somebody has peed in the pool." And double-free—passing the same pointer to free() twice—can put the same block on the free list twice, which means two separate malloc calls might return the same address to two different parts of your program. Two owners, same house, both redecorating. Heap corruption is the result, and it's the kind of bug that makes you question your career choices at 2am.

Fragmentation: the garage full of Christmas decorations

Once you have a free list, you have fragmentation. Your heap might have 500KB free total, but scattered across 50 individual gaps, none of which is big enough for your 200KB allocation. Dave's analogy: enough floor space in your garage for a motorcycle, except it's split between paint cans, Christmas decorations, and whatever that weird cable was for.

Allocators fight this with two techniques: splitting (don't give someone a 1,000-byte block when they asked for 100—split it and keep the rest) and coalescing (when you free a block, check if its neighbors are also free and merge them). Coalescing requires knowing who the neighbors are, which means boundary tags, address-ordered lists, separate free lists—every fix pulling another piece of complexity into the allocator.

This is why, Dave notes, "real malloc implementations are not five lines long. The five-line version works right up until the time when reality touches it."

Threads made everything worse

Then multithreading happened. A single global heap lock means every thread queues up for the same traffic light in the middle of your freeway interchange. Modern allocators respond with per-thread caches and multiple arenas—a thread can service small allocations locally without contending with the rest of the process.

Except now memory can get stranded. One thread's cache is sitting on chunks another thread desperately needs. More arenas reduce lock contention but increase total memory footprint. The pattern here is consistent: every solution arrives dragging a little wagon of new problems.

Security: the metadata was always a target

For years, heap metadata was a favorite exploitation surface. Overflow a buffer, overwrite the allocator's bookkeeping, and you might be able to trick free() into writing attacker-controlled values to attacker-controlled locations. Dave puts it cleanly: that's the bug that turns "oops, I copied too many bytes" into "oops, someone else owns your process now."

Modern allocators throw a lot at this: cookies, encoded pointers, guard pages, delayed reuse, quarantine lists, randomized layouts, debug modes that fill freed memory with recognizable poison patterns. AddressSanitizer catches use-after-free and out-of-bounds writes by surrounding allocations with poisoned regions.

Dave mentions that back in the Windows NT days, there was a debug allocator that placed guard pages before and after every allocation—specifically, hardware guard pages per the transcript. (Worth noting: whether NT's debug allocator used hardware guard pages specifically versus software-based poisoning is a technical distinction Dave references from personal experience; readers who want to verify the implementation details should treat this as Dave's recollection.) As Dave describes it, it was a ton of overhead to run that build, but it shook a lot of bugs loose.

The cost of all that suspicion is real, which is why your release allocator and debug allocator can behave completely differently. Bugs that reproduce in debug builds sometimes vanish in release, and vice versa. The debug heap is a building inspector checking every nail. The release heap is trying to close the subdivision before lunch.

Your memory isn't actually gone when you free it

Here's something that trips people up constantly: calling free() doesn't necessarily return memory to the OS. It returns the block to the allocator's internal economy. Your process's memory footprint in Activity Monitor might stay completely flat even after you've freed a million objects, because the allocator is holding onto that memory in anticipation that you'll need it again.

Large allocations are more likely to go back to the OS because they often have their own dedicated page mappings. Small allocations tend to stay in the heap's private inventory. So "my process looks fat but I freed everything" isn't necessarily a leak—it might just be a well-stocked warehouse.

The trade-off landscape has no exits

What I find genuinely interesting about Dave's walkthrough is that there's no triumphant resolution. There's no "and then they figured it out." There's just an expanding map of trade-offs:

• Optimize for speed → use more memory
• Optimize for low footprint → spend more CPU on coalescing
• Optimize for thread scalability → duplicate caches, increase footprint
• Optimize for security → pay in time and space
• Optimize for deterministic timing (embedded/real-time) → maybe avoid general-purpose malloc entirely in hot paths

There are real allocators—jemalloc, tcmalloc, mimalloc, hardened mallocs—each tuned differently for different workloads. Windows shipped the Low Fragmentation Heap to address bucket-based fragmentation that was turning older heaps into Swiss cheese. None of them is The Answer. They're each a particular set of bets.

Dave's practical upshot: "the fastest malloc call is the one you do not make." Not "never allocate"—that's absurd—but allocation patterns matter. Allocating inside a hot loop without thinking about it is pouring sand in the gears. Vectors that grow one element at a time, strings copied three times because the abstraction diagram looked cleaner—these are places where the machine is trying to tell you something.

---

Malloc is, as Dave puts it, "one of the great enabling hacks in software history"—the thing that lets static programs adapt to a dynamic world at runtime, that lets data structures grow, that lets your code meet the world as it is rather than as you guessed it would be at compile time. What starts as a pointer into a big empty array accretes bounds checks, alignment, headers, free lists, splitting, coalescing, bins, lock strategies, thread caches, security hardening, and debug tooling until it's a tiny operating system running inside your process.

The engineering that works this hard to be invisible is the engineering we usually forget to think about. Until the weekend we spend staring at a use-after-free bug at 2am, anyway. Then we think about it a lot.

---

— Zara Chen, Tech & Politics Correspondent, Buzzrag