DeepSeek V4: How It Made Million-Token AI

There's a particular kind of AI release that doesn't announce itself with a flashy benchmark at the top of a leaderboard. It announces itself with a price tag — or more precisely, the absence of one. DeepSeek V4 is that kind of release, and once you understand what's actually inside the 58-page technical report, the pricing feels less like a promotional move and more like a logical conclusion.

The headline number: a 75% discount on API costs compared to V3.2, landing at $0.435 per million input tokens and $0.87 per million output tokens — with cache hits dropping to $0.3625. For context, Gemini 3.1 Pro runs $2 in and $12 out. Anthropic's Opus 4.6 sits at $5 in and $25 out. The bycloud breakdown of the V4 paper put it bluntly: "With the same budget, a company can use Deep Seek for 7 years and only 4 months if they use Claude."

DeepSeek has since made that discount permanent, which transforms a promotional event into a structural statement about how they think AI should be priced.

The Memory Problem Nobody Wanted to Solve

To understand why the pricing is achievable, you have to understand the problem DeepSeek actually went after: long-context inference is brutally expensive, and almost nobody in open-source AI had developed original attention techniques to address it at scale.

Here's the intuition. Standard transformer attention works by maintaining a KV cache — essentially, a memory bank that stores information about every token the model has seen so far. At 1 million tokens, that's 1 million memory slots the model has to loop through every time it wants to predict the next word. The math gets ugly fast.

DeepSeek's answer is two new attention mechanisms that work in tandem.

Compressed Sparse Attention (CSA) starts by acknowledging that most nearby tokens are probably saying related things — a few tokens often form one idea. So instead of storing every token as its own memory entry, CSA compresses every four tokens into a single learned KV entry. Crucially, this isn't average pooling (just blending everything together indiscriminately). The model actually learns compression weights: each token contributes differently to the compressed summary, and the model decides those proportions during training. Even after compression, a million-token context is still large, so CSA adds a lightweight "indexer" — a fast scanner that looks across the compressed memory and picks only the relevant blocks to actually attend to. Compression first, then sparse retrieval.

Heavily Compressed Attention (HCA) takes the opposite philosophical stance. Instead of surgical compression-then-selection, HCA just... compresses more aggressively. Every 128 tokens collapse into a single KV entry — 32 times lighter than CSA. The tradeoff is precision: HCA isn't trying to retrieve a specific token. It's giving the model a cheap, blurry panorama of the whole context. Because the compressed memory is small enough, the model can attend to it densely without breaking the compute budget.

Both mechanisms run alongside a sliding window attention layer that preserves the most recent 128 tokens at full resolution, because whatever you just read is usually what matters most. The result is a three-layer memory system: sharp and recent via sliding window, medium-range and precise via CSA, global and cheap via HCA. The 90% memory reduction this produces compared to V3.2 isn't a happy accident — it's the whole point.

At 1 million tokens, V4 Pro uses only 27% of the compute flops and 10% of the KV cache that V3.2 would have required. Against the Llama-style GQA baseline — the attention used in the models most people think of as "standard" — V4 Pro's KV cache is 34 times smaller. V4 Flash, the smaller model, achieves 49 times smaller. These aren't marginal optimizations. They're a different way of thinking about what a long-context model is allowed to do.

The Scale Underneath

The attention story gets most of the oxygen, but the model underneath those mechanisms is itself worth pausing on.

V4 Pro has 1.6 trillion total parameters — the largest open-source model ever released — with 49 billion active at any given time (it's a Mixture-of-Experts architecture, so most parameters sit idle while the relevant ones fire). V4 Flash comes in at 284 billion total and 13 billion active. Both support 1 million token contexts natively.

Pre-training ran on 33 trillion tokens for Pro and 32 trillion for Flash — double what any previous DeepSeek run used. The training process incorporated a newer optimizer called Muon (originally developed by Keller Jordan in late 2024, first validated at scale by Kimi), which DeepSeek says provided faster convergence and better stability compared to the AdamW optimizer most large models rely on. Given that V4 also introduced manifold constraint hyperconnections (MHC) — a mechanism that expands the model's internal "residual stream" from a single path into multiple mixed streams — the training stability challenge was real. More moving parts, more ways for things to go sideways.

The post-training setup is equally deliberate. Rather than running the full unified model through a giant reinforcement learning stage that mixes math, code, and general conversation objectives, DeepSeek trained specialist models for each domain separately, then used on-policy distillation to fold their capabilities back into the single final model. As the bycloud breakdown explains, "mixing RL targets and constantly switching goals can create conflicts for a model." Distillation sidesteps that conflict by treating the final model more like a student learning from multiple teachers simultaneously, rather than one student being graded on completely different subjects in the same exam.

Where It Actually Stands

V4 Pro Max — which is just V4 Pro running in maximum reasoning effort mode, not a separate architecture — ranks among the strongest open-weight models on third-party benchmarks, winning Terminal Bench Hard and consistently placing second or third in open-model rankings. It outperforms Gemini 3.1 Pro on academic long-context benchmarks, which is the specific category DeepSeek appears to have engineered most aggressively toward.

The honest part of the paper (and honestly, it's refreshing): V4 still trails GPT-5.4 and Gemini 3.1 Pro on general reasoning, with DeepSeek describing the gap as "roughly 3 to 6 months behind frontier closed models." For agent tasks, ProMax is roughly on par with Kimi K2.6 and GLM 5.1, not clearly ahead.

There's also a telling moment in the benchmarking section. When comparing against Kimi K2.6 and GLM 5.1, DeepSeek notes it left some benchmark entries blank because those models' APIs were too slow to return responses during evaluation. The bycloud breakdown flags this as an underrated data point: "Capability alone is definitely not enough. If the model is strong but too expensive, too slow or too hard to serve reliably, then it becomes much less useful in the real world." An unreachable model is just an impressive press release.

The Stack Below the Model

One thing easy to miss in any V4 coverage is how much of this is an infrastructure story as much as a model story. The entire system — attention compression, KV cache reduction, FP4 quantization-aware training for the MoE expert weights (trained to survive low precision during inference, not just quantized post-hoc), and day-zero support for Huawei chips given the Nvidia export restrictions in China — is co-designed around a single question: how do you make a million-token context window something a real application can actually afford to run?

The FP4 detail is particularly interesting. Most models get quantized after training as a compromise — you accept some degradation in exchange for cheaper inference. DeepSeek trained V4 to simulate FP4 precision during training itself, meaning the weights were optimized to work well under that constraint from the start. It's the difference between building a house to survive an earthquake and bolting on reinforcements after the fact.

The bycloud breakdown calls the paper "engineering system paper as much as a model paper," and that framing is accurate. Most AI technical reports lead with capability; this one reads like a report on bottleneck elimination.

What makes this complicated to fully evaluate is that DeepSeek isn't a neutral actor. They're a Chinese lab operating under hardware constraints that Western competitors don't face, which both incentivizes this kind of efficiency-first engineering and raises legitimate questions about the computing environment the model runs on and who controls it. The openness of the technical report is genuinely valuable; the full picture of what's running the inference at scale is harder to see.

Still, the question the paper surfaces doesn't go away: when efficiency improvements of this magnitude get published openly, what exactly are the closed-source labs charging for?

Yuki Okonkwo is Buzzrag's AI & Machine Learning Correspondent.