DeepSeek V4.1 Flash: How 890 Bytes Per Token Reshaped Inference Costs

A developer who posted a 447-turn token log found the session cost $55.06 on GPT-6 Astra, with roughly two-thirds of that being the cache line alone. Run the identical log on DeepSeek V4.1 Flash and the bill comes to $0.36, according to the breakdown by Cloud Codes on the model's launch-day architecture.

That gap is the whole story. DeepSeek shipped V4.1 Flash on September 10, a 552-billion-parameter mixture-of-experts model under an MIT license, and the number it leads with isn't a benchmark score. It's bytes: 890 bytes of key-value cache per token, down from 389,120 bytes per token on DeepSeek's first model. Across four generations in 34 months, that's a 437-fold reduction. The video's framing is blunt and, on the economics, hard to argue with: "That isn't a discount. They rebuilt attention until the line item almost stopped existing."

The Cache, Explained Fast

When a model reads your prompt, it doesn't keep the words. It keeps what each layer computed about each word, the key-value (KV) cache. Think of it as notes on the desk that stop you from rereading the book. Those notes cost space to store and compute to write, so the cache line item hides two costs: processing the prompt and holding the bytes afterward.

DeepSeek attacked both, and the headline move sounds wrong the first time you hear it. The model's 40 layers are split into a bottom encoder (20 layers) and a top decoder (20 layers). Only the encoder reads your prompt. The decoder's global keys and values are projected once, straight off the encoder's final hidden state. One pass through half a network instead of a whole one. The technical report puts prefill complexity at O(N·L/2), as covered in MarkTechPost's launch writeup.

From there, the compression stacks up multiplicatively. Only four of the 40 layers (2, 8, 14, and 20) own a global cache at all; the other 36 borrow, either re-scoring shared keys with their own queries or just reusing the previous layer's short list. Across the encoder, every two tokens compress into one cache entry. And the surviving cache is stored in FP4, four bits per number. Add it up and a million-token context fits in about 890 megabytes. On DeepSeek's first model, the same context would have needed 389 gigabytes.

The Catch DeepSeek Wrote Itself

Every one of these mechanisms is an approximation, and DeepSeek's own limitations section says so. Two named failure modes: selection errors in the sparse attention, and approximate state reconstruction in what's called bounded replay (rebuilding the decoder's sliding windows only approximately, 20 times cheaper but not bit-identical). In the paper's own words, per the Cloud Codes walkthrough: the cache "depends on where the cache hit landed. Same prompt, different cache state, slightly different numbers." That's the lab's sentence, not a critic's.

The sparse indexer has a structural risk on top. Layer 20 scans the full context once and keeps a candidate pool of just over 16,000 positions. Every deeper indexing layer scores only that pool, never the full million tokens. So the pool is one layer's opinion: if a deeper layer needed an entry that didn't make layer 20's cut, nothing below can recover it. The bottleneck moved rather than vanished.

What the Scoreboard Actually Shows

The wins over Claude Opus 5 are narrow: a point and a half on Terminal-Bench 2.1, a fifth of a point on DeepSWE (74.2 vs 74.0). The losses are not narrow: 20.6 points behind on Terminal-Bench 4.0 and 19.5 behind on Humanity's Last Exam. DeepSeek's own report concedes that "parity on the averages does not imply that the model matches the frontier capabilities of leading closed-source systems."

There's also a cost no leaderboard prints. Artificial Analysis measured about 200 tokens per second the day after launch and flagged the model as very verbose, writing nearly twice the output of the median model it tests. Verbose output costs real money at scale.

The Swap Nobody Voted On

From September 14, requests to DeepSeek's flagship route into V4.1 Flash at Flash prices, with the flagship being phased out. If you tuned prompts against the old model, you weren't consulted. Simon Willison's comment on the Hacker News launch thread was about trust rather than quality: providers "should lean towards not swapping out models on their paying customers."

The counterweight came from the same thread: the weights are on Hugging Face under MIT license, so a team that needs a frozen model can freeze it indefinitely, an option nobody on a closed endpoint has. Prior complaints about this model family misbehaving in coding harnesses mostly traced to third-party gateways serving lower-quality quantizations. With open weights, the same model served by two hosts is two different products.

Where the Architecture Budget Goes

One more mechanism deserves attention: Engram. Two modules at layers 1 and 14 are lookup tables, not matrix multiplies. The model hashes recent token n-grams through 24 hash heads into tables of roughly 16 million rows each, totaling 196 billion parameters, more than a third of the model's weights, none of which multiply anything. A token touches only 24 rows, so memorization gets decoupled from per-token compute, and the tables live in about 200 GB of ordinary host RAM with background prefetching. We've followed this offload pattern before in our coverage of Engram's memory efficiency, and Alibaba shipped a Qwen flash model with 51 billion n-gram parameters in system memory just weeks earlier. Two labs, weeks apart, same move.

Anthropic and OpenAI spend theirs on capability because capability is the product. DeepSeek spent two generations on bytes per token because it sells cheap inference and then has to serve it. The trajectory continues what we documented in V4's 90% memory reduction and the economics behind million-token affordability.

Reading the Whole Board

The fairest reading: this model wins the access that costs money and loses the access that needs expert knowledge. If your agent rereads long context all day, the price-to-performance gap is enormous and the benchmark tie on agentic coding makes the case. If your hard problems are science-shaped, the 20-point Terminal-Bench 4.0 gap argues for paying Anthropic. Leaderboards rank by score and relegate price to a footnote, which is how a fifth of a point for a 76th of the money reads as a tie. DeepSeek's own paper has the sharpest line against that framing: "standard evaluation infrastructure becomes more susceptible to model gaming."

One falsifiable bet to close. DeepSeek says a bigger version of this architecture is coming. My call: it ships by the end of March 2027 and doesn't close that Terminal-Bench gap. The cache is down to 890 bytes a token and four generations of architecture got it there. The open question is whether a fifth compression still hides in attention, or whether the next one has to come from somewhere else entirely.

Yuki Okonkwo covers AI and machine learning for Buzzrag.