Prompt Caching: The Reason Claude Code Doesn't

If you've ever hit Claude Code's session limit mid-flow and wondered where all your tokens went, there's a good chance you never touched the thing that was supposed to save them. Prompt caching runs quietly in the background—no toggle, no setup required—and according to AI automation creator Nate Herk, it can reduce your effective token spend by up to 90%. Three hundred million tokens saved in a single week, he says, without changing a thing.

That number is hard to internalize. So let's actually look at what's happening under the hood, because the mechanics here are both simpler and more fragile than most people realize.

What caching actually is (and what it costs to break it)

Prompt caching works on a layered system. When you start a Claude Code session, there are three tiers of context being managed simultaneously: a system layer (base instructions, tool definitions, output style), a project layer (your Claude.md file, project-specific memory and rules), and a conversation layer (the actual back-and-forth messages). The system and project layers get cached globally or per-project. The conversation layer gets re-cached every turn—but that's expected and cheap.

The key number: cached tokens cost 10% of what fresh input tokens cost. So on a session where you've got 91 million tokens in context, you're effectively paying for 9 million. That's the math Herk is working with when he talks about his daily savings.

But here's where it gets interesting. That cache only lives for as long as its TTL—time to live. On a Claude subscription, you get a one-hour window. Step away from your session for 60 minutes, come back, and everything re-caches from scratch. If you're deep into a long session with a lot of context built up, that's not just inconvenient—it's expensive. Imagine being 16 messages deep into a complex coding project and triggering a full re-cache. Every token in that conversation suddenly needs to be re-processed at full price.

This is exactly the kind of thing that explains why [session costs](/ article/why-your-claude-code-sessions-cost-more-than-they-should) feel disproportionate to what you're actually doing in a session.

The TTL confusion that had everyone suspicious

There's a wrinkle here that caused real community drama a few months back. When Claude users started burning through their subscription limits faster than expected, speculation spread that Anthropic had quietly changed the cache TTL from one hour to five minutes without announcing it.

Herk's investigation found that they hadn't. The one-hour window on subscriptions is still intact. But the confusion is understandable, because the TTL situation is genuinely complicated:

Claude subscription (in-terminal / extension): 1-hour TTL ✅
API usage: 5-minute TTL by default (can be upgraded to 1 hour at extra cost)
Sub-agents on any plan: 5-minute TTL, no exceptions
Claude.ai web chat: Undocumented. Possibly subscription-equivalent, but not confirmed.

That last point is worth sitting with. Claude.ai's caching behavior isn't clearly documented, which means users pasting large documents into web chats are operating on assumptions. Herk's recommendation—use Projects for document-heavy work rather than raw chat—is grounded in reasonable inference, but it's inference nonetheless.

The broader issue here is documentation fragmentation. As Herk notes, "all of this is documented about Claude Code in the API, which are two very different things." For a feature this central to how efficiently the product works, that's a meaningful gap.

The three habits that actually protect your cache

Herk distills the practical side down to three behaviors he says cover 95% of users:

1. Don't let sessions go idle past an hour. This one's obvious once you understand the TTL, but easy to forget in practice. If you're stepping away from a deep session, the move is to hand it off before you go—not to hope you're back within 60 minutes.

2. Start fresh when you switch tasks. Use /compact or /clear to intentionally reset between projects. Bleeding context from one task into another doesn't just confuse the model—it wastes tokens on irrelevant cached context.

3. Use a session handoff instead of just clearing. This is the more interesting habit. Rather than hitting /clear and losing everything, Herk demonstrates a "session handoff" skill that summarizes the session's key decisions, files, and open threads before clearing. You paste that summary into the new session and pick up almost exactly where you left off. A 205,000-token session condensed and transferred in under a minute, context intact.

The handoff pattern is smart because it treats the cache's limitations as a design constraint to work with rather than fight against. Caching works on prefixes—it matches what it already knows. A fresh session with a dense, structured summary gives the cache something useful to anchor to immediately.

The setting that silently breaks everything

Here's the part of Herk's breakdown that I find most interesting, partly because it's counterintuitive enough to catch people off-guard.

There's a popular Claude Code setting where you use Opus for plan mode and Sonnet for execution—the idea being that you get Opus's planning capability and Sonnet's speed and efficiency. Herk has recommended this pattern before in other videos. And the logic for why it might save your session limit still holds in aggregate.

But here's what it also does: every time you toggle between plan mode and execution mode, you switch models. And as Herk explains, citing Thariq from Anthropic directly: "Each model has its own cache. Switching with model means the next request reads the entire conversation history with no cache hits even though the content is identical. The Opus plan model setting resolves to Opus during plan mode and Sonnet during execution. So each plan toggle is a model switch and starts a fresh cache."

Every. Single. Toggle. Fresh cache. On identical content.

This matters because it's the kind of optimization that feels like it should help—use the smarter model for hard thinking, the faster model for execution—but creates a hidden cost at the caching layer that isn't obvious from the surface behavior. The [hidden math behind session limits](/ article/hidden-math-behind-claude-session-limits) is often about these invisible state resets, not the conversations themselves.

There's a reasonable follow-up question here that Herk doesn't fully answer: given that each model switch nukes the cache, does the Opus-for-planning approach actually save tokens in net? He gestures at "yes, long-run," but the actual tradeoff depends on how often you're toggling and how deep your sessions run. Worth testing against your own usage patterns rather than taking it on faith.

Why Anthropic cares about this more than you might expect

One of the more illuminating details in Herk's breakdown is a quote from Thariq at Anthropic: "We actually run alerts on our prompt cache hit rate and declare SEVs if they're too low."

A SEV—severity incident—is the kind of internal alarm that gets people paged. The fact that Anthropic runs incident response protocols on cache hit rates tells you something about how load-bearing this feature is to the product's economics, not just user experience.

Herk's read on this is accurate: a high cache hit rate benefits everyone. Users get faster responses and more generous-feeling limits. Anthropic's serving costs drop. Long sessions stay practical. Low cache hit rates are genuinely lose-lose—users burn limits faster, Anthropic serves more compute for the same subscription revenue.

That alignment of incentives is actually kind of nice? It means the feature Anthropic built to save money also directly benefits users, and they have a structural reason to keep it working well.

What we still don't know

Prompt caching is one of those features where the public documentation lags behind what's actually happening in production. Claude.ai's caching behavior remains opaque. The precise interaction between sub-agent TTLs and subscription limits isn't fully documented. And the net impact of model-switching patterns like the Opus/Sonnet split requires more empirical testing than anyone seems to have published yet.

The practical upshot for most Claude Code users is more mundane: don't go idle for an hour, don't switch models mid-session unless you mean to, and if you're pasting big documents anywhere, do it inside a Project. Three habits. Ninety percent discount on your cached tokens.

The more interesting question is what this reveals about how subscription AI products are actually priced. Session limits aren't just about compute—they're about whether the caching layer is doing its job. If your sessions feel expensive, the answer usually isn't a higher tier plan. It's understanding where the cache breaks.

Yuki Okonkwo is Buzzrag's AI & Machine Learning Correspondent.