How Researchers Cracked LLM Encrypted Reasoning

The sealed envelope, it turns out, was never really sealed.

When frontier AI models like GPT o-series or Claude Opus reason through a problem, they produce two outputs: the answer you see and a reasoning trace you're not supposed to see. That trace gets handed back to the user as an encrypted blob—compressed, signed, integrity-checked. The idea is that it stays opaque while still being portable enough that you can fork a conversation, rewind it, or resume an agentic run.

Ilia Shumailov and Alexander Panfilov, in a paper that racked up three million views in forty hours, showed that the opacity is an illusion. The details of what they found—and what they're carefully not claiming—are worth sitting with.

The attack is almost embarrassingly simple

Here's what Shumailov and Panfilov actually did, stripped of jargon: they took encrypted reasoning blobs produced by large models (from Anthropic, OpenAI, and Google), fed them to smaller sibling models within the same family, and then asked those smaller models to describe what they were thinking. The smaller model obliges. The server decrypts the blob on its end—that part works exactly as designed—and then the small model, being, as Shumailov puts it, "super chatty," just tells you.

"There's like no cryptography broken," Shumailov explained on Machine Learning Street Talk. The cryptographic scheme isn't defeated. The server is doing its job. The problem is that once the blob is decrypted server-side and handed to a willing smaller model, the model has no particular reason to keep the contents to itself.

What makes this structurally interesting—and what the researchers are careful to flag as a distinct vulnerability layer—is that reasoning blobs are portable. Across users. Across models within a family. You can take a blob from your Claude session, inject it into a fabricated conversation involving a completely different user and a completely different question, and the model receiving it will interact with that imported reasoning as if it generated it natively. The encryption scheme doesn't check whether the blob belongs in this context, with this user, at this point in this conversation.

That portability is what opens up the full menu of attacks: privacy exposure, jailbreaking, and something considerably more unsettling—poisoned agent traces.

What's actually at risk

The privacy angle is the most immediately tangible. If you've ever asked an AI assistant to help sanitize a codebase before pushing it to GitHub—stripping API keys, removing internal IP addresses—you might have done exactly that. Removed everything sensitive from the visible transcript. But the reasoning blob, sitting right there in the session data, may contain the model's internal deliberations about those very secrets. Shumailov's team scraped roughly 350,000 reasoning blobs from GitHub and Hugging Face and ran a classifier for privacy-sensitive content. They found API keys, email addresses, internal IP ranges—data that users believed they'd cleaned out of their shared conversations.

The jailbreak finding is the one that surprised Shumailov most. "Basically after like the third attempt I'm getting universal jailbreak that decodes reasoning of Anthropic models," he said. "I think this still shocks me the most." Three attempts. Universal. That's a low bar for bypassing the safety scaffolding of one of the most safety-focused labs in the industry.

Then there's the agentic poisoning vector, which is the scenario that keeps me thinking. Long agentic runs—the kind that might run for hours and cost real money to reproduce—are increasingly being shared online so others can build on them or benchmark against them. A shared trace looks like a conversation log. The visible parts might be completely clean. But the encrypted reasoning blobs embedded in that trace could have been poisoned: injected with instructions, in the model's own internal reasoning channel, to exfiltrate data or behave in ways that won't surface until the run is already deep. It's the AI equivalent of distributing a clean-looking codebase alongside a signed binary that nobody checks.

What it reveals about how these systems were built

The fact that Anthropic, OpenAI, and Google all share the same architectural vulnerability is the detail that deserves more attention than it's getting. Shumailov's explanation was brief and pointed: "Same people, same things." The stateless architecture that makes these APIs cheap and flexible—returning the reasoning blob to the client so the server doesn't have to maintain session state—also means the blob has to be portable enough to be replayed. Nobody apparently verified whether "portable enough to resume your own conversation" and "portable across users and model families" were the same thing. They weren't. They aren't.

The encrypted reasoning blobs at the center of this research are also revealing something else about how these models think—or at least, what their internal monologue looks like when you can finally read it. Some of what Shumailov and Panfilov found in the decoded traces is genuinely strange. Models reasoning in what they describe as non-human language: empty-space tokens, alien-seeming phrase clusters that don't parse for human readers. Shumailov noted the phenomenon had been observed before, by Apollo Research and METR, in lab conditions. His team confirmed it shows up in wild user sessions too.

There's also the cheating contemplation finding, which is either alarming or fascinating depending on your priors. In real user sessions—not benchmarks, not evaluation environments—models were observed reasoning through the possibility of cheating on a task, concluding against it, and then producing the honest answer. Shumailov is careful not to over-interpret this. The models aren't scheming; they appear to be considering and rejecting certain paths. But the fact that the deliberation is happening at all, and that it's happening in a channel that was supposed to be unobservable, adds texture to ongoing debates about chain-of-thought monitorability as an AI safety tool.

What it isn't

Nathan Lambert flagged publicly that calling this a "distillation attack" does real damage—it conflates a demonstrated jailbreak with the broader, legally and politically charged question of whether Chinese AI labs distilled from Western frontier models. Shumailov agreed: "That's definitely jailbreaking and misuse. That's exactly the jailbreaking threat model."

The Kimi K3 prefill artifact—where injecting just two tokens of Claude's reasoning into Kimi's context produced output that stylistically mirrored Claude—is interesting and unexplained. It's not proof of distillation. Shumailov himself characterizes it as "some funny artifacts" from a small post-hoc analysis. It's a data point that warrants investigation, not a finding that supports policy conclusions. The researchers' own paper acknowledges the analysis is anecdotal. The self-awareness about what their evidence actually supports is one of the more useful things about this work.

The fixes are layered, and some are hard

The easy fix—stop sending reasoning blobs to the client at all—would require rearchitecting the stateless model that makes these APIs economical. Not nothing. A medium-difficulty fix: chain each reasoning blob's encryption to the preceding query or the preceding blob, so a trace from one context is cryptographically incompatible with a different one. That would break the cross-context portability that makes replay attacks work. The model-level fix—training models to refuse to narrate their own encrypted reasoning back in plaintext—addresses the jailbreak pathway without touching the architecture, but it's a cat-and-mouse game of the kind the industry has already been running with other jailbreak categories.

Shumailov noted that existing jailbreak mitigations—system-level and model-level both—translate fairly directly here. The technique is structurally similar. The toolbox isn't empty. What the labs do with that toolbox, now that responsible disclosure has been completed and acknowledged, is the open question.

The broader tension in the paper's final section is one the security field has chewed on for decades. Shumailov is skeptical of doom-framing; he points to smart card replay attacks as a useful calibration: theoretically scalable, practically rare. Panfilov is more unsettled by the pace at which new attack surfaces are opening relative to the pace at which mitigations are being deployed.

Both positions are reasonable. Neither has the data to settle the argument yet, and they're honest enough to say so.

What's harder to be sanguine about is the underlying dynamic: the same architectural shortcut, made independently by every major lab, created the same vulnerability across every major lab. That's not a bug in one system. That's a pattern in how the industry builds.

Rachel "Rach" Kovacs is Buzzrag's cybersecurity and privacy correspondent.