Modern reasoning models think before they speak — and the thinking is supposed to stay hidden. When you use a frontier AI API, the provider shows you the final answer but encrypts the chain-of-thought: the intermediate reasoning the model used to get there. That hidden trace can contain sensitive material — the model’s unfiltered deliberation, details about its instructions, sometimes data it was told not to reveal.
A paper published on arXiv on August 10, 2026 — “Stealing Reasoning Traces from Proprietary LLM APIs” (arXiv:2608.09867), from researchers at the ELLIS Institute Tübingen and the Max Planck Institute — showed that this encryption wasn’t the protection everyone assumed. The encrypted reasoning blocks returned by OpenAI, Anthropic and Google reasoning APIs turned out to be cross-session compatible. And that compatibility opened the door to theft.
How the attack worked
The core finding is elegant and unsettling. The researchers demonstrated that an encrypted reasoning block generated in one session could be injected into a different session — and that injecting a block from a stronger model into a weaker model from the same provider forced the weaker model to output the hidden reasoning verbatim.
Think about what that means in practice. The encryption was meant to bind the reasoning trace to its session: even if you intercepted the ciphertext, it should have been useless anywhere else. Instead, the blocks were portable across sessions. An attacker who could obtain an encrypted block — from logs, from a shared system, from any of the places ciphertext tends to accumulate — could feed it to a cheaper, weaker model and have the hidden reasoning read back to them in plaintext.
The paper demonstrates four distinct attack vectors built on this flaw, each a different route to the same prize: the model’s hidden deliberation, decrypted by the provider’s own models.
The scale of what leaked
To show this wasn’t just a lab curiosity, the researchers went looking in the wild. Across 6,708 public agent trajectories — the logged step-by-step records of AI agents doing tasks, which are increasingly shared, published and archived — they recovered 182 credentials and 367 items of personally identifiable information from reasoning traces.
That number deserves emphasis. Agent trajectories are a normal artifact of the agentic AI boom: developers log what their agents did, share traces for debugging, publish them in papers and demos. If hidden reasoning can be extracted from those traces, then every public trajectory is potentially carrying secrets its publisher never intended to disclose — API keys the agent was given, personal data it encountered mid-task, internal instructions it was told to keep quiet.
What got fixed
Here’s the good news, and it’s worth stating clearly: all three vendors — OpenAI, Anthropic and Google — deployed server-side mitigations, and the researchers report the original attack is no longer reproducible on patched builds. The paper is a case study in how this is supposed to work: researchers find the flaw, disclose it, vendors fix it, the paper documents both.
But the episode leaves two uncomfortable lessons. First, “encrypted” is a claim about a system, not a property of a ciphertext. The blocks were encrypted; they just weren’t bound to anything, which made the encryption nearly decorative against this attack. Second, the agentic era is generating a vast new corpus of semi-public model outputs — trajectories, logs, traces — whose privacy properties nobody has fully mapped. The 182 credentials the researchers recovered came from data that was already public.
Why it matters for builders
If you build on LLM APIs, the practical takeaways are concrete: treat reasoning traces — even encrypted ones — as sensitive data, with the same handling you’d give any secret. Audit what your agent logs retain. And assume that any “hidden” model output is hidden only until someone finds the key, which in this case turned out to be another model from the same provider. If you run your own evaluations, our piece on the threat-model coverage gap is worth reading next — automated red-teaming has the same blind-spot problem, and hidden model behavior keeps escaping the tests built to catch it.
The paper sits alongside a growing body of work probing what AI systems reveal unintentionally — from Anthropic’s own admission that its safety benchmarks are saturating to the steady drip of prompt-injection research. The throughline: as AI systems get more capable and more embedded, the gap between what they show and what they know becomes a security boundary. This paper showed one such boundary was thinner than advertised — and, credit where due, the industry moved fast once it knew.
Why providers hide the thinking
The hidden chain-of-thought is one of the most consequential design decisions in modern AI products — and one of the least discussed. Providers hide it for overlapping reasons: the raw reasoning can contain instructions the user shouldn’t see, deliberation about policy that would be trivially jailbroken if visible, and occasionally the model’s unfiltered first draft before its better judgment kicks in. There’s also a commercial motive: the reasoning trace is part of what makes a frontier model frontier, and exposing it would hand competitors a distillation dataset.
Encryption was supposed to square the circle — let the trace travel with the session for debugging and continuity, without letting anyone read it. The Tübingen/MPI paper showed the circle wasn’t squared: the encryption protected the trace in transit but didn’t bind it to its session, so the provider’s own models could be used as decryption oracles. It’s a classic cryptographic lesson relearned in an AI context: encrypting data is not the same as controlling what can be done with it.
The disclosure playbook worked — this time
Credit the process as well as the finding. The researchers disclosed to all three vendors, all three shipped server-side mitigations, and the paper documents the fixes rather than hoarding the exploit. That’s the coordinated-disclosure machinery functioning as designed — and it matters that it worked here, because the incentives around AI vulnerability disclosure are still being negotiated in real time.
The harder problem the paper leaves behind is the growing public corpus of agent traces. Every shared trajectory, every published demo log, every debugging artifact is a potential carrier for secrets its publisher can’t see — because the secrets are in the reasoning, and the reasoning was supposed to be encrypted. The 182 credentials and 367 PII items the researchers recovered came from data nobody thought was sensitive. After this paper, that assumption is no longer available.
Featured image: openclipart (elconomeno) via Wikimedia Commons (CC0 1.0)
