Encryption Bypassed: How AI Models Easily Expose Users' Sensitive Data

Researchers from Germany have published a paper titled 'Stealing Reasoning Traces from Proprietary LLM APIs' demonstrating how AI models from Anthropic, OpenAI, and Google can leak sensitive user data. When performing reasoning tasks, these models generate an encrypted string called a 'reasoning log' that is sent back to the server with each new response to provide context. The researchers found that reasoning logs can be copied and used in other models from the same company, including older or modified versions. If a modified model has its safety measures removed via jailbreak, it can decode the encryption of the logs. Using 6,708 public repositories from GitHub and Hugging Face, they successfully decrypted 315,320 reasoning logs, which contained 62 API keys, 33 passwords, 24 access tokens, 30 personal email addresses, and names/addresses of project maintainers. Many of these details were only present in the reasoning logs, not elsewhere in the chat session. Security expert Voldemaras Kadys of Cybernews notes the encryption was not broken in a traditional sense; instead, the sharing of encrypted reasoning traces between compatible models allows weaker models to become a master key for decryption. He advises treating reasoning logs as sensitive data and never sharing them.