Microsoft Unveils 'Whisper Leak' Attack: A Novel Threat to AI Chatbot Privacy
Microsoft has revealed a groundbreaking security vulnerability in AI chatbots, dubbed the 'Whisper Leak' attack. This innovative side-channel attack could enable malicious actors to extract sensitive information about user conversations, even when encrypted. The attack targets remote language models, allowing attackers to infer the topics of user prompts, posing a significant threat to user and enterprise privacy.
The 'Whisper Leak' attack leverages the streaming nature of language models, where responses are generated incrementally. By analyzing encrypted traffic, attackers can extract packet size and timing sequences, using trained classifiers to identify sensitive topics. This is particularly concerning as many AI chatbots use HTTPS encryption to secure user communications.
Microsoft's research team, along with security experts Jonathan Bar Or and Geoff McDonald, demonstrated the attack's effectiveness using various machine learning models. They found that models from Mistral, xAI, DeepSeek, and OpenAI could achieve over 98% accuracy in identifying specific topics, even in encrypted conversations. This means a government agency or internet service provider could potentially monitor and identify users discussing sensitive topics, such as money laundering or political dissent, despite encryption.
The attack's power lies in its ability to improve with time, as attackers gather more training data. Microsoft and its partners have developed countermeasures, such as adding random text sequences to responses, to mitigate the risk. However, the threat remains a concern, especially with the increasing susceptibility of open-weight LLMs to adversarial manipulation, as highlighted in a recent evaluation by Cisco AI Defense.
This discovery underscores the need for robust security measures in AI chatbots. Developers must enforce security controls, fine-tune models to resist attacks, and conduct regular security assessments to protect user privacy and data.
