Paper · arXiv 2410.22884
Stealing User Prompts from Mixture of Experts
Itay Yona, Ilia Shumailov, Jamie Hayes, Nicholas Carlini
Attackers can exploit architectural flaws in Mixture-of-Experts models to extract user prompts by manipulating query batches in a specific manner.
Abstract
Mixture-of-Experts (MoE) models improve the efficiency and scalability of dense language models by routing each token to a small number of experts in each layer. In this paper, we show how an adversary that can arrange for their queries to appear in the same batch of examples as a victim's queries can exploit Expert-Choice-Routing to fully disclose a victim's prompt. We successfully demonstrate the effectiveness of this attack on a two-layer Mixtral model, exploiting the tie-handling behavior of the torch.topk CUDA implementation. Our results show that we can extract the entire prompt using O({VM}^2) queries (with vocabulary size V and prompt length M) or 100 queries on average per token in the setting we consider. This is the first attack to exploit architectural flaws for the purpose of extracting user prompts, introducing a new class of LLM vulnerabilities.