ConfusedPilot Attack Can Manipulate RAG-Based AI Systems

Caute_cautim — Wed, 16 Oct 2024 03:15:04 GMT

Hi All

Attackers can add a malicious document to the data pools used by artificial intelligence (AI) systems to create responses, which can confuse the system and potentially lead to misinformation and compromised decision-making processes within organisations.

Researchers from the Spark Research Lab at the University of Texas (UT) at Austin discovered the attack vector, which they've dubbed ConfusedPilot because it affects all retrieval augmented generation (RAG)-based AI systems, including Microsoft 365 Copilot. This includes other RAG-based systems that use Llama, Vicuna, and OpenAI, according to the researchers.

https://www.darkreading.com/cyberattacks-data-breaches/confusedpilot-attack-manipulate-rag-based-ai-systems

Regards

Caute_ Cautim

topic ConfusedPilot Attack Can Manipulate RAG-Based AI Systems in Industry News

ConfusedPilot Attack Can Manipulate RAG-Based AI Systems