Video: EVIL-AI project0:27
Evil-AI Benchmark: security testing for AI agents that control smart environments
A servo on a desk turns because an AI agent was tricked into turning it. Evil-AI Benchmark, the group's open-source framework from the IEEE Access paper of the same name (listed under Publications), attacks language-model agents that control smart environments, and every tool call they make drives real hardware. Across eight open models, 250 attacks each, the attack success rate ranged from 0.8% to 54%. The benchmark is open source, so anyone can test their own agent. Narrated by Aygün Varol, doctoral researcher at Tampere University.