Distributed Attacks in Multi-Agent Environments

Senior Author on (Radev et al., 2026), accepted at the Agents in the Wild (Spotlight) and Trustworthy AI for Good (Oral, Outstanding Paper in the Cooperative AI Track) workshops at ICML 2026. Co-mentored with Benjamin Arnav at SPAR (Sep 2025 – May 2026), with project funding from Coefficient Giving starting Feb 2026.

Abstract

As agentic coding systems decompose work across multiple model instances, a critical safety question is whether those instances can coordinate to achieve a hidden malicious objective while remaining aligned with user intent. We introduce SCHEME, a benchmark of 17 task instances across 7 settings and 8 real open-source libraries, each pairing a legitimate software-engineering task with a covert side task. Every setting is designed so that no proper subset of agents can succeed alone: agents must decompose a shared sabotage plan, relay partial requirements under different communication topologies, and execute mutually consistent edits, testing genuine multi-agent coordination rather than individual capability. Evaluating with GPT 5.1 Codex and Gemini 3.1 Pro, we find coordinated sabotage is already practical, with Gemini completing the covert objective while succeeding on the legitimate task in 84% of samples and Codex in 46%. A failure analysis shows that recovery, not failure incidence, drives the model gap: both models suffer similar numbers of communication failures per trajectory, but Gemini recovers from 73% of occurrences against Codex’s 56%. However, the agents’ code edits remain detectable: a trusted monitor flags 100% of Gemini and 81% of Codex sabotage attempts at FPR2%, even without access to agent communication. Even with stealth-aware prompting, undetected sabotage stays at 0–2.4%; telling agents a monitor is present halves Gemini’s sabotage success without increasing the rate of undetected sabotage.