AI & ML
When Agents Collude: Weak-to-Strong On-Policy Distillation for Mitigating Strategic Deception in Multi-Agent Systems
arXiv cs.MA0 views
arXiv:2610.33011v1 Announce Type: new Abstract: While recent advances in autonomous agents enable them to coordinate on increasingly complex tasks, prior work shows that this growing capacity for coordination can create safety risks, including collusion, in which agents jointly pursue misaligned actions while concealing the true basis of those actions from oversight. We study this risk in a simulated insider-trading environment, comparing single agents with sequential and decentralized teams across seven language models from three families. By varying agents' system-level objectives and pressure conditions, we examine when coordinated agents act on material nonpublic information, conceal the basis of a misaligned trade in a report, and maintain that deception under follow-up questioning. Compared to single-agent decision-making, multi-agent interaction raises misaligned-action rates from 35.4% to 68.2% and persistent-deception rates from 31.0% to 62.2%, both measured over all runs. These effects are largest under existential pressure, where teams sustain deception through follow-up questioning in 87% of all runs. Qualitative analysis reveals shared cover stories and coordinated rationales, suggesting group-level strategic deception rather than isolated reporting errors. To mitigate this behavior, we introduce PACT, a weak-to-strong on-policy distillation framework for mitigating collusive deception. Using generic honesty prompts, PACT trains a stronger student on its own sampled continuations, matching its next-token distribution to that of a smaller, frozen honesty-aligned teacher via reverse-KL supervision. Under existential pressure with ethical instructions, PACT reduces multi-agent persistent deception from 80% to 3%. Our results show that multi-agent coordination amplifies strategic deception and that on-policy honesty alignment can substantially mitigate these failures.
Source:https://arxiv.org/abs/2610.33011PreviousLLM-Augmented Agent-Based Simulation of Cyber Social AgentsNextUMAS: System-Level Uncertainty Quantification for Multi-Agent LLM Systems
Related posts

Apple 'HomeView Ultra' With 9-Inch 360º Screen Detailed in New Report
0MacRumors

Apple Isn't Done With 2026: Here's What's Still Coming
0MacRumors

Mark Gurman: Apple Initially Aimed to Release iPhone Duo in November
0MacRumors

iOS 27.2.1 Coming Soon, iOS 27.3 Beta Likely Available Tomorrow
0MacRumors
Grokipedia offers lessons in how AI can help Wikipedia
0Hacker News New
Are Prediction Markets Missing the Voters Who Could Decide the 2026 Midterms?
0Hacker News New