AI & ML
Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers
arXiv cs.CR0 views
arXiv:2606.21037v3 Announce Type: replace Abstract: The empirical foundation of cyber deception relies on human-centered hypotheses, but the rapid emergence of autonomous, AI-enabled attackers challenges whether this foundation transfers to AI agents. To address this, we introduce an automated evaluation framework adapted from the Honeyquest instrument to assess LLM attacker judgment at scale. Our 21-LLM cohort spanned 10 providers, diverse architectures and specializations, open- and closed-weight models, and parameter scales from 8B to over 1T. We evaluated the performance of this LLM cohort (yielding 10,962 responses) against the 47-participant human baseline across an identical set of 174 reconnaissance queries. Our empirical evaluation reveals three key findings that establish LLMs as a distinct attacker class: (1) every model in our cohort falls for deceptive traps at a significantly higher rate than human attackers; (2) the defensive attention-diversion effect observed in humans is statistically absent in our LLM cohort; and (3) a critical recognition-action gap, where LLMs successfully articulate trap recognition in their reasoning but exploit the deceptive elements anyway 73.4% of the time. Across the 21 models, trap recognition in reasoning text did not predict fell-for-trap behavior (Spearman r = +0.08, p = 0.73). Ultimately, these findings demonstrate that human-centered deception hypotheses do not reliably transfer to AI attackers, highlighting the critical need for new research into AI-native active defense frameworks.
Source:https://arxiv.org/abs/2606.21037PreviousWatermarking Should Be Treated as a Monitoring PrimitiveNextFrom Review to Authorization: Key-Isolated Threshold Signing for LLM Agents
Related posts
LifeVal
0OpenAI Blog
Claude’s new rules against game cloning won’t stop the AI copying boom
0Ars Technica

OpenAI pitches data center power plants as a way to protect nearby areas
0The Verge

Anthropic clamps down on Claude’s AI video game clones
0The Verge
Readwise takes on Amazon’s Kindle and Goodreads with its new app for book lovers
0TechCrunch
Five regression tests for channel-aware AI reply drafts
0dev.to