AI & ML
Towards a Unified Misuse Monitoring Benchmark
arXiv cs.CR0 views
arXiv:2610.07089v1 Announce Type: new Abstract: LLM agents increasingly act in multi-actor environments, exposing them to misuse from multiple sources: decomposition attacks, where a harmful request is split into innocuous sub-requests, and prompt injection attacks, where a compromised tool delivers a malicious instruction. Existing evaluations treat these threats separately and ask whether a trajectory is harmful, rather than when it becomes harmful. We propose monitoring the agent's responses, where its actions are externalised, and ask whether the first point where monitors identify harm lands within a harm window (from the agent's first harmful commitment to goal execution). We develop a unified formalism for trace-level misuse monitoring and use it to construct a benchmark of ~6,200 conversation transcripts between a user, an LLM agent, and the external environment, spanning both threats in a shared schema, with a labelled harm window, corresponding benign controls, and matched instances of refusals to these requests. Across 17 monitor configurations, we find that our proposed action-framed monitors perform well on both threats under classical metrics (AUC: 0.95 and 0.99 respectively), while content-framed monitors collapse on injection attacks (AUC: 0.52). We also show that classical position-blind metrics paint an optimistic picture of monitor performance, since all monitors localise decomposition attacks poorly under the interval metric, which measures the ability to localise harm. Broadly, we illustrate the need for a unified study of misuse monitoring.
Source:https://arxiv.org/abs/2610.07089PreviousWhich Image Property Carries the Jailbreak? A Controlled Dissection of Image-to-Text JailbreaksNextJailbreaking Open-Weight LLMs via Random Embedding Perturbations
Related posts
Scrape LinkedIn job postings without logging in (Python + Google Sheets)
0dev.to
Same model, same defences: 7 document attacks down to 0, chat attacks barely moved
0dev.to
Skill to Create Product Videos
0Hacker News
SOML – A Config Format for Humans
0Hacker News
Egoless Mathematics in the AI Era
0Hacker News
LifeVal
0OpenAI Blog