AI & ML
On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection
arXiv cs.CR0 views
arXiv:2610.32531v1 Announce Type: new Abstract: Prompt injection benchmarks assess whether large language models (LLMs) and agents resist malicious instructions embedded in external content. However, a system could perform well by rejecting attacks that obviously abandon the user's request, while still accepting changes that retain most of a legitimate answer but make it incorrect or harmful. Testing more instructions for unrelated outcomes would not necessarily reveal this weakness. We call attacks seeking responses designed to resemble legitimate answers while introducing incorrect or harmful changes near-target prompt injection. We introduce NTBench, the Near-Target Benchmark, which builds on legitimate tasks from existing datasets. For each task, a legitimate reference outcome guides the generation of target--instruction pairs, with separate tracks targeting incorrect and harmful outcomes in responses or agent tool invocations. Harmful targets can also accompany otherwise correct task completion. We combine LLM-based checks with human validation of the intended deviations, then assess whether injected instructions induce them using response judgments or tool-call checks, excluding deviations already observed without injection. We evaluate seven LLMs on ten datasets for language tasks and two agent benchmarks. Under a common budget of three attack executions per sample, both tracks achieve higher overall attack success rates than the fixed-target baseline for every evaluated model in each setting. On language tasks, the increases range from 9.3 to 35.8 percentage points for incorrect targets and 1.7 to 17.3 percentage points for harmful targets. These results show that resistance to the tested fixed-target attacks does not imply resistance to related incorrect or harmful outcomes, and that reference-guided targets broaden the failures revealed by robustness evaluation.
Source:https://arxiv.org/abs/2610.32531PreviousBeyond Copying: Lifelong Smart Contract Security Task Construction via Protocol MutationNextPrivacy-Preserving Federated Loss Exploration for Vision-Based Perception under Non-IID Data
Related posts

Tech Firms Dismantle Botnet That Turned 1.3M TV Boxes Into Proxies
0Krebs on Security
ISC Stormcast For Tuesday, November 3rd, 2026 https://isc.sans.edu/podcastdetail/10162, (Tue, Nov 3rd)
0SANS ISC
Why an AI Logo Is Not Done Until It Is a Vector File
0dev.to Design
Descript: The Text-Based Editing Revolution and the Blueprint for Audio-Visual Velocity
0dev.to Productivity

Planning Three Meals a Day at a Phu Quoc Resort
0dev.to Productivity
《What I Learned Building a Pay-Per-Use Voice Cloning Platform》
0dev.to Productivity