AI & ML
Deletion, Substitution, Repair and Transplant: Component Attribution in BARBICAN, a Layered Prompt-Injection Gateway for MCP
arXiv cs.CR0 views
arXiv:2610.33198v1 Announce Type: new Abstract: A neural classifier can fail completely inside a layered defense without any end-to-end number moving. We demonstrate this on BARBICAN, a fail-closed gateway for language-model tool mediation whose arbitration stage spent an entire recorded evaluation receiving text mangled by a misconfigured tokenizer. Correcting the misconfiguration lifts that stage's corpus-wide area under the ROC curve (AUC) from 0.392 to 0.855 and pushes system recall down, from 80.67% to 80.34%; a stub that answers "malicious" scores 80.77%, higher than either. The reason emerges once the stage's input distribution is examined: upstream routing hands it a band that is 96.82% malicious, and the enforcement design lets an incriminating verdict block while an exculpating one releases an input only through one narrow branch. Yet deleting it costs 673 detections. Substituting its verdict with a constant instead moves between 12 and 140 of 25,000 decisions. Transplanted into the arbitration stage, an off-the-shelf classifier that separates the band's inputs at an AUC of 0.916 changes 16 of 25,000 decisions. Measured against such detectors with operating points chosen away from the test data, the gateway leads on its own benchmark distribution and falls behind once that distribution changes. In our earlier gateway, the review model's band is 92.94% malicious and a constant verdict is as accurate as the review model. We argue that an ablation figure for any routed stage is unreadable without the class balance of the band that stage receives, and that deletion experiments need substitution experiments beside them.
Source:https://arxiv.org/abs/2610.33198PreviousSpoofing the Toolbox: Optimizing Tool Descriptions to Deceive LLM AgentsNextA-GE: Privacy-Preserving Age Assurance via Fine-Grained Motion
Related posts
Cilium 1.20.0 Requires BTF Flag in Linux Kernel, Raspberry Pi OS Kernel Incompatible Without It
0dev.to
YC Labs and Olam Labs ran 50 AI agents in a Minecraft civilization experiment
0dev.to
Lavoro su piattaforma e algoritmi: cosa cambia con la direttiva europea per chi progetta il software
0dev.to
LifeVal Benchmark Targets Everyday Personal Assistant Tasks Shaped by ChatGPT User Feedback
0dev.to
Oracle files to sell $20B in new stock, bonds to finance AI buildout
0Hacker News
7.9% to 48.1%: LLM rewrote Qwen weights, no training
0dev.to