AI & ML
Adversarial Benchmarks and BenchDoor Attacks: Exploiting Benchmark Incentives for Backdoor Attacks
arXiv cs.CR0 views
arXiv:2610.27533v1 Announce Type: new Abstract: Machine learning benchmarks act as guideposts for model development and spur the creation of training datasets intended to improve benchmark scores. We introduce BenchDoor, a family of data poisoning attacks that exploits this incentive. Beyond preserving benign performance, BenchDoor requires benchmark gains that encourage adoption of data that also implants a backdoor. The adversary either manipulates a benchmark's training or test data, creating an adversarial benchmark, or supplies an independent training dataset while leaving the benchmark unchanged. We formalize five attack scenarios, including training on rewritten test examples and applying similarity-based decontamination. Across multiple-choice Q&A, text classification, and open-ended Q&A with Qwen and Gemma models, BenchDoor achieves positive benchmark gains for all 30 evaluated configurations, with held-out attack success rates of 82.33% to 100.00% at poisoning rates of 2% to 10%. Both objectives remain achievable after synthetic data generation and similarity-based decontamination, with attack success rates of at least 90.33%. These findings show how benchmarks can serve as attack vectors both through their own data and through training resources that deliver improvements on them.
Source:https://arxiv.org/abs/2610.27533PreviousBeyond the Prompt: Recovering Hidden Concepts in Text-to-Image ModelsNextKNOX: Hardware-assisted Regulatory Sandbox for Generative Models
Related posts
C64 Basic Dungeon Crawler: Stairs, Levels and Levelling Up (Part 11)
0Hacker News
Anthropic's internal AI use, at retail prices, now exceeds its employee pay
0Hacker News
Launch HN: Momentic (YC W24) – Agents that catch bugs by using your app
0Hacker News

Behavioral Design Unpacked: Nudges, Funnels & Persuasion | UX Unpacked #3
0dev.to UX
National Grid buys Teesside land for UK largest substation to power AI data center
0dev.to News

Leonardo AI Alternative in 2026
0dev.to Discuss