Security
Characterizing and Defending Utility-Hijacking in Instruction-Tuning Data Selection
arXiv cs.CR0 views
arXiv:2610.29886v1 Announce Type: new Abstract: Instruction-data selectors choose which instruction-following records are used for fine-tuning, improving alignment quality, raising training efficiency, and filtering out harmful records. A selector's choices thus directly affect the fine-tuned model's safety, yet despite their widespread use, the security of these selectors remains under-studied. We conduct the first systematic study of adaptive supplier-side attacks against utility-oriented instruction-data selectors and introduce utility-hijacking. Utility-hijacking shapes harmful records to remain compatible with a selector while preserving their malicious payload, causing top-k selection to concentrate rather than filter the poison. Achieving both is non-trivial, because optimizing a record for score tends to wash out its payload, while bluntly preserving the payload makes the record look low-value. Our generate-and-rerank procedure generates, per instruction, many payload-bearing candidates and submits only the highest-scoring survivor. Evaluated across three payloads (over-refusal, false-fact injection, and insecure code), six selectors, and victim models from Qwen2.5-7B to Qwen3-32B, utility-hijacking drives refusal from 0% to 94% and insecure code up to 56/90, while false-fact enrichment intensifies with scoring-model scale from 2.84x to 7.04x. To defend against this attack, we propose CONCERT, a targeted defense needing no content detector or poison labels. It flags suppliers selected far more often than chance allows, then re-selects under the same budget, keeping only records that two dissimilar selectors both rank highly. On a Qwen3-32B victim, CONCERT reverses the poison concentration from 6.67x to 0x (code) and 3.59x to 0.67x (facts), and reduces downstream harm by 100% and 75% respectively, at unchanged capability.
Source:https://arxiv.org/abs/2610.29886PreviousCompaction Laundering: Provenance Destruction in Context-Compacting Coding AgentsNextFree-XOR TFHE via Lazy Bootstrapping
Related posts
Mutipath TCP: Use and Detection, (Mon, Nov 2nd)
0SANS ISC
AI Vulnerability Finding
0Schneier on Security
[$] Managing the kernel CVE flood
0LWN.net
5 Stars for sale: Inside the underground fake review trade
0Hacker News New
Show HN: Hands-on security exercises for students and instructors
0Hacker News Show
Investors Dish on New Startups & What They'd Back Next
0Dark Reading