AI & ML
Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection
arXiv cs.CR0 views
arXiv:2610.00685v2 Announce Type: replace-cross Abstract: With the rapid adoption of large language models (LLMs) and parameter-efficient fine-tuning (PEFT) methods, the risk of backdoor attacks has become more severe. Existing backdoor purification methods typically rely on at least one of the strong assumptions, such as prior knowledge of triggers, access to clean references, or aggressive retraining, and they often lack comprehensive evaluations. These constraints substantially limit their practical applicability. To overcome these challenges, our work proposes purifying LoRA-tuned LLMs without these assumptions and even without post-hoc retraining of the suspect parameters. Our objective is to significantly reduce the attack success rates (ASR) while preserving both (i) the base model's general capabilities and (ii) the new downstream skills learned through the adapter. Through a series of examinations, we progressively scale our approach from a single layer in a text classification setting to a full-parameter LLM in the generative task. Through careful data curation and feature approximation, we extract high-fidelity backdoor directions and, for each layer or head, construct orthogonal null spaces in both the input and output channels, onto which the LoRA updates are projected. Empirically, our null-space projection method reduces the ASR from nearly 100% to less than 10%, while preserving the base model's benign performance and the adapter's learned abilities during downstream task adaptation.
Source:https://arxiv.org/abs/2610.00685PreviousBackdoor Containment via Expert Quarantine and Shutdown in LLMsNextInformation Loss Is All You Need: $TC^0$ Pseudorandomness from Worst-Case Lattices
Related posts
cBridge: Do You Need Gas on Both Chains?
0dev.to
Honor Puts a 4-DOF Titanium Gimbal Inside a Phone at 7,999 Yuan
0dev.to

Protesters break into Microsoft data center site in Amsterdam
0The Verge

After printing 40,000 whistles, this is the 3D printer I’d buy
0The Verge

A New Community For In-House UX People
0Smashing Magazine
A flaw in ChatGPT, a lesson for our front-ends
0dev.to