Security
When AI Agrees Too Much: Manipulating Automated Code Review via Bias Injection
arXiv cs.SE0 views
arXiv:2603.18740v5 Announce Type: replace Abstract: Automated Code Review (ACR) systems integrating Large Language Models (LLMs) are increasingly adopted in software development workflows, ranging from interactive assistants to autonomous agents in CI/CD pipelines. In this paper, we study how LLM-based vulnerability detection in ACR is affected by the framing effect: the tendency to let the presentation of information override its semantic content in forming judgments. We examine whether adversaries can exploit this through contextual-bias injection (crafting PR metadata to bias ACR security judgments) as a supply-chain attack vector against real-world ACR pipelines. To this end, we first conduct a large-scale exploratory study across 6 LLMs under five framing conditions, establishing the framing effect as a systematic and widespread phenomenon in LLM-based vulnerability detection. We then design a realistic and controlled experimental environment, evaluating 33 CVEs across 20 real-world projects and two popular ACR pipelines (Claude Code and CodeRabbit), to assess the susceptibility of real-world ACR pipelines to vulnerability re-introduction attacks. We employ two attack strategies: a template-based attack inspired by prior related work, and a novel LLM-assisted refinement attack. We find that template-based attacks are ineffective and may even backfire, as direct biasing attempts raise suspicions. Our refinement attack, on the other hand, is successful in 32/33 (97%) cases, exploiting a fundamental asymmetry: attackers can iteratively refine attacks against a local clone of the review pipeline, while defenders have only one chance to detect them. Overall, our findings highlight the dangers of over-relying on ACR and stress the importance of human oversight and contributor trust in the development process.
Source:https://arxiv.org/abs/2603.18740PreviousSWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?NextSemLoc: Structured Grounding of Free-Form LLM Reasoning for Fault Localization
Related posts
[$] Managing the kernel CVE flood
0LWN.net
5 Stars for sale: Inside the underground fake review trade
0Hacker News New
Show HN: Hands-on security exercises for students and instructors
0Hacker News Show
Investors Dish on New Startups & What They'd Back Next
0Dark Reading

NVIDIA Brev security breach may have exposed SSH keys used to access customer VMs — investigation finds unauthorized activity spanning 16 days, but no evidence of misuse
0Tom's Hardware
Testing and rebuilding Node.js snapshot artifacts after CVE-2026-106074
0dev.to Node