AI & ML
PrivacyPR: A Dataset of Real-World Privacy Fixes for Benchmarking Privacy-Aware Code Review
arXiv cs.CR0 views
arXiv:2611.18293v1 Announce Type: new Abstract: Coding agents increasingly review code, yet whether they catch the privacy defects that laws such as the GDPR require software to avoid remains largely untested, partly because few datasets connect real privacy defects to their accepted fixes and review context. We present PrivacyPR, a dataset of 4,131 merged GitHub pull requests (PRs) from 1,469 repositories, created between June 2018 and January 2024. Each PR includes its commits, diffs, review discussion, and LLM-generated labels for privacy relevance, fix type, and GDPR principles. PrivacyPR provides 1,169 high-confidence privacy fixes and 1,643 non-fix PRs, together with Verified, a manually checked split of 250 privacy fixes and 250 non-fix PRs, and Small, a 100 + 100 subset for evaluation. On Verified, both annotators endorsed 72% of the LLM's GDPR-principle assignments. Beyond supporting empirical study of how privacy defects arise and are fixed, PrivacyPR provides a benchmark for privacy-aware code review on real, developer-accepted fixes. Its evaluation harness reverts each fix to reintroduce the defect. On Small, two coding agents detect only 42-51% of defects under a generic review prompt, but 88-91% when explicitly asked to check privacy, while flagging 11-15% of non-fix PRs. These results show that privacy defects can be detected by coding agents, but that detection depends strongly on review intent and comes with false alarms. Data, code, reviews, judge verdicts, and an extensible evaluation harness are publicly available at https://doi.org/10.5281/zenodo.23744765.
Source:https://arxiv.org/abs/2611.18293PreviousFrom Bytes to Behavior: Request-Level Attribution from Live GPU Hidden StatesNextOn the Same Track: Privacy Leaks in Electric Rail Transport via Magnetic Fields
Related posts
Building an AI Agent Backend with FastAPI: From User Request to AI Response
0dev.to AI

SoduWork Custom Model Setup: Connect an OpenAI-Compatible Gateway and Handle the 180-Second Stall
0dev.to AI
AI startup hiring snapshot, 16 November 2026: 6,481 open roles, half at four companies
0dev.to AI
EC2 已经停止,账单为什么还在变?用资源清单代替猜测
0dev.to AWS

Amazon Black Friday 2026:5 Key Metrics and a Step-by-Step Data Collection Guide
0dev.to AWS
How to Trim Audio for an Online Course Lesson — Free, in Your Browser
0dev.to WebDev