AI & ML
The Hitchhiker's Guide to Program Analysis, Part III: Mostly Harmless LLMs
arXiv cs.SE0 views
arXiv:2606.15122v2 Announce Type: replace Abstract: Filtering out a static-analysis warning asserts that no feasible execution can trigger the reported error, whereas confirming a bug requires only one that does. Recent work uses LLMs to filter out warnings. However, LLM-generated explanations alone cannot establish that the reported error is impossible, and mistakenly filtering out a warning conceals a real bug. Sound static analysis can provide this guarantee, but scalable analyses leave behind the very warnings we seek to filter, whereas sufficiently precise whole-program analysis is impractical. Fortunately, analyzing a small part of the program is often enough to show that a warning is a false alarm. We therefore assign LLMs a different role: rather than judging the validity of warnings, they identify relevant code and construct a harness that supplies the context needed for precise, targeted static analysis. We present Evident, which combines LLMs with the static analyzer Frama-C/Eva. Because a generated harness may exclude inputs or paths that trigger real bugs, Evident restricts how the harness initializes inputs and validates it using the value flow reported by the original detector. A warning is filtered out only when validation passes and Eva rules out the reported error in the harness; otherwise, the warning is kept. On 200 real Android kernel-driver warnings, including 40 confirmed bugs, Evident filters out 48.8-63.8% of the 160 false alarms across four LLMs without missing any confirmed bug, whereas baselines that let LLMs make the final decision miss confirmed bugs with every evaluated model. With GPT-5.5, although these baselines achieve 2.0-5.5 percentage points higher overall accuracy by filtering out more false alarms, they miss 27.5-45.0% of the confirmed bugs. Evident also rediscovers a confirmed vulnerability that had previously been overlooked by both LLM-based filtering and manual review.
Source:https://arxiv.org/abs/2606.15122PreviousProperties with Holes: Meta-Property-Based Testing of Data-Intensive Computing FrameworksNextThe Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages
Related posts
I Launched My Company Website, Made Task Monitor Free, and Opened Custom Project Requests
0dev.to AI
How I turn Hacker News, GitHub and npm into product ideas every 3 hours
0dev.to AI
Lenovo's 'Encrypted' Fingerprint USB Drive Is an SD Card. How to Check Yours
0dev.to Linux

Email: OpenAI plans to merge ChatGPT's Chat and Work tabs on desktop, rolling out to select users before a wider release; Codex remains a separate experience (Ivan Mehta/TechCrunch)
0Techmeme

Memo: Alphabet is issuing €6B in euro-denominated debt across seven tranches, attracting €17B+ of demand; the company sold €9B of bonds earlier in 2026 (Reuters)
0Techmeme

S-1: Anthropic reports Q3 revenue up 12x YoY to nearly $17B, a net loss of $23.3B, up from $12.1B in Q3 2025, and operating expenses up 292% to $17.5B (Bailey Lipschultz/Bloomberg)
0Techmeme