DevOps
Do Coding Agents Understand Why Systems Crash---and Why They Don't?
arXiv cs.SE0 views
arXiv:2610.29332v1 Announce Type: new Abstract: Coding agents are increasingly capable of fixing real-world bugs, yet they still struggle with large systems codebases such as the Linux kernel. To understand why, we test how agents reason about execution in these codebases through crash reasoning: predicting from source code whether a given input crashes the system. A correct prediction alone, however, does not establish this ability, since it may come from merely spotting suspicious code. In this paper, we propose source interventions: we pair source trees that differ by a single commit, so that the same input crashes only one of them, and test whether agents predict crashes more often on the crashing tree. Experiments with four coding agents on SysRB, our dataset of 934 such pairs from Linux and OSS-Fuzz projects, yield three main findings. First, agents are biased toward predicting no crash: they rarely raise false alarms but miss many real crashes, and even the best agent misses over a quarter of them. Second, locating the code that determines the crash outcome is not enough: agents often misjudge its effect, especially when the crash occurs far from that code. Third, testing only on crashing code is misleading: agents that merely notice suspicious code predict crashes on both source trees, which would look like correct reasoning on the crashing tree alone. Together, these findings indicate that current coding agents cannot yet reliably reason about execution in complex systems codebases, and that paired evaluation is necessary to measure this ability.
Source:https://arxiv.org/abs/2610.29332PreviousCloudIR-Bench: A Benchmark for Root Cause Analysis and Mitigation of Cloud IncidentsNextRegoMender: A Multi-agent Framework for Automated Rego Policy Repair
Related posts
Expressões Cron sem mistério: Como montar agendamentos sem errar na produção
0dev.to Linux

From Zero to Live: Containerise a Python App, Deploy It to Azure Container Apps and Automate It with GitHub Actions
0dev.to Docker

How I Built a Shift-Left DevSecOps Pipeline for FastAPI & Kubernetes (And Caught 3 Real High-Severity CVEs with Trivy) published: true
0dev.to Docker
Tor Browser 16.0 released
0LWN.net
Show HN: NetIntel – Deterministic Operational Intelliegence for Linux
0Hacker News Show
The advantages of mobile Linux over AOSP
0Lobsters