Backend
Beyond QASecClaw: Agent Scaling, Verifier Feedback, and Evidence Reliability in Software Assurance
arXiv cs.SE0 views
arXiv:2610.36947v1 Announce Type: new Abstract: Can additional specialist language-model contexts improve software assurance when deterministic tool evidence is already available? We study this question through five experiments spanning authored Java warning controls, OWASP Benchmark assessment, Defects4J test generation, SV-COMP assertion verdicts, and deliberately misleading evidence packages. The evaluated systems use Qwen2.5-Coder-7B-Instruct, with a 1.5B sensitivity analysis, and complete 13,638 model requests. On eight related development warnings, increasing the context count from 1 to 50 increases token use 49-fold without improving unsafe clearance. On 160 OWASP cases, model configurations trade clearance against unresolved outcomes; the ten-context arm assesses only 96 cases. On 32 SV-COMP tasks, raw verifier feedback raises mean correct model verdicts from 12.7 to 22.0, but also raises incorrect verdicts from one to seven; a JBMC-plus-JVM validation baseline returns 26 correct verdicts and six unresolved outcomes. On two sets of 120 evidence packages, models reject 70-100% of applicable counterexamples, while construction-specific scope checks classify every package correctly. No target defect is discovered in three Defects4J bugs, one unsupported by the test adapter. These descriptive results support deterministic evidence admission for the tested settings, while exposing limits of verdict aggregation, feedback interpretation, and the available evaluation coverage.
Source:https://arxiv.org/abs/2610.36947PreviousA lightweight C Code Generator for Heterogeneous TinyML SystemsNextDoes More Thinking Pay Off? Reasoning Cost and Correctness of LLMs in Code Generation and Bug Fixing
Related posts
Convert YAML to JSON in the Browser (and JSON to YAML) Without Uploading the File
0dev.to
Vercel agent-desktop: computer use for coding agents through the accessibility tree
0dev.to
WARN Act and Job Search — How to Use the 60-Day Notice Period to Find Your Next Job
0dev.to
Node.js SMS Verification Code: Generating and Sending One-Time Passwords via API Integration
0dev.to
Introducing Takeout Mender – A Free Tool to Recover Metadata from Google Takeout
0dev.to
DeepSeek's Open-Source "Super-Agent" Codes Without Being Asked
0dev.to GitHub