Backend
Applying Security by Design at the Point of Execution: How Governed Security Requirements Affect the Security of AI-Generated Code
arXiv cs.SE0 views
arXiv:2610.10659v1 Announce Type: new Abstract: Security by design asks that security requirements are defined before code is written. Secure-code benchmarks typically measure the opposite situation: the agent receives the task without requirements and is scored with security tests it has not seen. We measured what changes when security requirements, selected from a governed security-by-design knowledge base (SbD-ToE) and delivered through a Model Context Protocol (MCP) server, are given to the generator at the point of execution. On DualGauge, which scores with a language-model judge, we ran 59 Python tasks: the share of tasks passing all security tests rose from 44.1% to 78.0% and the share of security tests passed from 77.4% to 93.0%. On BaxBench, which runs functional tests and real exploits in containers, we ran 28 backend scenarios: among functionally correct solutions, the share with no successful exploit rose from 65% to 86%, comparable to the authors' Oracle Security Reminder (85%), which they call an unrealistic upper bound; the requirement delivery reached it without knowing the tests. The study has limits. The pre-specified joint secure-and-functional metric did not improve significantly on either benchmark. The code written with the governed requirements is stricter, and it failed functional tests that fix values the task never states; we attribute those failures case by case in a post-hoc analysis. Each experiment used one model and one generation per task. The requirements come from the author's open-source manual; we do not assess its completeness. Governed security requirements delivered at the point of execution appear to improve the security of generated code. Current benchmarks cannot tell whether that code meets the security requirements that apply to it, only whether it shows the weaknesses their evaluators were built to detect. We propose measuring conformance with the applicable requirements.
Source:https://arxiv.org/abs/2610.10659PreviousCan LLMs Simulate Novice Programmers' Misconceptions?NextCross-Provider Review as a Runtime Contract for Coding Agents: A Controlled Pilot and Fault-Injection Study
Related posts

Copilot Fusion Takes Local Models, and the Offline Boundary Is Somewhere Else
0dev.to AI

A CTO Checklist for Evaluating AI Features Before Production Release
0dev.to AI
Humanoid Robot Demo Makes the Case for Keeping the Human Form
0dev.to AI
Supabase anon key vs service role key: what each one opens
0dev.to Security
After the eMudhra CA compromise: audit your own TLS stack in 30 minutes
0dev.to Security
Exposed Building Automation Services: Bavaria and DACH
0dev.to Security