Backend
Do They Do as They Are Told? Investigating Coding Agents' Adherence to Explicit Instructions in Context Files
arXiv cs.SE0 views
arXiv:2611.13709v1 Announce Type: new Abstract: Developers increasingly rely on agent context files (ACFs), persistent, project-scoped artifacts such as AGENTS.md, to steer the behavior of autonomous coding agents without renegotiating expectations at every interaction. Yet this governance mechanism rests on an untested assumption: that instructions written into an ACF are actually carried out, rather than ignored or loosely followed. Existing studies describe what ACFs contain, but not whether individual instructions are followed, leaving open which guidance can be trusted. We address this gap through an empirical study of real-world ACF snapshots and their linked agent-generated commits: characterizing the explicit, actionable instructions ACFs encode, operationalizing them into executable measurement scripts, and using these scripts to assess instruction-level compliance. We find that most observable instructions can be operationalized and that agents follow ACF guidance more often than not, but adherence splits sharply between instructions followed consistently and those never respected, with differences across instruction categories and coding agents. This split shows where ACF guidance works and where it does not, pointing to instruction types that may need rewriting and to agents that may need enforcement beyond the ACF. These findings lay the groundwork for systematic approaches to specifying, verifying, and enforcing project-specific instructions in autonomous software development.
Source:https://arxiv.org/abs/2611.13709PreviousAn Empirical Study on Improving LLM-Based Issue Classification through Domain-Informed AnalysisNextOpenRE-Bench: A Benchmark Harness for Multi-Agent Requirements Engineering with Mined GitHub Issues
Related posts

JavaScript Deobfuscator Output: Reading It Layer by Layer
0dev.to

Building a Python Framework to Monitor IDMC Taskflow Execution
0dev.to
Best self-hosted PaaS in 2026: six tools compared
0dev.to
DKIM Domain Verification: A Compliance-Evidence Guide for Fintech Email
0dev.to
Realtime Presence Sidebar: Node.js Room Cleanup and Update Guarantees
0dev.to
Weekly Digest Retry Design Explained (SaaS Queue, Cron, DLQ, Cost)
0dev.to