AI & ML
EmbedGym: Letting Coding Agents Develop and Debug Embedded Firmware Without Physical Hardware
arXiv cs.SE0 views
arXiv:2611.21845v1 Announce Type: new Abstract: Modern coding agents perform well on general software benchmarks because they can run code and unit tests in a closed loop. Embedded firmware development, however, breaks this loop: a program that compiles can still fail on real hardware due to pin polarity, bus protocol, timing, or networking bugs that an agent at a text terminal cannot observe. Existing embedded coding agents either skip hardware execution, drive GPIO-only simulators with multi-second delays, or test on physical boards in a static environment. In this paper, we present EmbedGym, an interactive virtual hardware environment that lets coding agents develop and debug embedded firmware without physical hardware. EmbedGym uses instruction-level microcontroller unit (MCU) emulators to execute the same binary that would be flashed onto a physical board, connects them to 41 peripheral models, an analog circuit simulator, and virtual WiFi/Bluetooth Low Energy (BLE) endpoints, and exposes an interface for agents to change sensor inputs, press buttons, and inspect hardware states. On top of it, we build EmbedGym-Bench, which consists of 598 tasks across 6 MCU-framework pairs graded by hidden timed test vectors, using a five-stage multi-agent workflow that synthesizes peripheral models and tasks from datasheets and driver libraries. We expose EmbedGym to a coding agent at three levels and evaluate eight models over more than 14K trajectories. Even when compilation succeeds 99.7% of the time, 19.9% of compiled firmwares fail at runtime (up to 73% on individual targets). In contrast, interactive hardware testing in EmbedGym increases pass rates across models by +4.9% to +14.7% (up to +31% on individual targets). Our results also show that an agent with EmbedGym outperforms prior embedded agent frameworks, and lightweight models with EmbedGym match frontier models without it at 30-70x lower cost.
Source:https://arxiv.org/abs/2611.21845PreviousCan Coding Agents Write and Transpile Unit TestsNextOperating an Agent-Driven Dataset Factory: Discovery, Construction, and Public Release
Related posts
What Might the Next Generation of AI Look Like?
0dev.to Discuss
NVIDIA Q3 FY27 Earnings: $115.8B Revenue, $390B Supply Gamble and the Post-Jensen AI Infrastructure Test
0dev.to Discuss
Do developers actually think about code energy efficiency, or is it just about cloud bills?
0dev.to Discuss
Best AI Humanizer Tools for SEO Articles and Long-Form Blog Posts
0dev.to Productivity
Beyond Notes: 10 Hidden Power‑Ups in iPhone’s Notes App
0dev.to Productivity

I let Claude map a business process through MCP, and it writes code instead of drawing boxes
0dev.to Productivity