DevOps
Wrong yet Confident: Automated Detection of Silent Failures in Location-Dependent Software
arXiv cs.SE0 views
arXiv:2611.21615v1 Announce Type: new Abstract: We report an experience paper applying round-trip offset---an automated testing oracle for location-dependent software---at deployment scale across 32,019 GPS delivery transactions, corresponding to 24,501 unique coordinates, from a single USDA COVID-19 food assistance deployment in Bayam\'on, Puerto Rico. The oracle tests each geocoder's implicit spatial accuracy claim by measuring the deviation between an input coordinate and the coordinate recovered after reverse-then-forward geocoding, without requiring manual labeling of correctness. We evaluated one community-contributed and two commercial geocoders against this operationally validated corpus. The experience surfaces two distinct failure classes, which we report with their respective denominators because they are not comparable quantities. First, 2,441 of the 24,501 unique coordinates (10.0%) are invisible to all three platforms: an acknowledged failure with no resolution, and a property of the corpus rather than of any one geocoder. Second, and platform-specific: 10.0% of the 13,221 coordinates Google resolves to a civic address exhibit round-trip offsets exceeding 2~km---5.4% of the full corpus---returned with no error signal, against 0.5% for the second commercial platform and 0.04% for the community geocoder. Round-trip offset detects all large-offset failures observed in this dataset while standard resolution-rate evaluation detects none. We document an eight-mode fault taxonomy, a structured comparison of detection methods, and concrete lessons for integrating automated geocoder verification into CI/CD pipelines. The key lesson: resolution rate measures availability; round-trip offset measures correctness. In non-standard addressing environments these two metrics select for opposite behaviors, and only the latter catches the failures that matter for autonomous action.
Source:https://arxiv.org/abs/2611.21615PreviousDoes Your Agent Need to Know Its Budget? A Placebo-Controlled Experiment on LLM Coding AgentsNextRevision-Consistent Code Context for Agents That Edit Their Environment
Related posts
Choosing a Real Device Farm? 10 Questions to Ask Before You Sign Up
0dev.to
Talos Linux
0Hacker News
Docker Engine: which versions still get updates
0dev.to Docker
Big Brains, Tiny Board: Pupavision, an Agentic Physical AI Instrument for Monitoring Drosophila Pupariation and Lifecycle Transitions
0arXiv cs.HC

Fixing Silent Misconfigurations in GitLab Runner: Adding Concurrency Alias Support
0dev.to DevOps

I let Claude Code debug my UART logs without ever reading the whole log
0dev.to OpenSource