Backend
Auditing Empirical Comparisons in Quantum Software
arXiv cs.SE0 views
arXiv:2607.00516v2 Announce Type: replace Abstract: Empirical quantum-software papers often claim that one compiler produces smaller circuits, one optimizer is more reliable, or one ansatz achieves better solution quality. Such comparisons depend on benchmark scope, circuit construction, compilation, sampling, backends, noise models, optimizer choices, and resource budgets. Existing testing, benchmarking, and reproducibility methods help assess programs and experiments but do not directly audit whether the reported comparison is supported by the evidence available in the paper and artifact. We present CLAIMSTAB-QC, a source-bounded framework that records baselines, metric, relation, and admissible evidence; locks the audit design before outcomes are computed; and reports a scoped relation outcome or an explicit evidence boundary. Strict scalar-directional comparisons are classified as Sustained, Unresolved, or Not preserved within the locked audit scope. We examine 455 comparative claims from 119 quantum-software papers. A materialization gap emerges: 175 claims can be represented for audit planning, 79 become scalar-directional planning records, and 53 yield lockable audit or diagnostic designs, but only 8 expose enough matched evidence to audit the original comparison without proxy reconstruction. Among these 8, 2 are Sustained, 4 Unresolved, and 2 Not preserved. Controlled diagnostics over 24 benchmark-relevant quantum-workload comparisons further show that simpler checks can preserve apparent directions despite weaker support under locked designs. These results indicate that performance orderings should be reported with matched evidence sufficient to audit their stated scope.
Source:https://arxiv.org/abs/2607.00516PreviousThe Impact of AI Coding Assistants on Software Engineering: A Longitudinal StudyNextTTHE: Test-Time Harness Evolution
Related posts
Building a PDF toolbox that never uploads your files: pdf-lib, vanilla JS, and zero backend
0dev.to JavaScript
Romanian CUI Validation in TypeScript, Zod and React
0dev.to JavaScript
The first proof no human could read
0dev.to JavaScript

Why Rust Web Development Sucked for Web Developers (Until Now)
0dev.to Rust
Sensor Response-Time Reduction using Long-Short Term Memory Network Forecasting
0arXiv eess.SP
The Impact of AI Coding Assistants on Software Engineering: A Longitudinal Study
0arXiv cs.SE