AI & ML
Reasoning-Trace Overflow in Code Generation: An Empirical Study of Explicit Reasoning Models
arXiv cs.SE0 views
arXiv:2611.06095v1 Announce Type: cross Abstract: Explicit reasoning can help language models generate code, but it also introduces a practical risk: the model may spend so much of its output budget reasoning that it never produces usable code. We study this behaviour in relatively small open-weight reasoning models, framing it as reasoning-trace overflow. Across seven models and six code benchmark settings, we measure how much of a fixed 32k-token output budget is spent on reasoning, whether that reasoning improves pass@1, and whether lightweight inference-time controls can mitigate overflow. Our results show that the value of reasoning is highly conditional. Overflow is rare on simpler function-level tasks, but becomes common on harder code-generation benchmarks and for particular model families: on several model--benchmark pairs, more than half of generations exhaust the full 32k-token budget while still reasoning. Most complete overflow is associated with repetitive reasoning loops, although substantial long reasoning also occurs without looping. Disabling reasoning can therefore be competitive for overflow-prone models, simply because it ensures that an answer is produced. At the same time, reasoning remains valuable for stronger models on harder algorithmic and code-understanding tasks. Finally, prompt steering can substantially reduce overflow for some models but is inconsistent, while budgeted transitions provide more direct control and can improve both performance and token efficiency for overflow-prone models. These findings suggest that explicit reasoning should be treated as a controllable resource for code generation, rather than an always-on capability.
Source:https://arxiv.org/abs/2611.06095PreviousViBench 2.0: A Benchmark for Long-Horizon Software BuildingNextHow the Misuse of a Dataset Harmed Semantic Clone Detection
Related posts
A Bayesian Optimized Biquad Filter-Burg Model for Real-Time EEG Phase Estimation
0arXiv eess.SP
Thought-deciphering via hierarchical EEG-to-text alignment
0arXiv eess.SP
Neuromorphic Decoding and Dynamical Image Reconstruction from Populations of Spiking Retinal Ganglion Cells
0arXiv eess.SP
Benchmarking Sensor-Fault Robustness in Forecasting
0arXiv eess.SP
Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves
0arXiv eess.SP
Multi-Threshold Sampling: Signal Space, Sampling Operators, and First Recorded Crossing-Time Distributions
0arXiv eess.SP