Backend
Smoothing the Ramp, Not the Peak: Scheduling-Induced Power Dynamics of LLM Inference and Their Grid-Scale Consequences
arXiv eess.SY0 views
arXiv:2608.01250v2 Announce Type: replace Abstract: Large language model inference serving is a fast-growing electricity load whose power dynamics remain uncharacterized from a grid-planning perspective. Using real, measured GPU power traces, we show that chunked prefill scheduling, a latency-motivated technique already deployed by default in production LLM serving, is a controllable knob that regulates power ramp rate without touching peak power. Contrary to the intuitive hypothesis that splitting a long prompt's computation into smaller steps should flatten its power spike, peak power stays relatively the same while mean ramp rate falls substantially. Critically, this ramp-rate benefit is not a fixed property of the policy: it grows monotonically with system saturation, and we confirm this along two independent axes: concurrency and long-prompt request load (from near zero at low whale incidence to 39.6% at high whale fraction/size). We translate this single-GPU mechanism into an operational grid quantity, regulation-reserve procurement, posed and solved as a chance-constrained problem using a model-free bootstrap directly resampling real measured power traces. For independently operating servers, chunked prefill reduces the fast-ramping reserve requirement by an estimated 24% at heavy load (95% CIs within 13-34%) and 12% at moderate load, consistently across reliability levels from 95% to 99.9%. Repeated with 100 trials per policy on two-GPU tensor-parallel deployments, the reduction is about 36% for a dense 14B model and 10% for a sparse mixture-of-experts model. The benefit holds for ramps resolved at up to 2 s and persists at a reduced 16-20% when servers' load changes are synchronized. Together, these results identify a no-cost lever for fast-timescale demand shaping that is available today, whose benefit is largest when data centers run hottest.
Source:https://arxiv.org/abs/2608.01250PreviousProgrammable Sequential Memory in Coupled Oscillator NetworksNextStochastic MPC under Heavy-Tailed Disturbances: An Extreme Value Theory Approach
Related posts
I Had a Perfect Sprint Board. 18 AI Subagents Still Wrecked the Repo.
0dev.to GitHub
What Is Attribution? AI Music Needs a Trail Back to the People Who Made the Work
0dev.to News
A Free Google Alerts Alternative: Monitoring Global News With the GDELT DOC API
0dev.to News
Butter Scraped Over Too Much Bread
0dev.to Discuss
From Six Seven to Six Zeros to What Next?
0dev.to Discuss
When a document assistant refuses: quota, context, and the missing model call
0dev.to C#