AI & ML
eFAIR: Fair LLM Serving under Hourly Electricity Commitments
arXiv cs.DC0 views
arXiv:2611.24119v1 Announce Type: new Abstract: Datacenters can procure electricity in day-ahead markets by committing to energy purchases for individual hours. For LLM services operating within these hourly budgets, maximizing aggregate throughput alone may still leave individual requests waiting or receiving much slower service than desired. Fair serving therefore requires improving service for poorly served requests while reserving electricity for later arrivals whose timing and demand are unknown. To compare service across requests, we define normalized throughput as each request's actual output rate divided by the product of its standalone throughput and its service entitlement, a positive weight representing its desired service level. Using this measure, we formulate fair serving as leximin optimization under an hourly electricity constraint, prioritizing requests with the lowest normalized throughput. To address this problem, we present eFAIR, an online scheduler that coordinates request dispatching, electricity budget allocation, and GPU frequency selection through a common normalized-throughput target. Specifically, request dispatching prioritizes requests by desired service level and progress; electricity budget allocation reserves electricity for current and expected future work; and frequency selection uses electricity savings to raise GPU frequencies for requests with higher desired service levels. Evaluations on H20 and H200 configurations, each with 32 replicas, show that eFAIR improves the tenth percentile of normalized throughput by 28.0%-31.0% over the best budget-feasible baseline, while keeping absolute deviations from the hourly electricity commitment below 0.26%. eFAIR also improves throughput balance among concurrent requests and aligns electricity allocation more closely with service demand over time. Together, these results demonstrate how eFAIR improves request-level fairness within an hourly commitment.
Source:https://arxiv.org/abs/2611.24119PreviousLEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge DevicesNextMultilevel Graph Partitioning on GPU Supercomputers
Related posts
How Music LLMs Actually Work: From Tokens to Waves (MusicGen, DiT, and Building Your Own Engine)
0dev.to Machine Learning
蒸馏 Lora RHLF 都是什么意思
0dev.to Machine Learning

Teaching an offline plant doctor to spot rust
0dev.to Machine Learning
Is Your Model Good Enough? Precision vs Recall
0dev.to Machine Learning
How to Trim Audio for a Starlit Night Video — Free, in Your Browser
0dev.to Tutorial
How to Write Numbers in Words on Birth Certificates and Identity Documents
0dev.to Tutorial