Backend
IA-MoE: An Intensity-Aware Mixture of Experts for DNN Inference
arXiv cs.DC0 views
arXiv:2611.19669v2 Announce Type: replace Abstract: In edge-cloud collaborative inference, the layers of a deep neural network (DNN) are placed on edge and cloud virtual machines (VMs) for every request, while the edge load and the cross-cloud bandwidth change over time. We propose IA-MoE, an intensity-aware Mixture of Experts (MoE) framework that scores each layer-VM pair by end-to-end delay and energy and places the layers one at a time on the best-scoring VM. Such greedy placement is myopic when trained on per-layer labels: offloading to the cloud slows the first offloaded layer while the gain appears only in later layers, so the policy keeps too many requests at the edge. Training on the remaining request time after each decision removes this myopia and reduces the mean delay by 12-56% on congested workloads. In CloudSimSDN simulations with three DNNs and two public request traces, IA-MoE has the lowest mean delay of all compared methods on congested VGG16 workloads, across cross-cloud bandwidths of 150-600 Mbps, fluctuating bandwidth and saturated edge servers. A random forest trained on the same label is 5-27% slower, and only a heuristic that sends whole requests to the cloud uses less energy, at up to 20 times the delay. When the edge is lightly loaded, offloading does not pay and simple heuristics perform as well. IA-MoE also beats a Double Deep Q-Network (DQN) agent with the same features in all 19 tested settings, with a mean delay lower by a factor of 1.3 to 10.5, and trains in minutes instead of hours.
Source:https://arxiv.org/abs/2611.19669PreviousWhy Kubernetes Cannot Contain a Full Lazy-Pulling Cache: Two Impossibility Results and a Bounded, Honest Design for Model ServingNextFlux: Optimal Scheduling of Optical Circuit Switches for LLM Training
Related posts
Our Guard Checked That We Still Talked About the Fix, Not That We Still Did It
0dev.to

How to Build an Invoice App with Next.js and Firebase Using Claude (With Prompts Included)
0dev.to
One calendar URL, five subscribe links, and the Outlook one Microsoft has never documented
0dev.to
The Hidden Engineering Challenge Behind Smart Construction Sites
0dev.to
WooCommerce Orders Stuck on Pending Payment: How to Fix
0dev.to
WordPress Staging Site: How to Test Changes Safely
0dev.to