AI & ML
ARK: Avoiding Routing Collisions for KV Cache Transfer in Disaggregated LLM Inference
arXiv cs.DC0 views
arXiv:2610.41160v1 Announce Type: cross Abstract: Disaggregated LLM inference separates the prefill and decode phases across GPU pools, generating massive KV-cache transfers. Because these transfers last hundreds of milliseconds to seconds, they behave as mega elephant flows that dominate link bandwidth utilization. In this regime, stateless ECMP can perform poorly: hash collisions may overload one spine link while leaving others idle, stretching transfer times by seconds. Yet this same persistence makes coordination practical. Since these flows are long-lived, even lightweight one-to-all coordination can be amortized over their lifetime. We present ARK, a distributed elephant-flow path reservation mechanism. ARK coordinates senders to choose source ports whose hashes map concurrent flows onto distinct spines, without requiring switch changes or receiver-side packet reordering. Packet-level RDMA simulations show that ARK reduces mean and P95 FCT by up to 27.3% and 34.0% under moderate load, and further reduces mean TTFT by up to 12.9%.
Source:https://arxiv.org/abs/2610.41160PreviousZenKO: Guided and Iterative Quantum-Classical Method for Molecular Electronic Structure using Sample-Based Quantum DiagonalizationNextTIRx: A Unified Tile Primitive ML Compiler Across Heterogeneous Backends
Related posts

Copilot Fusion's Local Model Preview: Only One of Three Stages Runs on Your Machine
0dev.to AI

Meituan shipped LongCat-2.5
0dev.to AI
Standard bf16 Rounding Discards 99% of Your RL Policy Updates
0dev.to AI
Why Are Startups Choosing Coworking Space in Pune in 2026?
0dev.to AI
What a 32GB GPU actually lets you run locally
0dev.to AI

LLM weak random number generator detection test
0dev.to Security