AI & ML
InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area Network
arXiv cs.DC0 views
arXiv:2610.32638v1 Announce Type: new Abstract: LLM inference is increasingly deployed as a core service for enterprise and interactive applications, but sovereignty, compliance, latency and resource-locality requirements can limit the suitability of centralised cloud deployments. Telco operators offer geographically distributed compute and network infrastructure for addressing these constraints. However, existing LLM serving frameworks are designed for tightly coupled data centres and provide limited support for coordinating geographically distributed and heterogeneous resources. We present InferSlice, a reference architecture for Service-Level Objective (SLO)-driven LLM inference over a telco infrastructure, integrating network slicing, GPU partitioning, elastic serving and observability. Led by BT in collaboration with the University of Bristol and NVIDIA, and realised on the JOINER UK national-scale wide-area network (WAN), InferSlice is evaluated on service differentiation across tiers, elastic scale-out, client-edge bandwidth as a bottleneck, and the reach of tensor, pipeline, data parallelism, and prefill/decode disaggregation over the WAN. Our results show that, on a general-purpose telco network, tiers yield distinct latency-throughput regimes, and weight streaming shortens scale-out time. Replication is placement-insensitive but memory-bound; pipeline parallelism tolerates near- and mid-cross-site hops, while tensor parallelism and disaggregation fail beyond a single subnet hop. We share practical insights from operating SLO-driven inference on a telco network, lessons learned and open questions, and hope to encourage further research on distributed and heterogeneous LLM inference systems.
Source:https://arxiv.org/abs/2610.32638PreviousToward Open ISA Supercomputing: A Comparative Study of RISC-V, Arm, and x86 Architectures on HPC WorkloadsNextBeyond Hotness: High-Resolution Memory Characterization of Cloud and LLM Workloads for Tiered Memory with ARM SPE
Related posts

Here’s how to listen live to Apple’s Q4 2026 earnings call
09to5Mac
Can AI Write CAD? Testing LLM-Generated 3D Models for Manufacturability
0HackerNoon
Guardz Integrates Palo Alto Networks Prisma Browser Protection to Address Growing Workspace Security
0HackerNoon

LinearB Launches Agentic Studio to Steer AI Coding Agents in a $127 Billion Market by 2032
0HackerNoon
We Built a Test to Catch Contact Forms That Lie. It Lied to Us Five Times First
0HackerNoon

The TechBeat: Why Knowledge Graphs and Ontologies Are Making a Comeback in the Age of AI (11/2/2026)
0HackerNoon