AI & ML
Request-Adaptive Execution for Efficient Hybrid LLM Inference
arXiv cs.DC0 views
arXiv:2610.37432v1 Announce Type: new Abstract: Hybrid large language models (LLMs) have emerged as a promising way to combine the modeling capability of the Attention mechanism with the efficiency of state space models (SSMs). However, most existing hybrid LLMs still inherit a static design philosophy: once the Attention-to-SSM layout is chosen, every request must follow the same hybrid execution. This one-layout-fits-all design is inefficient for real-world serving workloads, where different requests rely on contextual information in different ways. This paper presents Morph, a serving system that co-designs the model architecture and runtime for request-adaptive hybrid LLM inference. Its model architecture pairs Attention and SSM layers within each block and uses prefill-oriented gating to select a request-specific layout that remains fixed throughout decoding. Its serving runtime preserves batching efficiency under divergent layer choices through selective batching and enables correct prefix reuse through hybrid-aware context caching. Experiments on six datasets show that Morph achieves a better quality-efficiency trade-off than existing static hybrid baselines. System-level evaluation further shows that Morph improves the serving throughput by up to 3.87x over baselines while substantially reducing inference latency.
Source:https://arxiv.org/abs/2610.37432PreviousTAO: Tool-Aware Joint Scheduling for LLM-Based Agentic ServingNextQuality-Driven Energy-Efficient Big Data Aggregation in WBANs
Related posts
The Autonomous Edge — Issue #13: Cheaper Agents, Harder Governance (Week of November 2, 2026)
0dev.to AI

Plano Físico vs Plano Funcional en AWS Networking: cómo los arquitectos senior resuelven lo que otros no ven
0dev.to AWS
We built Origami Tutor: free paper-craft lessons you can turn and replay in 3D
0dev.to WebDev
From two authorized photos to a short AI rap video (consent-first workflow)
0dev.to WebDev
Check a File Hash in the Browser, Without Uploading the File
0dev.to WebDev
Your AI keys were leaked at a platform. A local guard you can paste catches the abuse before the invoice.
0dev.to Security