AI & ML
PageWeaver: KV-Guided Query Unions for Sparse Attention
arXiv cs.DC0 views
arXiv:2610.11201v1 Announce Type: new Abstract: Dynamic sparse attention limits the KV pages selected by each query, but a small support does not necessarily yield efficient GPU work. Query unions share page loads and populate Tensor Core tiles; their cost depends on which queries are grouped together. We present PageWeaver, an execution design that uses selected-page affinity to assemble query groups while preserving each query's original support and complete output ownership. A bounded GPU search produces query IDs, and an ID-aware two-CTA kernel consumes them without materializing reordered Q tensors or cross-page partial outputs. A direct KV-page union implementation provides a complementary design study of nonlocal reuse and reduction cost. With FP8 KV throughout, the H200 Union8 implementation achieves a 1.70x geometric-mean complete-call speedup over the measured FlashInfer path on six captures. Online regrouping further lowers latency by 3.26-7.66% on five selected 64K-context captures. Whole-model prefill throughput is 7.88-14.36% above the tested native path; the incremental regrouping benefit is smaller, with observed median gains of 0.47-0.73% at 32K/64K and regressions at 8K. A B300 comparison identifies cases where preparation cost and a stronger native kernel remove the advantage. These results separate execution-group reuse from the complete cost of exploiting it online.
Source:https://arxiv.org/abs/2610.11201PreviousZepp: Accelerating Distributed MoE Serving under Relaxed Balance ConstraintsNextLLM Agents as Resilience Engineers for Scientific Applications
Related posts
Most voted crypto projects on Coinvote, Oct 26 to Nov 1, 2026
0dev.to Data
Bluesky This Week — trending topics & AI brand mentions (2026-11-02)
0dev.to Data
Gemini 4 Carbon e DeepSeek V4.1 Flash Cyber: a Guerra Cibernética Autônoma e o Topo do Artificial Analysis
0dev.to Machine Learning
What can an AI agent do for you every day?
0dev.to Machine Learning
Mon modèle de paris semblait excellent sur 3 671 prédictions. Il n'a jamais battu le bookmaker.
0dev.to Machine Learning
HR manager LinkedIn About section examples you can adapt
0dev.to Career