AI & ML
GPUSparse: Fast and Exact Learned Sparse Retrieval on a GPU-Resident Inverted Index
arXiv cs.DC0 views
arXiv:2606.26441v2 Announce Type: replace-cross Abstract: Learned sparse models such as SPLADE match the quality of dense bi-encoders and keep the exact-match behavior of lexical retrieval. They are served on CPUs, using inverted-index traversal algorithms such as WAND. These algorithms are fast because they prune: they skip documents that cannot enter the top-k. Pruning makes the result approximate, and its control flow is sequential and datadependent, which suits a CPU and not a GPU. We argue that a GPU should not run a faster traversal. It should remove traversal altogether. A GPU has enough memory bandwidth to score every posting, so exactness need not be traded for speed. We present GPUSparse, which keeps the inverted index in GPU memory and scores queries with one fused scatter-add kernel. Nothing is pruned. GPUSparse searches all 8.8M MS MARCO passages in 1.27 ms per query, at 787 queries per second on one H100. Its results are exact: Recall@10, @100 and @1000 against an exhaustive FP32 reference are all 1.000, and quality at full scale matches Pyserini's exact SPLADE retrieval to three decimals. Exactness costs hardware rather than latency. Under core pinning, one H100 is worth about seven CPU cores. GPUSparse is 5.6x faster than one core and 2.8x slower than a 32-core socket. A CPU socket is therefore cheaper for approximate serving. The GPU is the better choice when exactness is required, where GPUSparse is 3.0x faster than exact CPU block-max pruning. We also decompose GPU latency. Traversal takes 55 to 80% of it and selection 18 to 38%, and the split moves with scale, which reconciles three conflicting published reports.
Source:https://arxiv.org/abs/2606.26441PreviousTileMaxSim: Roofline Regimes and Compression Limits for Fused GPU MaxSim ScoringNextHow Much Reconstruction Does Quantum Machine Learning Need? Late Fusion of Uncoupled Quantum Subcircuits
Related posts
SaaS Partner Scout: A Script for Identifying GitHub SaaS Project Contributors
0dev.to GitHub
WARN Act and Retaliation — Can Your Employer Punish You for Asking About a Layoff Notice?
0dev.to Data
WARN Act and Mental Health: What a Layoff Notice Does to Your Wellbeing, and Where to Get Support
0dev.to Data
Dimensionare un pallettizzatore dal lato del progettista: i tre numeri che non stanno nelle brochure
0dev.to Design
How to Choose the Best Stretch Tent Supplier for Your Next Purchase
0dev.to Design
How to spend three slow days in a city you have never visited
0dev.to Productivity