AI & ML
TileMaxSim: Roofline Regimes and Compression Limits for Fused GPU MaxSim Scoring
arXiv cs.DC0 views
arXiv:2606.26439v2 Announce Type: replace-cross Abstract: MaxSim scoring dominates the cost of multi-vector retrieval. Several groups built fused GPU kernels for it independently during 2026, and all report one measure of success: the fraction of peak memory bandwidth reached. That measure assumes the operator is limited by the bytes it moves. We show the assumption fails in two ways, and that two other quantities decide performance: which side of the roofline ridge a shape falls on, and how many dependent gathers the scoring path performs. We first locate the boundary. Fused arithmetic intensity is Nq*Nd /(Nq + Nd) FLOP per byte, against an H100 dense-FP16 ridge of 295.4. The operator is therefore memory-bound at ColBERT shapes and compute-bound at ColPali-scale query lengths. Published analyses report one side each and do not reconcile them. We measure the crossover at three document lengths, and a closed form predicts it inside the measured bracket each time. Past the crossover, percent of peak bandwidth falls to 4% while percent of peak FLOPs rises above 40%. The reported measure is therefore defined on only one side of a boundary that real workloads cross. We also validate the underlying traffic model against DRAM counters, to 0.003%. We then show that moving fewer bytes does not make the kernel faster. Fusing product-quantized asymmetric distance computation into the kernel moves 15.9x fewer bytes than exact FP16 on tensor cores and runs 3.0x slower. Counters rule out the usual causes: it reaches 1.23% of peak DRAM throughput at 98.2% occupancy. The limit is 6.5*10^9 dependent gathers, a count that does not fall when the codes get smaller. A hand-written CUDA version with a staged table recovers 4.7x and remains slower than exact scoring. Finally, we give the first head-to-head comparison of three fused MaxSim kernels, over 28 shapes, with every result checked against an exhaustive FP32 reference.
Source:https://arxiv.org/abs/2606.26439PreviousAMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention ServingNextGPUSparse: Fast and Exact Learned Sparse Retrieval on a GPU-Resident Inverted Index
Related posts
I Launched My Company Website, Made Task Monitor Free, and Opened Custom Project Requests
0dev.to AI
How I turn Hacker News, GitHub and npm into product ideas every 3 hours
0dev.to AI
Lenovo's 'Encrypted' Fingerprint USB Drive Is an SD Card. How to Check Yours
0dev.to Linux

Email: OpenAI plans to merge ChatGPT's Chat and Work tabs on desktop, rolling out to select users before a wider release; Codex remains a separate experience (Ivan Mehta/TechCrunch)
0Techmeme

Memo: Alphabet is issuing €6B in euro-denominated debt across seven tranches, attracting €17B+ of demand; the company sold €9B of bonds earlier in 2026 (Reuters)
0Techmeme

S-1: Anthropic reports Q3 revenue up 12x YoY to nearly $17B, a net loss of $23.3B, up from $12.1B in Q3 2025, and operating expenses up 292% to $17.5B (Bailey Lipschultz/Bloomberg)
0Techmeme