AI & ML
Fast Gauss Sums via Flash Attention
arXiv cs.NA0 views
arXiv:2609.04910v2 Announce Type: replace-cross Abstract: Gauss kernel sums are the computational core of maximum mean discrepancies (MMDs), kernel gradient flows, Stein variational gradient descent (SVGD), and many other kernel methods. At the same time, softmax attention has received an extraordinary amount of hardware-aware code engineering, culminating in FLASH attention. We show that Gauss kernel sums with arbitrary, signed weights can be evaluated via FLASH attention: two small input augmentations turn the normalized softmax reduction into the unnormalized Gauss sum, without writing a single line of custom GPU code. For feature dimension $D>8$ in fp16, this approach beats compiled PyTorch code in speed and memory, and PyKeOps kernels in speed and accuracy at comparable or lower memory. Its memory scaling remains linear.
Source:https://arxiv.org/abs/2609.04910PreviousA Volume of Fluid Immersed Boundary Method for Industrial Polymer MixingNextQuasilinear multiplication in the Cayley-Dickson tower
Related posts
Why an Embedded Game Fails on Some Phones, and How to Debug It
0dev.to
Microsoft 365 vs Office 2024: The Break-Even Math
0dev.to
Master AI Evals: Build Reliable, Scalable Model Evaluation Pipelines
0dev.to
What to give someone who misses a place
0dev.to
Google is pushing OEMs against GrapheneOS and bootloader unlocking
0Hacker News
VESTA: Visual Exploration with Statistical Tool Agents
0arXiv stat.CO