AI & ML
SULA: Robust Langevin Sampling via Drift-Diffusion Saturation
arXiv stat.CO0 views
arXiv:2610.19670v1 Announce Type: new Abstract: The unadjusted Langevin algorithm (ULA) and its stochastic-gradient variant, SGLD, are workhorses of modern Bayesian machine learning, but can be fragile when faced with a target distribution possessing a non-Lipschitz drift. In such settings, the expected squared jump distance can grow rapidly, leading to numerical instability or the need for prohibitively small step-sizes. We propose the saturated unadjusted Langevin algorithm (SULA) that pairs a saturated gradient drift with a gradient-informed state-dependent diffusion, while preserving first-order consistency with the Langevin diffusion. The drift and diffusion terms are chosen to guarantee that the one-step squared update is independent of the gradient. Consequently, SULA is robust to the choice of step-size and preserves estimation and prediction quality. We provide sufficient conditions under which SULA with a step-size $h$ converges weakly to the Langevin diffusion at the standard $O(h)$ rate, the SULA chain is geometrically ergodic, and that the bias of its invariant measure relative to the target is $O(h)$. Further, we develop SG-SULA, a stochastic-gradient version of SULA that rescales the stochastic gradient using a Gaussian CDF correction under Gaussianity of the gradient noise. Numerical experiments on various examples demonstrate SULA's robustness to tuning and SG-SULA's improved predictive performance.
Source:https://arxiv.org/abs/2610.19670PreviousAnalytically Optimal Sample Allocations for Nested Multivariate Multifidelity Control Variate EstimatorsNextTheoretical and Practical Advances in Hierarchical Bayesian Deep Learning Methodology
Related posts
Cloudflare DNS Zone Builder cuts peak memory 85% and halves build delay
0dev.to News
O dia em que a cliente pediu desconto num bolo de pistache (e pagou R$ 30 num cafe)
0dev.to Design
O telefone tocou as 19h: Por que cancelam seu contrato primeiro na crise?
0dev.to Design
Which Cursor model is best for everyday work?
0dev.to Productivity
Is AI going to get cheaper or more expensive?
0dev.to Machine Learning
Điểm tin Trí tuệ Nhân tạo & Mô hình Ngôn ngữ lớn (LLMs) tại Việt Nam (03/11/2026)
0dev.to Machine Learning