AI & ML
Is $\sqrt{d}$ Separation Necessary for Gradient EM to Learn Gaussian Mixtures in High Dimensions?
arXiv stat.ML0 views
arXiv:2610.07551v1 Announce Type: new Abstract: Learning Gaussian mixture models (GMMs) using the Expectation-Maximization (EM) algorithm and its gradient-based variants is a fundamental problem in machine learning. It is known that randomly initialized (gradient) EM fails to learn multi-component GMMs in the exact-parameterized setting, where the number of components matches that of the ground-truth GMM. Recently, global convergence of gradient EM has been established in the over-parameterized setting, where more components are used, provided that the ground-truth components are well separated. In particular, the minimum separation between ground-truth components is required to scale as $\Omega(\sqrt{d})$, where $d$ is the dimension. In this paper, we show that this dimensional dependence is unavoidable in high-dimensional settings. Specifically, we consider a hybrid EM algorithm that uses standard EM updates for the mixing weights and gradient EM updates for the component means. For any $\epsilon > 0$, we prove that when the dimension is sufficiently large, in the worst case a separation of order $\Omega(d^{0.5-\epsilon})$ is insufficient to guarantee global convergence of population gradient EM in sub-exponential time under random initialization, even in the over-parameterized regime. Our result establishes an almost optimal worst-case lower bound on the ground-truth separation required for learning Gaussian mixtures via gradient EM in high dimensions.
Source:https://arxiv.org/abs/2610.07551PreviousTwo-Sample Testing for Random Graphs without Vertex CorrespondenceNextExplicit Asymptotic Bounds for Sequential Calibration Beyond $T^{2/3}$
Related posts
HR manager LinkedIn About section examples you can adapt
0dev.to Career
🚀 Ready to explore the world of AWS Cloud and build something amazing together? ☁️
0dev.to Programming
The Second Scrape Should Be Almost Free
0dev.to Programming
Updates to Authtics Advisories
0dev.to Programming
My betting model looked great for 3,671 predictions. It never beat the bookmaker.
0dev.to TypeScript

Build efficient agents with knowledge distillation on Amazon SageMaker AI
0AWS Machine Learning Blog