AI & ML
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
arXiv stat.ML0 views
arXiv:2606.13796v2 Announce Type: replace Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution. Existing theoretical works bound finite-round error accumulation in the context of diffusion models, but two questions remain open:~what distribution does the recursion converge to, and how fast? We answer both, isolating a mechanism distinct from imperfect learning: even with perfect score estimation and exact sampling, the early stopping of the reverse diffusion (required for numerical stability) drives a progressive drift away from the data distribution. We prove that this recursion converges geometrically to a unique limiting distribution, which admits a closed-form characterization as an infinite mixture of increasingly Gaussian-smoothed versions of the data distribution. A Hermite spectral decomposition of this limit reveals that recursive training acts as a low-pass filter: higher-order modes, which encode fine non-Gaussian structure, are attenuated much more strongly than coarse modes. This spectral picture motivates annealed truncation schedules that progressively shrink truncation times across retraining rounds; we prove that any schedule converging to $0$ asymptotically eliminates recursive compounding. Finally, we show our idealized characterization is robust: in the presence of discretization and score estimation errors, the learned distribution remains in a Wasserstein-2 ball around the ideal limit, with mode-dependent contraction rates that contract high-order errors faster than low-order ones. We validate the theory on synthetic Gaussian mixtures and CIFAR-10.
Source:https://arxiv.org/abs/2606.13796PreviousTight Generalization Bounds for Noiseless Inverse OptimizationNextDiffusion-Driven State Space Models
Related posts
Microsoft's Azure Linux 3.0.2022 Ships With More Than 470 Security Fixes
0Phoronix

How to watch Leeds vs Tottenham: Live streams & TV channels for early kick-off in Premier League
0TechRadar
A dead star's mysterious rings reveal what could happen when our Sun dies
0Science Daily Tech
Rubin Observatory discovers two hidden companions of the Milky Way. Hundreds more may be waiting
0Science Daily Tech

How I Switched: a human-approved change desk for AI agents
0dev.to
Your favourite name is taken. Do not ruin it trying to save it
0dev.to