AI & ML
Recursively Trained Diffusion Models: Limiting Collapse Distribution and Spectral Characterization
arXiv stat.ML0 views
arXiv:2606.13796v2 Announce Type: replace Abstract: Recursive training of generative models on their own outputs can lead to model collapse, a compounding drift away from the true data distribution. Existing theoretical works bound finite-round error accumulation in the context of diffusion models, but two questions remain open:~what distribution does the recursion converge to, and how fast? We answer both, isolating a mechanism distinct from imperfect learning: even with perfect score estimation and exact sampling, the early stopping of the reverse diffusion (required for numerical stability) drives a progressive drift away from the data distribution. We prove that this recursion converges geometrically to a unique limiting distribution, which admits a closed-form characterization as an infinite mixture of increasingly Gaussian-smoothed versions of the data distribution. A Hermite spectral decomposition of this limit reveals that recursive training acts as a low-pass filter: higher-order modes, which encode fine non-Gaussian structure, are attenuated much more strongly than coarse modes. This spectral picture motivates annealed truncation schedules that progressively shrink truncation times across retraining rounds; we prove that any schedule converging to $0$ asymptotically eliminates recursive compounding. Finally, we show our idealized characterization is robust: in the presence of discretization and score estimation errors, the learned distribution remains in a Wasserstein-2 ball around the ideal limit, with mode-dependent contraction rates that contract high-order errors faster than low-order ones. We validate the theory on synthetic Gaussian mixtures and CIFAR-10.
Source:https://arxiv.org/abs/2606.13796PreviousTight Generalization Bounds for Noiseless Inverse OptimizationNextDiffusion-Driven State Space Models
Related posts
Stripe doesn't open accounts in 37 of the 85 countries we checked. Here are the legal ways to get paid anyway
0dev.to
Do AI Models Care About Each Other? Anthropic's Multi-Agent Welfare Study Says Yes — With a Catch
0dev.to
Building VARYN: Why Permission Boundaries Matter More Than Another AI API Call
0dev.to
App Store 2026 in numbers: AI, pricing, abandoned apps
0Hacker News
ASUS Launches New Zenbook 14 in India With Ceraluminum Design and OLED Display
0FOSS Bytes
TOP BEARINGS SRL conduce în Intermedieri produse diverse Argeș
0dev.to Data