AI & ML
Lipschitz-Constrained Discriminators in Generative Modeling: Asymptotic Behavior and Support Mismatch
arXiv stat.ML0 views
arXiv:2611.18394v1 Announce Type: new Abstract: We study the asymptotic behavior of optimal discriminators associated to $L-$Lipschitz regularized KL-divergences of two probability distributions $Q$ and $P$ that interpolate between the rescaled Wasserstein$-1$ metric by some parameter $L>0$ and the KL divergence. The parameter $L$ plays the role of a Lipschitz constraint and is viewed as a regularization parameter: small values of $L$ enforce a more geometry-aware discriminator, while as $L$ increases the geometric regularization is progressively relaxed. It is a known result [3] that such divergences converge to the KL divergence as $L\to\infty$. In this paper, we characterize the asymptotic behavior of the optimal discriminators as $L\to\infty$ in different regimes: Firstly, under $Q\ll P$ and the Radon-Nikodym derivative $\frac{dQ}{dP}$ is bounded from above and bounded away from zero. Secondly, when $Q$ and $P$ are discrete distributions satisfying $Q\ll P$, and finally when $Q$ and $P$ are non absolutely continuous discrete distributions. This result motivates training strategies which involve discriminators, such as adversarial generative models. In practice, the data distribution is often represented by a discrete empirical measure, while, especially at the early stages of the training, the model distribution may have little or no overlap with the empirical data distribution.
Source:https://arxiv.org/abs/2611.18394PreviousLaMe: a Joint Latent Mediation Framework for High-Dimensional SystemsNextMesh-Free Weak-Form Regression for Learning Interaction Kernels from Particle Trajectories
Related posts
The King’s Trust and OpenAI launch £1m partnership
0OpenAI Blog

AI Skills and the Gender Gap: Who Gets Ahead in the Changing Job Market?
0HackerNoon

AI in Leadership: Managing People, Not Just Work
0HackerNoon
Neuroscience: What is Human Intelligence in the Brain?
0HackerNoon
How to Prevent Silent Failures in Multi-Agent AI Systems
0HackerNoon
I Created an AI Cat That Thinks for Herself. Now I Have More Questions Than Answers
0HackerNoon