AI & ML
Regularization vs. Convolution in Kernel-Based Hypothesis Testing
arXiv stat.ML0 views
arXiv:2611.06778v1 Announce Type: cross Abstract: The maximum mean discrepancy (MMD) test is a popular nonparametric test on general domains. Two lines of work have studied its optimality through structurally different approaches. The first, a convolution approach on $\mathbb{R}^d$, uses translation-invariant kernels with a shrinking bandwidth and is minimax optimal over Sobolev balls under $L^2$-separation; the second, a regularization approach on general measurable spaces, uses spectral regularization of the kernel covariance operator and is minimax optimal over Hellinger- and $\chi^2$-separated alternatives. In this paper, we study the regularization approach over Sobolev balls under $L^2$-separation, which reveals the connection between the two approaches. We show that, with a translation-invariant kernel on $\mathbb{R}^d$, both are ways of meeting a single condition on the kernel's spectral density: the spectral regularized discrepancy captures the $L^2$-separation $\Delta$ of a Sobolev alternative of smoothness $s$ as soon as the regularization parameter $\lambda$ is below the spectral density on frequencies up to $\Delta^{-1/(2s)}$. A kernel with a shrinking bandwidth $h$ meets this condition with $\lambda \asymp h^d$, and under this coupling the constant regularizer, which gives the MMD test itself, attains the Sobolev minimax rate. This explains, from a spectral perspective, the need for a shrinking bandwidth in the convolution approach. With a non-constant spectral regularizer, our analysis requires the bandwidth to shrink only when the spectral density decays faster than any polynomial, as for the Gaussian kernel. Thus, for Mat\'ern and Laplace kernels, a fixed bandwidth with $\lambda \to 0$ attains the Sobolev minimax rate. Finally, an adaptive test over a grid of bandwidths and regularization parameters is minimax optimal, up to a $\log\log$ factor, over both classes without prior knowledge of the smoothness.
Source:https://arxiv.org/abs/2611.06778PreviousFinite-width shift of the edge of chaos in deep neural networksNextFundamental Limits of Coherent Imaging in The Presence of Atmospheric Turbulence
Related posts
Using profiles of cognitive capability to assess AI suitability for workplace tasks
0arXiv cs.HC
Metamaterial tactile displays with programmable virtual pixels
0arXiv cs.HC
MAIA: Multi-Agent Intent Articulation for Requirement Discovery in Art Commissions
0arXiv cs.HC
Patients With Personality: Realistic Patient Simulation through Controlled Diversity and Selective Disclosure
0arXiv cs.HC
Rethinking the UI of GenUI: A Tale of Two Designs
0arXiv cs.HC
AromaGen: Interactive Generation of Rich Olfactory Experiences with Multimodal Language Models
0arXiv cs.HC