AI & ML
A Transport Geometry of Benign Overfitting
arXiv stat.TH0 views
arXiv:2604.08625v4 Announce Type: replace-cross Abstract: Benign overfitting, the phenomenon that interpolating noisy data can be statistically harmless, is usually characterized through effective-rank conditions on the covariance spectrum. These conditions are fragile: a perturbation of vanishing relative operator and Frobenius norm can turn a benign covariance into one whose minimum-norm interpolator has non-vanishing excess risk. We argue that the natural geometry of benign overfitting is optimal transport of the log-eigenvalues, measured in units of the sample size $n$. In log-spectral coordinates, the deterministic equivalent of the risk becomes a functional of two measures smoothed by a logistic kernel centred at the interpolation threshold $\log\kappa$. We compute its exact first variation: the threshold moves by a sensitivity-weighted average of the log-eigenvalue displacements, and the influence of each eigenvalue decays exponentially with its log-distance to the threshold. We then derive global, dimension-free stability bounds. The variance functional is $8/(27n)$-Lipschitz in the 1-Wasserstein distance between log-spectra, with a sharp constant, and is controlled more generally by a threshold-localized transport discrepancy; an analogous bound controls the bias. Consequently, any rank-$k$ update of the covariance, of arbitrary norm, changes the variance functional by at most $3k/n$, and benign overfitting survives every perturbation whose localized transport cost is $o(n)$. The results extend to ridge regression and to infinite-dimensional trace-class covariances. Matching lower bounds show that the $n$-scaling is necessary and cannot be replaced by operator norms, Frobenius norms, or dimension-normalized spectral distances. Simulations, one-step feature learning in two-layer networks, and random features of real images support the theory and delineate its limits.
Source:https://arxiv.org/abs/2604.08625PreviousTiny but uniform improvements of adaptive BH procedures via compound e-valuesNextCellwise and Casewise Robust Multivariate Regression with Inference
Related posts

10 Open-Source Robotics Repositories for 3D Perception, VLA Policies, and Sensor Calibration
0dev.to
My self-modifying system tried to wire a gate-passing module into its own protection enforcer. It was refused three times.
0dev.to
Last night an AI agent performed brain surgery on a mouse
0Hacker News
The Card-Update Page Gets One Visit. Design for That Visit.
0dev.to UX
SaaS Partner Scout: A Script for Identifying GitHub SaaS Project Contributors
0dev.to GitHub
WARN Act and Retaliation — Can Your Employer Punish You for Asking About a Layoff Notice?
0dev.to Data