General
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
arXiv stat.TH0 views
arXiv:2607.24041v3 Announce Type: replace Abstract: Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
Source:https://arxiv.org/abs/2607.24041PreviousA proximal gradient algorithm for composite log-concave samplingNextAsymptotics and finite sample bounds for prediction and smoothing in Wright-Fisher hidden Markov models
Related posts
Vellinko – Sign PDFs in the browser without uploading them
0Hacker News New
Show HN: Vivio House, an interactive 3D headquarters
0Hacker News New
Pushover – Simple Notifications
0Hacker News New
The Gift of It's Your Problem Now (2021)
0Hacker News New
Implied News: News X Prediction Markets
0Hacker News New
Show HN: OpenDocBot - bring your own model to Word, Excel and PowerPoint
0Hacker News New