Backend
Simultaneous analysis of approximate leave-one-out cross-validation and mean-field inference II: General convex penalties
arXiv stat.TH0 views
arXiv:2611.21877v1 Announce Type: new Abstract: For smooth penalties and group Lasso, analyzed approximate leave-one-out (ALO) cross-validation by comparing its weights with the trace correction used in mean-field inference. For a $C^2$ penalty $R$, the matrix entering these corrections is $A=[\nabla^2R(\widehat b)+X^TDX]^{-1}$, where $\widehat b$ is the full-data estimator and $D$ is the diagonal matrix of fitted loss curvatures; for group Lasso, the penalty and loss curvatures are first restricted to the active support before taking the pseudoinverse. The obstacle to extending this argument to general convex penalties was the lack of an explicit formula for the matrix $A$ entering these corrections. For every finite-valued strongly convex penalty, we construct such a matrix from ordinary derivatives of the penalty response at the observed subgradient. Its projection--pseudoinverse formula separates the response space and penalty curvature, fixed by conditioning, from the explicit design dependence. This supplies the missing ingredient to extend the probabilistic argument of Part~I. The construction recovers the inverse Hessian for smooth penalties and the active-group inverse for group Lasso, including inactive boundary groups. In the proportional regime, for Gaussian designs with general positive-definite covariance and covariance-scaled strong convexity, we obtain unconditional bounds comparing both ALO and mean-field predictions with exact leave-one-out predictions in regression and single-index models. The expected average squared prediction error is $O(n^{-1})$, and the expected absolute criterion error is $O(n^{-1/2})$, under bounded, 1-Lipschitz loss scores and suitable test functions. The matrix and ALO weights use only observed data; the mean-field correction additionally uses the design covariance. The construction, recovery identities and statistical guarantees are formalized in Lean~4.
Source:https://arxiv.org/abs/2611.21877PreviousWhen Is Membership Graded? Identifiability and Resolution Limits of a Graded Gaussian Mixture ModelNextAnytime-valid inference for conditional hazard and survival functions with streaming data
Related posts
Autoscaling Adds Waiters, Not Ovens. Here's What Actually Breaks at 10x Load
0dev.to
[백엔드 모의면접] 중복 결제는 어떻게 막을까?
0dev.to
Server-Side Pagination Looked Easy. It Took Me Longer Than Any React Concept Yet.
0dev.to
Before You Create an AWS Account, Spend 45 Minutes Learning These Security Tools
0dev.to
Claude “Invalid Signature” in a Thinking Block: How to Debug It Safely
0dev.to
How to catch an expiring or broken SSL certificate before your visitors do
0dev.to