AI & ML
Optimal Deep Nonparametric Regression with Uniformly Bounded Lipschitz Constants
arXiv stat.ML0 views
arXiv:2611.23979v1 Announce Type: new Abstract: Stability under input perturbations is necessary for deep neural network estimators in many applications involving noisy covariates or estimated features. Weight norm constraints are widely used to control sensitivity to input perturbations by bounding the Lipschitz constant of neural networks. However, existing approximation results under fixed bounds on standard norm products are insufficient to establish optimal statistical rates. In this paper, we establish a sharp approximation theory for deep ReLU networks under a split spectral norm product constraint. This theoretical result shows that sufficiently smooth H\"{o}lder functions can be uniformly approximated to arbitrary accuracy as the network depth increases, with the norm-product bound held fixed. We then propose a penalized least squares estimator for deep nonparametric regression. The resulting estimator has a uniformly bounded Lipschitz constant while still achieving the optimal nonparametric rate up to a logarithmic factor. As applications, we establish optimal prediction rates for two-stage learning and provide robustness guarantees under adversarial input perturbations. Numerical studies show that the penalized estimator has well-controlled sensitivity to input perturbations and prediction accuracy comparable to that of classical deep network methods.
Source:https://arxiv.org/abs/2611.23979PreviousImproved Leverage Score Sampling for Constrained Active Linear RegressionNextReliFlow: Reliability-Calibrated Residual Transport for Combinatorial Single-Cell Perturbation Prediction
Related posts
Razorpay webhook not verified: anyone can fake a payment
0dev.to
From FAQPage Schema to Answerable AI Documentation
0dev.to
Wiring Android's MediaPipe LLM Inference API to a Multi-Turn Conversation Engine
0dev.to
What Is LLM Throughput? Tokens per Second, Concurrency, and Batching
0dev.to
Diese Woche haben mir zwei Tools etwas vorgeschlagen und es direkt abgeschickt.
0dev.to
AI Agent Observability: What to Log Before Production
0dev.to