Backend
Random Projections for Spectral Algorithms: Optimal Rates in Sobolev Norms
arXiv stat.ML0 views
arXiv:2610.30948v1 Announce Type: new Abstract: Random projections (RP) offer an effective method for reducing computational and storage overhead while preserving the geometric structure of data. However, existing studies have primarily focused on the optimal generalization performance of specific kernel-regularized algorithms with RP in the well-specified setting and under restrictive conditions. In this paper, we provide a comprehensive and improved analysis of the generalization performance of RP-based spectral algorithms under general conditions \change{without sacrificing statistical efficiency}. By leveraging the embedding property of the reproducing kernel Hilbert space (RKHS) and a refined analysis of the operator similarity, we establish optimal learning rates in Sobolev norms that match the minimax lower bounds up to logarithmic factors. For two common implementations of RP, randomized sketches and Nystr\"{o}m sub-sampling (uniform or leverage-based), we show that the projection dimension required for optimality is proportional to the average or maximal effective dimension, yielding a significant reduction in computational cost while maintaining the statistical efficiency. Our results do not rely on the uniform boundedness assumption on the regression function and are valid across a wide range of source conditions, i.e., $s> \alpha_0-1/\beta$, where $s,\beta$, and $\alpha_0$ denote the smoothness index, capacity index, and the embedding index, respectively. In the benign case when $\alpha_0=1/\beta$, the optimality holds for all $s\in (0,2\tau]$ with $\tau$ and $\gamma$ denoting the qualification index and norm index. Experimental results demonstrate both the theoretical validity and the practical effectiveness of the proposed method.
Source:https://arxiv.org/abs/2610.30948PreviousContinuous-depth Deep Gaussian ProcessesNextDeep Learning for Random Objects in Metric Spaces: Beyond Manifold Assumptions
Related posts
Ask HN: Who is hiring? (November 2026)
0Hacker News Ask

Is there a nicer syntax for `if let ... Some(destructred_field) else None`?
0Rust Users Forum
Convert YAML to JSON in the Browser (and JSON to YAML) Without Uploading the File
0dev.to
Vercel agent-desktop: computer use for coding agents through the accessibility tree
0dev.to
WARN Act and Job Search — How to Use the 60-Day Notice Period to Find Your Next Job
0dev.to
Node.js SMS Verification Code: Generating and Sending One-Time Passwords via API Integration
0dev.to