AI & ML
Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models
arXiv stat.ML0 views
arXiv:2610.08420v1 Announce Type: cross Abstract: We establish a polynomial sample complexity separation between symmetry-aware and symmetry-agnostic feature learning. We study growing-rank multi-index models with high-dimensional Gaussian covariates in $\mathbb{R}^d$ and $r=\Theta(d^\delta)$ teacher directions forming a cyclic symmetry orbit, where $0<\delta<1/2$. We compare three ways of exploiting this structure: architectural weight sharing, data augmentation over the full symmetry group, and learning without access to the symmetry. In particular, we analyze a symmetry-tied convolutional network, an untied network, and the same untied network trained with full-group data augmentation, using spherical online SGD with correlation loss. For a class of polynomial links with information exponent $p\ge3$, we prove matching sample complexity bounds up to logarithmic factors: the tied and augmented learners achieve weak directional recovery in $\widetilde{\Theta}(d^{p-1})$ samples, whereas the symmetry-agnostic learner requires $\widetilde{\Theta}(rd^{p-1})$. For the pure quadratic Hermite link, the same separation holds for weak recovery of the teacher subspace, with sample complexities $\widetilde{\Theta}(d)$ and $\widetilde{\Theta}(rd)$, respectively. Thus, full-group data augmentation matches the sample efficiency of architectural weight sharing, and both provide a polynomial advantage over training without symmetry. For $p\ge3$, the proof reveals a two-stage mechanism: fluctuations at initialization select one direction in the teacher orbit, after which localized growth amplifies its overlap to the weak recovery scale while competing overlaps remain near their initialization scale.
Source:https://arxiv.org/abs/2610.08420PreviousSpectra: Exact Component Transport for Test-Time Prior Adaptation in Simulation-Based InferenceNextReinforcement Learning for Hierarchical Reasoning Rewards: Minimax-Optimal Rates with Transformers
Related posts
Launch a Business for Under $500: Real Odds, 4 Founders
0dev.to AI
Google, Microsoft, OpenAI and Anthropic Back a Shared AI Training Opt-Out Signal
0dev.to AI
Did AI Decode the Voynich Manuscript? Claims Tested
0dev.to AI
AIGB 解决 Teletype.in 群组管理痛点
0dev.to AI

Qual a melhor arquitetura de dados para a sua aplicação de IA?
0dev.to AI
Connect a Stateless Streamable HTTP MCP Server to Claude, ChatGPT, and Gemini for Tour Route Research
0dev.to WebDev