AI & ML
Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning
arXiv stat.ML0 views
arXiv:2610.32849v1 Announce Type: cross Abstract: During joint training, backpropagation allows later layers to reshape earlier representations, a feedback mechanism absent from purely layer-wise learning. We extend Neural Low-Degree Filtering (Neural LoFi), a recently proposed layer-wise spectral surrogate for feature learning, to capture this interaction. Starting from the exact backpropagation gradients, we derive a local coupling between a neuron's incoming weights and its corresponding next-layer column. In a controlled small-initialization regime with smooth activations and fixed surroundings, these dynamics amplify singular modes of a cross-layer correlation operator. Each growing mode pairs an earlier feature with an upper-layer response through shared neuron coefficients. This motivates a backward-coupling algorithm in which upper-layer sensitivities generate sample-dependent effective labels for earlier spectral steps, while corrected features refine upward transmission. A function-space variational principle characterizes the selected feature-response pairs by maximizing the label correlation of their product under kernel-norm constraints. For three-layer networks, we derive self-consistent continuous-time dynamics for the filtered hidden-layer features, the activation kernels, and the network output under the proposed backward coupling. In a hierarchical multi-index model, we establish a feature-recovery separation: backward coupling recovers lower-layer directions invisible to the original forward label-weighted statistic. The linear specialization recovers residual-dependent singular-mode dynamics, while paired modes provide a dynamical mechanism for linked low-rank covariance components consistent with the rainbow-network picture. Experiments with fully connected and convolutional networks on binary CIFAR-10 show improved accuracy over forward Neural LoFi at matched widths.
Source:https://arxiv.org/abs/2610.32849PreviousGeneralization properties of the token perceptron: the role of embedding and vocabularyNextMixed Engression: Distributional Regression for Multivariate Mixed-Type Responses
Related posts

Here’s how to listen live to Apple’s Q4 2026 earnings call
09to5Mac
Can AI Write CAD? Testing LLM-Generated 3D Models for Manufacturability
0HackerNoon
Guardz Integrates Palo Alto Networks Prisma Browser Protection to Address Growing Workspace Security
0HackerNoon

LinearB Launches Agentic Studio to Steer AI Coding Agents in a $127 Billion Market by 2032
0HackerNoon
We Built a Test to Catch Contact Forms That Lie. It Lied to Us Five Times First
0HackerNoon

The TechBeat: Why Knowledge Graphs and Ontologies Are Making a Comeback in the Age of AI (11/2/2026)
0HackerNoon