AI & ML
Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning
arXiv stat.ML0 views
arXiv:2611.22158v1 Announce Type: cross Abstract: Self-Supervised Learning (SSL) aims to align augmented views while preserving representation diversity and preventing collapse. Existing non-contrastive methods often address these goals using multiple weighted loss terms. We introduce Centroid Spectral Coupling (CSC), a single-term SSL objective that implicitly couples view alignment, representation diversity, and collapse resistance. CSC constructs centroids from independently normalized augmented-view pairs and applies a ridge-based spectral objective to their uncentered second-moment matrix. Preserving centroid magnitudes encodes view agreement while encouraging balanced representation energy across spectral directions. We establish an exact decomposition into nonnegative alignment and spectral-imbalance components, characterize the geometry of global optima, and derive finite-loss guarantees for alignment and diversity. We also show that complete constant-output collapse is a strict saddle in normalized representation space. Experiments on CIFAR-10, CIFAR-100, and Imagenette show that CSC achieves the highest linear Top-1 accuracy among the evaluated non-contrastive methods on all three benchmarks, and the highest kNN Top-1 accuracy on both CIFAR datasets. A ridge-parameter sensitivity study reveals an empirical alignment--diversity trade-off: increasing $\kappa$ strengthens view alignment but reduces representation effective rank. These results show that a unified spectral objective yields competitive visual representations without explicit contrastive negatives or separately weighted alignment and diversity penalties.
Source:https://arxiv.org/abs/2611.22158PreviousCOSI: A Framework for Robustly Valid InferenceNextTraining Neural Networks to be Locally Adaptive
Related posts

Nvidia board faction angling for Jensen Huang return as CEO, report claims
0Tom's Hardware
New discovery reveals what may be holding back room-temperature superconductors
0Science Daily Tech
NASA pushes Mars helicopter blades beyond the speed of sound
0Science Daily Tech
How to make AI data centres more sustainable
0Nature News

The UK and some other countries don’t get more Siri AI usage with iCloud+
09to5Mac

MacPaw’s Moonlock comes to iPhone, helping users detect potential scams
09to5Mac