AI & ML
Neural Classifiers as Error-Correcting Codes
arXiv math.IT0 views
arXiv:2611.18239v1 Announce Type: cross Abstract: A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.
Source:https://arxiv.org/abs/2611.18239PreviousLinear Dynamic Range Extension for Rydberg Atomic Receiver via Band-Limited ReconstructionNextHard/Soft NLoS Detection via Combinatorial Data Augmentation for 6G Positioning
Related posts
Questions swirl around FAA's new SMART AI system, though officials say no link to ground stop
0Phys.org Tech
Hong Kong rolls out AI facial recognition across CCTV network
0Phys.org Tech
Amazon loses 40 satellites in failed rocket launch
0Phys.org Tech
Purism Aiming To Deliver New Librem 5 Smartphone With RISC-V SoC
0Phoronix

Nvidia amassed $530 billion in financial guarantees to corner the AI market — supply commitments rose by nearly a quarter of a trillion dollars in a single quarter
0Tom's Hardware
Daily briefing: Are one-person firms the future of biotech?
0Nature News