AI & ML
Annealed correlation and quenched noise sensitivity in biased deep neural networks
arXiv stat.ML0 views
arXiv:2610.27983v1 Announce Type: new Abstract: In earlier work by Jonasson, Steif and Zetterqvist (2023), fully connected deep neural networks with layers of width $n$, i.i.d.\ standard Gaussian weights, sign activations and no bias terms were shown to be annealed and quenched noise sensitive with respect to any joint probability distribution $\mathbb{Q}^{(n)}$ on the input $\boldsymbol{\omega}$ and its perturbation $\boldsymbol{\eta}$ satisfying a mild positive-correlation condition and $\mathbb{Q}^{(n)}(\boldsymbol{\eta} \in \{\boldsymbol{\omega},-\boldsymbol{\omega}\}) \to 0$. Here we add independent Gaussian biases of the natural order and show that quenched noise sensitivity is retained over a broad range of depths, indeed under the slightly weaker assumption that $\mathbb{Q}^{(n)}(\boldsymbol{\omega} = \boldsymbol{\eta}) \to 0$. Thus, subject to the positive-correlation condition, quenched noise sensitivity holds under the minimal requirement that the perturbed input is asymptotically unlikely to coincide with the original input. At the same time, the annealed covariance remains bounded away from zero and is entirely accounted for by fluctuations in the network-dependent output means.
Source:https://arxiv.org/abs/2610.27983PreviousAsk to Learn: Query-Based Collaborative Reinforcement LearningNextNear-Optimal Regret Bounds for UCB-Style Gaussian Process Bandits via Data Thinning
Related posts

What Arconatese 3-4 Pontedera tells us about Serie D - Girone D
0dev.to Watercooler
Your AI agent got an HTTP error. Should it retry?
0dev.to Discuss
Is AI dangerous, or just misunderstood?
0dev.to Discuss
Why we price a developer tool per fix instead of per seat
0dev.to Productivity
My LLM-Powered Sidekick for Painless PR Descriptions
0dev.to Productivity
Reward-Adaptive Backtracking GFlowNet: A Technique Worth Understanding
0dev.to Productivity