Open Source
Transient Amplification in Cascaded Gradient Methods: Exact Minimax Laws and Tracking without Resonance
arXiv stat.ML0 views
arXiv:2606.04031v3 Announce Type: replace-cross Abstract: Leader-follower gradient iterations, including bilevel and two-time-scale methods, can exhibit large finite-time excursions despite asymptotic stability. For cascaded iterations with block matrix J = [[A,0],[C,D]], symmetric diagonal blocks of norm at most gamma < 1, and coupling norm at most c, we prove an exact, dimension-free minimax formula: at every horizon t, the worst-case amplification equals the spectral norm of [[gamma^t,0],[c t gamma^(t-1),gamma^t]], attained by an aligned two-dimensional quadratic. Consequently, the worst transient is on the order of 1 + c/(1-gamma), and eigenvalues alone cannot predict finite-time gain. Under martingale noise, the aligned system has stationary follower variance sigma^2 c^2 (1+gamma^2)/(1-gamma^2)^3, establishing cubic inverse-gap scaling. For quadratic followers, the coupling is linked to follower sensitivity R by C = (I-D)R. When the leader is isotropic and the follower map is positive semidefinite, amplification is at most 1 plus the operator norm of R, independently of the spectral gaps. A look-ahead Newton update eliminates follower memory and is minimax-optimal among linear updates that track the leader exactly. Experiments validate the bounds and show that accelerated or inexact follower solvers can exceed the sensitivity bound when their effective maps have negative eigenvalues. Code and experiment scripts: https://github.com/Axym-Labs/transient-amplification
Source:https://arxiv.org/abs/2606.04031PreviousPOETS: Uncertainty-Aware LLM Optimization via Compute-Efficient Policy EnsemblesNextOnline Distributional Prediction via Latent Cluster Geometry Under Drift and Corruption
Related posts
Show HN: 3D pixel-art basketball game – no deps, no build step
0Hacker News New
How Android Apps Leak Device Identifiers Across the Native–Web Boundary
0Hacker News New
Show HN: The DNA of DOOM – How code structure survives across 44 ports
0Hacker News New
Ask HN: Freelancer? Seeking freelancer? (November 2026)
0Hacker News Ask

'America hates data centers' — New data highlights public sentiment
0Tom's Hardware
Anime Expeditions Codes (November 2026)
0FOSS Bytes