AI & ML
Unifying Central Flow, Rod Flow, and the Edge Coupling
arXiv stat.ML0 views
arXiv:2611.08358v1 Announce Type: new Abstract: At the edge of stability, the iterates of gradient descent oscillate along the sharp directions of the Hessian. This poses a significant challenge for continuous-time models as the usual continuous-time approximation -- the gradient flow -- admits no oscillations. Several continuous-time models have been proposed to capture the behavior at the edge of stability, among them central flow and rod flow. Although both models can be interpreted as performing a coarse-graining in time, the connection between them remains poorly understood. Here we show that both flows can be derived from a third object, the edge coupling: a functional on consecutive iterates whose critical points are the fixed points and period-two orbits of gradient descent. It is revealed that the two flows correspond to coarse-grainings at different time scales. Rod flow coarse-grains over only the fastest time scale, while central flow coarse-grains over both the fast and intermediate time scales. Furthermore, using the edge coupling, we derive rod flows and central flows for momentum methods such as heavy ball and Adam. We verify the derived flows experimentally on representative neural network architectures, where they track the discrete iterates throughout all phases of training -- including the transition from the stable regime to the edge of stability.
Source:https://arxiv.org/abs/2611.08358PreviousSeries-wise Cross-validation for Global Time Series ForecastingNextDouble Machine Learning for Panel Data: Identification, Specification, and a Bank Regulation Effect
Related posts

Supercharge Your GPUs: Managing AI Clusters with Slurm
0dev.to AI
From Prompt to Blueprint: My Workflow with an AI Floor Planner and an Image-to-DWG Converter
0dev.to AI
Building AI in India: My Journey with GodEngine.ai and AskShiva.ai
0dev.to AI
AI agents are your new customers. I built a checker to see which stores they can actually read
0dev.to AI
VibeVoice: Turn an Hour of Audio Into a Full Transcript in One Go
0dev.to AI
I built an app with ChatGPT or Claude. How do I put it online, with a database and logins?
0dev.to WebDev