edge of stability

Does SGD really happen in tiny subspaces? ICLR 2025 Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult JMLR 2025 Learning Dynamics of Deep Matrix Factorization Beyond the Edge of Stability ICLR 2025 Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos ICLR 2025 A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks JMLR 2024 Sharpness-Aware Minimization and the Edge of Stability JMLR 2024 A new characterization of the edge of stability based on a sharpness measure aware of batch gradient distribution ICLR 2023 Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond ICML 2023 Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability NeurIPS 2023 Learning threshold neurons via edge of stability NeurIPS 2023 Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width NeurIPS 2023 SAM operates far from home: eigenvalue regularization as a dynamical phenomenon ICML 2023 Second-order regression models exhibit progressive sharpening to the edge of stability ICML 2023 (S)GD over Diagonal Linear Networks: Implicit bias, Large Stepsizes and Edge of Stability NeurIPS 2023 Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory NeurIPS 2023 Understanding Edge-of-Stability Training Dynamics with a Minimalist Example ICLR 2023 Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability NeurIPS 2022 Understanding Gradient Descent on the Edge of Stability in Deep Learning ICML 2022 Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NeurIPS 2022 Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability ICLR 2021