Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning

Libin Zhu; Chaoyue Liu; Adityanarayanan Radhakrishnan; Mikhail Belkin

2024 ICML ICML 2024

Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Mathematics & Optimization

Authors

Libin Zhu, Chaoyue Liu, Adityanarayanan Radhakrishnan, Mikhail Belkin

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Mathematics & Optimization > Optimization > Stochastic Methods Deep Learning > Learning Types > Representation Learning

Keywords

tangent kernels batch size sgd agop catapults training loss spikes

Download PDF