conftrace_

deep learning optimization

12 papers

Explore in graph

Co-occurring keywords

neural network optimization (1293) deep learning (2118) neural network training (309) adaptive gradient method (28) stochastic gradient descent (1091) adam optimizer (54) convergence guarantee (169) gradient descent (1144) model parallelism (43) momentum method (75)

Papers

ADOPT: Modified Adam Can Converge with Any $\beta_2$ with the Optimal Rate NIPS 2024
Revisit the Power of Vanilla Knowledge Distillation: from Small Scale to Large Scale NIPS 2023
Fixed-Weight Difference Target Propagation AAAI 2023
AdaBlock: SGD with Practical Block Diagonal Matrix Adaptation for Deep Learning AISTATS 2022
A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022
To update or not to update? Neurons at equilibrium in deep models NIPS 2022
On the Acceleration of Deep Learning Model Parallelism With Staleness CVPR 2020
Ouroboros: On Accelerating Training of Transformer-Based Language Models NIPS 2019
A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI AAAI 2019
On The Power of Curriculum Learning in Training Deep Networks ICML 2019
Generalized Batch Normalization: Towards Accelerating Deep Neural Networks AAAI 2019
BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning CVPR 2018