adam optimizer

How Do Adam and Training Strategies Help BNNs Optimization ICML 2021 MTAdam: Automatic Balancing of Multiple Training Loss Terms EMNLP 2021 On the Adequacy of Untuned Warmup for Adaptive Optimization AAAI 2021 On the distributional properties of adaptive gradients UAI 2021 The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks ICML 2021 Adam with Bandit Sampling for Deep Learning NeurIPS 2020 Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descent IJCAI 2020 Bayesian filtering unifies adaptive and non-adaptive neural network optimization methods NeurIPS 2020 SAdam: A Variant of Adam for Strongly Convex Functions ICLR 2020 Towards Better Generalization of Adaptive Gradient Methods NeurIPS 2020 Towards Theoretically Understanding Why Sgd Generalizes Better Than Adam in Deep Learning NeurIPS 2020 A Sufficient Condition for Convergences of Adam and RMSProp CVPR 2019 HyperAdam: A Learnable Task-Adaptive Adam for Network Training AAAI 2019 Memory Efficient Adaptive Optimization NeurIPS 2019 Nostalgic Adam: Weighting More of the Past Gradients When Designing the Adaptive Learning Rate IJCAI 2019 Three Mechanisms of Weight Decay Regularization ICLR 2019 Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients ICML 2018 L4: Practical loss-based stepsize adaptation for deep learning NeurIPS 2018