Optimization

RMSprop converges with proper hyper-parameter ICLR 2021 Robust Model Compression Using Deep Hypotheses AAAI 2021 Robust Optimization for Multilingual Translation with Imbalanced Data NeurIPS 2021 Scaling Up Exact Neural Network Compression by ReLU Stability NeurIPS 2021 Scaling-Up Robust Gradient Descent Techniques AAAI 2021 Sequence Length is a Domain: Length-based Overfitting in Transformer Models EMNLP 2021 SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs NeurIPS 2021 Shape Matters: Understanding the Implicit Bias of the Noise Covariance COLT 2021 SKFAC: Training Neural Networks With Faster Kronecker-Factored Approximate Curvature CVPR 2021 Slot Machines: Discovering Winning Combinations of Random Weights in Neural Networks ICML 2021 Smoothness Matrices Beat Smoothness Constants: Better Communication Compression Techniques for Distributed Optimization NeurIPS 2021 Speeding Up Transformer Training By Using Dataset Subsampling - An Exploratory Analysis EMNLP 2021 Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness ICML 2021 Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks NeurIPS 2021 Step-Ahead Error Feedback for Distributed Training with Compressed Gradient AAAI 2021 Stochastic Anderson Mixing for Nonconvex Stochastic Optimization NeurIPS 2021 Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence AISTATS 2021 Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks AAAI 2021 Tangent Space Backpropagation for 3D Transformation Groups CVPR 2021 Tensor Normal Training for Deep Learning Models NeurIPS 2021 The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks COLT 2021 The Implicit Bias for Adaptive Optimization Algorithms on Homogeneous Neural Networks ICML 2021 Tilting the playing field: Dynamical loss functions for machine learning ICML 2021 Time-Multiplexed Coded Aperture Imaging: Learned Coded Aperture and Pixel Exposures for Compressive Imaging Systems ICCV 2021 To Optimize, or Not to Optimize, That Is the Question: TelU-KU Models for WMT21 Large-Scale Multilingual Machine Translation EMNLP 2021