Neural Network Optimization

Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs CVPR 2021 Optimizing Deeper Transformers on Small Datasets ACL 2021 Orthogonal Over-Parameterized Training CVPR 2021 Particle Dual Averaging: Optimization of Mean Field Neural Network with Global Convergence Rate Analysis NeurIPS 2021 Penalty Method for Inversion-Free Deep Bilevel Optimization ACML 2021 PML: Progressive Margin Loss for Long-Tailed Age Classification CVPR 2021 Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization ICML 2021 Prioritized Architecture Sampling With Monto-Carlo Tree Search CVPR 2021 ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks CVPR 2021 ReconfigISP: Reconfigurable Camera Image Processing Pipeline ICCV 2021 Reconsidering the Past: Optimizing Hidden States in Language Models EMNLP 2021 Regularizing Neural Networks via Adversarial Model Perturbation CVPR 2021 Rethinking BiSeNet for Real-Time Semantic Segmentation CVPR 2021 Rethinking Channel Dimensions for Efficient Model Design CVPR 2021 Reverse engineering learned optimizers reveals known and novel mechanisms NeurIPS 2021 Revisiting Weight Initialization of Deep Neural Networks ACML 2021 ReZero is all you need: fast convergence at large depth UAI 2021 RMSprop converges with proper hyper-parameter ICLR 2021 SA-BNN: State-Aware Binary Neural Network AAAI 2021 SBO-RNN: Reformulating Recurrent Neural Networks via Stochastic Bilevel Optimization NeurIPS 2021 Scale-Aware Automatic Augmentation for Object Detection CVPR 2021 Scaling-Up Robust Gradient Descent Techniques AAAI 2021 Second-Order Neural ODE Optimizer NeurIPS 2021 Slot Machines: Discovering Winning Combinations of Random Weights in Neural Networks ICML 2021 Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction NeurIPS 2021