Neural Network Optimization

ProxSGD: Training Structured Neural Networks under Regularization and Constraints ICLR 2020 PsuedoProp at SemEval-2020 Task 11: Propaganda Span Detection Using BERT-CRF and Ensemble Sentence Level Classifier SemEval 2020 Pure and Spurious Critical Points: a Geometric Study of Linear Networks ICLR 2020 Quantization Aware Training with Absolute-Cosine Regularization for Automatic Speech Recognition Interspeech 2020 Random Reshuffling: Simple Analysis with Vast Improvements NeurIPS 2020 Rational neural networks NeurIPS 2020 Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate NeurIPS 2020 Reformer-TTS: Neural Speech Synthesis with Reformer Network Interspeech 2020 Regularization of Distinct Strategies for Unsupervised Question Generation EMNLP 2020 Regularized Context Gates on Transformer for Machine Translation ACL 2020 Regularized linear autoencoders recover the principal components, eventually NeurIPS 2020 Regularizing Neural Networks via Minimizing Hyperspherical Energy CVPR 2020 Reinforcing Neural Network Stability with Attractor Dynamics AAAI 2020 Reparameterizing Mirror Descent as Gradient Descent NeurIPS 2020 Rethinking Skip Connection with Layer Normalization COLING 2020 Retiarii: A Deep Learning Exploratory-Training Framework OSDI 2020 RNNs Incrementally Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients? ICLR 2020 Robust Recovery via Implicit Bias of Discrepant Learning Rates for Double Over-parameterization NeurIPS 2020 Scheduled DropHead: A Regularization Method for Transformer Models EMNLP 2020 ScopeFlow: Dynamic Scene Scoping for Optical Flow CVPR 2020 Search Tree Pruning for Progressive Neural Architecture Search (Student Abstract) AAAI 2020 Second-Order Information in Non-Convex Stochastic Optimization: Power and Limitations COLT 2020 SelectScale: Mining More Patterns from Images via Selective and Soft Dropout IJCAI 2020 Self-Distillation Amplifies Regularization in Hilbert Space NeurIPS 2020 Self-Orthogonality Module: A Network Architecture Plug-in for Learning Orthogonal Filters WACV 2020