Neural Network Optimization

Improving Transformer Models by Reordering their Sublayers ACL 2020 Improving Transformer Optimization Through Better Initialization ICML 2020 Incorporating Noisy Length Constraints into Transformer with Length-aware Positional Encodings COLING 2020 Increasing Learning Efficiency of Self-Attention Networks through Direct Position Interactions, Learnable Temperature, and Convoluted Attention COLING 2020 Inefficiency of K-FAC for Large Batch Size Training AAAI 2020 Inference Strategies for Machine Translation with Conditional Masking EMNLP 2020 Information Geometry of Orthogonal Initializations and Training ICLR 2020 InstaNAS: Instance-Aware Neural Architecture Search AAAI 2020 Instead of Rewriting Foreign Code for Machine Learning, Automatically Synthesize Fast Gradients NeurIPS 2020 Interpolation Technique to Speed Up Gradients Propagation in Neural ODEs NeurIPS 2020 INTERSPEECH 2020 Deep Noise Suppression Challenge: A Fully Convolutional Recurrent Network (FCRN) for Joint Dereverberation and Denoising Interspeech 2020 Is normalization indispensable for training deep neural network? NeurIPS 2020 Joint Detection of Sentence Stress and Phrase Boundary for Prosody Interspeech 2020 Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout NeurIPS 2020 KungFu: Making Training in Distributed Machine Learning Adaptive OSDI 2020 Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks ICML 2020 Language Model Prior for Low-Resource Neural Machine Translation EMNLP 2020 Large Batch Optimization for Deep Learning: Training BERT in 76 minutes ICLR 2020 Learning Architectures from an Extended Search Space for Language Modeling ACL 2020 Learning compositional functions via multiplicative weight updates NeurIPS 2020 Learning Dynamic Routing for Semantic Segmentation CVPR 2020 Learning Graph Convolutional Network for Skeleton-Based Human Action Recognition by Neural Searching AAAI 2020 Learning Parities with Neural Networks NeurIPS 2020 Learning Rank-1 Diffractive Optics for Single-Shot High Dynamic Range Imaging CVPR 2020 Learning Rate Adaptation for Differentially Private Learning AISTATS 2020