Neural Network Optimization

Nonlinear Residual Echo Suppression Using a Recurrent Neural Network Interspeech 2020 NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm (Extended Abstract) IJCAI 2020 NUIG: Multitasking Self-attention based approach to SigTyp 2020 Shared Task EMNLP 2020 Ode to an ODE NeurIPS 2020 O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers NeurIPS 2020 On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent ICML 2020 On Correctness of Automatic Differentiation for Non-Differentiable Functions NeurIPS 2020 On Infinite-Width Hypernetworks NeurIPS 2020 On Second Order Behaviour in Augmented Neural ODEs NeurIPS 2020 On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems NeurIPS 2020 On the Computational Power of Transformers and Its Implications in Sequence Modeling CoNLL 2020 On the Convergence of Nesterov’s Accelerated Gradient Method in Stochastic Settings ICML 2020 On the Convergence of SARAH and Beyond AISTATS 2020 On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms AISTATS 2020 On the distance between two neural networks and the stability of learning NeurIPS 2020 On the Loss Landscape of Adversarial Training: Identifying Challenges and How to Overcome Them NeurIPS 2020 On the Regularization Properties of Structured Dropout CVPR 2020 On the Sub-layer Functionalities of Transformer Decoder EMNLP 2020 On the training dynamics of deep networks with L_2 regularization NeurIPS 2020 On Warm-Starting Neural Network Training NeurIPS 2020 Optimal Lottery Tickets via Subset Sum: Logarithmic Over-Parameterization is Sufficient NeurIPS 2020 Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions NeurIPS 2020 Optimization Learning: Perspective, Method, and Applications IJCAI 2020 Optimization Theory for ReLU Neural Networks Trained with Normalization Layers ICML 2020 Optimizing Mode Connectivity via Neuron Alignment NeurIPS 2020