Neural Network Optimization

Learning ReLUs via Gradient Descent NeurIPS 2017 Less than a Single Pass: Stochastically Controlled Stochastic Gradient AISTATS 2017 meProp: Sparsified Back Propagation for Accelerated Deep Learning with Reduced Overfitting ICML 2017 Natasha: Faster Non-Convex Stochastic Optimization via Strongly Non-Convex Parameter ICML 2017 Noisy Softmax: Improving the Generalization Ability of DCNN via Postponing the Early Softmax Saturation CVPR 2017 On the Flatness of Loss Surface for Two-layered ReLU Networks ACML 2017 Optimal Rates for Multi-pass Stochastic Gradient Methods JMLR 2017 Preventing Gradient Explosions in Gated Recurrent Units NeurIPS 2017 Sobolev Training for Neural Networks NeurIPS 2017 Train longer, generalize better: closing the generalization gap in large batch training of neural networks NeurIPS 2017 Adaptive Newton Method for Empirical Risk Minimization to Statistical Accuracy NeurIPS 2016 Asynchronous Parallel Learning for Neural Networks and Structured Models with Dense Features COLING 2016 Deep Residual Learning for Image Recognition CVPR 2016 Dynamic Network Surgery for Efficient DNNs NeurIPS 2016 Fast Algorithms for Convolutional Neural Networks CVPR 2016 Generalization Properties and Implicit Regularization for Multiple Passes SGM ICML 2016 Globally Optimal Training of Generalized Polynomial Neural Networks with Nonlinear Spectral Methods NeurIPS 2016 HyperNet: Towards Accurate Region Proposal Generation and Joint Object Detection CVPR 2016 Learning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks CVPR 2016 Network Morphism ICML 2016 Optimal Architectures in a Solvable Model of Deep Networks NeurIPS 2016 Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization NeurIPS 2016 Refining Architectures of Deep Convolutional Neural Networks CVPR 2016 Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters ICML 2016 SEBOOST - Boosting Stochastic Learning Using Subspace Optimization Techniques NeurIPS 2016