Neural Network Optimization

Taming the Wild: A Unified Analysis of Hogwild-Style Algorithms NeurIPS 2015 The Loss Surfaces of Multilayer Networks AISTATS 2015 Training Very Deep Networks NeurIPS 2015 Adaptivity of Averaged Stochastic Gradient Descent to Local Strong Convexity for Logistic Regression JMLR 2014 A Deep and Tractable Density Estimator ICML 2014 A Differential Equation for Modeling Nesterov’s Accelerated Gradient Method: Theory and Insights NeurIPS 2014 A Synaptical Story of Persistent Activity with Graded Lifetime in a Neural System NeurIPS 2014 Dropout: A Simple Way to Prevent Neural Networks from Overfitting JMLR 2014 Fast Stochastic Alternating Direction Method of Multipliers ICML 2014 GeNGA: A Generalization of Natural Gradient Ascent with Positive and Negative Convergence Results ICML 2014 Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NeurIPS 2014 Learning by Stretching Deep Networks ICML 2014 Learning Polynomials with Neural Networks ICML 2014 Max-Margin Infinite Hidden Markov Models ICML 2014 Natural Evolution Strategies JMLR 2014 Online and Stochastic Gradient Methods for Non-decomposable Loss Functions NeurIPS 2014 On the Computational Efficiency of Training Neural Networks NeurIPS 2014 Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization NeurIPS 2014 Proximal Quasi-Newton for Computationally Intensive L1-regularized M-estimators NeurIPS 2014 SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives NeurIPS 2014 Stochastic Proximal Gradient Descent with Acceleration Techniques NeurIPS 2014 Accelerated Mini-Batch Stochastic Dual Coordinate Ascent NeurIPS 2013 Accelerating Stochastic Gradient Descent using Predictive Variance Reduction NeurIPS 2013 Adaptive dropout for training deep neural networks NeurIPS 2013 Adaptive Step-Size for Policy Gradient Methods NeurIPS 2013