Neural Network Optimization

Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent NeurIPS 2011 Nearest Neighbor based Greedy Coordinate Descent NeurIPS 2011 Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning NeurIPS 2011 Super-Linear Convergence of Dual Augmented Lagrangian Algorithm for Sparsity Regularized Estimation JMLR 2011 An analysis on negative curvature induced by singularity in multi-layer neural-network learning NeurIPS 2010 Erratum: SGDQN is Less Careful than Expected JMLR 2010 Parallelized Stochastic Gradient Descent NeurIPS 2010 Smoothness, Low Noise and Fast Rates NeurIPS 2010 Understanding the difficulty of training deep feedforward neural networks AISTATS 2010 Accelerated Gradient Methods for Stochastic Optimization and Online Learning NeurIPS 2009 A Generalized Natural Actor-Critic Algorithm NeurIPS 2009 A Smoothed Approximate Linear Program NeurIPS 2009 Convergent Temporal-Difference Learning with Arbitrary Smooth Function Approximation NeurIPS 2009 Efficient Learning using Forward-Backward Splitting NeurIPS 2009 Periodic Step Size Adaptation for Single Pass On-line Learning NeurIPS 2009 SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent JMLR 2009 Deep Learning with Kernel Regularization for Visual Recognition NeurIPS 2008 Learning to classify complex patterns using a VLSI network of spiking neurons NeurIPS 2007 Noise Tolerant Variants of the Perceptron Algorithm JMLR 2007 Nonlinear Boosting Projections for Ensemble Construction JMLR 2007 One-Pass Boosting NeurIPS 2007 On higher-order perceptron algorithms NeurIPS 2007 Topmoumoute Online Natural Gradient Algorithm NeurIPS 2007 An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models NeurIPS 2006 A Very Fast Learning Method for Neural Networks Based on Sensitivity Analysis JMLR 2006