conftrace_

second-order optimization

93 papers

Explore in graph

Co-occurring keywords

newton method (64) natural gradient (75) stochastic optimization (1060) hessian approximation (41) kronecker factorization (18) fisher information matrix (46) second-order method (38) neural network optimization (1293) neural network training (309) gradient descent (1144)

Papers

Second-Order Kernel Online Convex Optimization with Adaptive Sketching ICML 2017
Approximate Newton Methods and Their Local Convergence ICML 2017
Practical Gauss-Newton Optimisation for Deep Learning ICML 2017
A Kronecker-factored approximate Fisher matrix for convolution layers ICML 2016
A Multi-Batch L-BFGS Method for Machine Learning NIPS 2016
Efficient Second-Order Gradient Boosting for Conditional Random Fields AISTATS 2015
Optimizing Neural Networks with Kronecker-factored Approximate Curvature ICML 2015
Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix ICML 2015
Fast Second Order Stochastic Backpropagation for Variational Inference NIPS 2015
Second-Order Non-Stationary Online Learning for Regression JMLR 2015
Newton Greedy Pursuit: A Quadratic Approximation Method for Sparsity-Constrained Optimization CVPR 2014
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NIPS 2014
Least Squares Revisited: Scalable Approaches for Multi-class Prediction ICML 2014
Fast algorithms for sparse principal component analysis based on Rayleigh quotient iteration ICML 2013
Newton-Like Methods for Sparse Inverse Covariance Estimation NIPS 2012
Krylov Subspace Descent for Deep Learning AISTATS 2012
SGD-QN: Careful Quasi-Newton Stochastic Gradient Descent JMLR 2009
Periodic Step Size Adaptation for Single Pass On-line Learning NIPS 2009