conftrace_

second-order optimization

93 papers

Explore in graph

Co-occurring keywords

newton method (64) natural gradient (75) stochastic optimization (1060) hessian approximation (41) kronecker factorization (18) fisher information matrix (46) second-order method (38) neural network optimization (1293) neural network training (309) gradient descent (1144)

Papers

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning AAAI 2021
Second Order Techniques for Learning Time-series with Structural Breaks AAAI 2021
Second-Order Neural ODE Optimizer NIPS 2021
M-FAC: Efficient Matrix-Free Approximations of Second-Order Information NIPS 2021
Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update NIPS 2021
Curvature-corrected learning dynamics in deep neural networks ICML 2020
What Deep CNNs Benefit From Global Covariance Pooling: An Optimization Perspective CVPR 2020
New Insights and Perspectives on the Natural Gradient Method JMLR 2020
WoodFisher: Efficient Second-Order Approximation for Neural Network Compression NIPS 2020
Stochastic Subspace Cubic Newton Method ICML 2020
Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization NIPS 2020
SPAN: A Stochastic Projected Approximate Newton Method AAAI 2020
Distributed Newton Can Communicate Less and Resist Byzantine Workers NIPS 2020
Hessian Aided Policy Gradient ICML 2019
The Common-directions Method for Regularized Empirical Risk Minimization JMLR 2019
Curvature-Exploiting Acceleration of Elastic Net Computations ICML 2019
Cubic Regularization with Momentum for Nonconvex Optimization UAI 2019
EA-CG: An Approximate Second-Order Method for Training Fully-Connected Neural Networks AAAI 2019
Limitations of the empirical Fisher approximation for natural gradient descent NIPS 2019
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks CVPR 2019
Exact natural gradient in deep linear networks and its application to the nonlinear case NIPS 2018
Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis NIPS 2018
Second-Order Stochastic Optimization for Machine Learning in Linear Time JMLR 2017
Fast Black-box Variational Inference through Stochastic Trust-Region Optimization NIPS 2017
Diving into the shallows: a computational perspective on large-scale shallow learning NIPS 2017