second-order optimization

Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularization NeurIPS 2020 Distributed Newton Can Communicate Less and Resist Byzantine Workers NeurIPS 2020 New Insights and Perspectives on the Natural Gradient Method JMLR 2020 SPAN: A Stochastic Projected Approximate Newton Method AAAI 2020 Stochastic Subspace Cubic Newton Method ICML 2020 What Deep CNNs Benefit From Global Covariance Pooling: An Optimization Perspective CVPR 2020 WoodFisher: Efficient Second-Order Approximation for Neural Network Compression NeurIPS 2020 Cubic Regularization with Momentum for Nonconvex Optimization UAI 2019 Curvature-Exploiting Acceleration of Elastic Net Computations ICML 2019 EA-CG: An Approximate Second-Order Method for Training Fully-Connected Neural Networks AAAI 2019 Hessian Aided Policy Gradient ICML 2019 Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks CVPR 2019 Limitations of the empirical Fisher approximation for natural gradient descent NeurIPS 2019 Preconditioner on Matrix Lie Group for SGD ICLR 2019 The Common-directions Method for Regularized Empirical Risk Minimization JMLR 2019 Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization NeurIPS 2018 Exact natural gradient in deep linear networks and its application to the nonlinear case NeurIPS 2018 Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis NeurIPS 2018 Guide Actor-Critic for Continuous Control ICLR 2018 Kronecker-factored Curvature Approximations for Recurrent Neural Networks ICLR 2018 Approximate Newton Methods and Their Local Convergence ICML 2017 Diving into the shallows: a computational perspective on large-scale shallow learning NeurIPS 2017 Fast Black-box Variational Inference through Stochastic Trust-Region Optimization NeurIPS 2017 Practical Gauss-Newton Optimisation for Deep Learning ICML 2017 Second-Order Kernel Online Convex Optimization with Adaptive Sketching ICML 2017