conftrace_

fisher matrix

6 papers

Explore in graph

Co-occurring keywords

second-order optimization (93) natural gradient (75) second-order method (38) kronecker product (33) gradient preconditioning (3) neural network (6616) kronecker factorization (18) empirical fisher (4) second order optimizer (1) block diagonal preconditioning (1)

Papers

A Mini-Block Fisher Method for Deep Neural Networks AISTATS 2023

KrADagrad: Kronecker approximation-domination gradient preconditioned stochastic optimization UAI 2023

Tensor Normal Training for Deep Learning Models NIPS 2021

A Kronecker-factored approximate Fisher matrix for convolution layers ICML 2016

Scaling up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix ICML 2015

Natural Neural Networks NIPS 2015