conftrace_

second-order optimization

93 papers

Explore in graph

Co-occurring keywords

newton method (64) natural gradient (75) stochastic optimization (1060) hessian approximation (41) kronecker factorization (18) fisher information matrix (46) second-order method (38) neural network optimization (1293) neural network training (309) gradient descent (1144)

Papers

A Mini-Block Fisher Method for Deep Neural Networks AISTATS 2023
Second-Order Optimization with Lazy Hessians ICML 2023
A Computationally Efficient Sparsified Online Newton Method NIPS 2023
Curvature-Aware Training for Coordinate Networks ICCV 2023
Optimal Shrinkage for Distributed Second-Order Optimization ICML 2023
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022
Second-Order Sensitivity Analysis for Bilevel Optimization AISTATS 2022
Gradient Descent on Neurons and its Link to Approximate Second-order Optimization ICML 2022
Combining (Second-Order) Graph-Based and Headed-Span-Based Projective Dependency Parsing ACL 2022
Symmetry Teleportation for Accelerated Optimization NIPS 2022
Adaptive Second Order Coresets for Data-efficient Machine Learning ICML 2022
FedNL: Making Newton-Type Methods Applicable to Federated Learning ICML 2022
Enhancing Adversarial Training With Second-Order Statistics of Weights CVPR 2022
The Hessian Screening Rule NIPS 2022
Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods NIPS 2022
THOR, Trace-based Hardware-driven Layer-Oriented Natural Gradient Descent Computation AAAI 2021
LocalNewton: Reducing communication rounds for distributed learning UAI 2021
SKFAC: Training Neural Networks With Faster Kronecker-Factored Approximate Curvature CVPR 2021
Tensor Normal Training for Deep Learning Models NIPS 2021
Tractable structured natural-gradient descent using local parameterizations ICML 2021
A Trace-restricted Kronecker-Factored Approximation to Natural Gradient AAAI 2021
Analytic Insights into Structure and Rank of Neural Network Hessian Maps NIPS 2021
On the Variance of the Fisher Information for Deep Learning NIPS 2021
An Inertial Newton Algorithm for Deep Learning JMLR 2021
SONIA: A Symmetric Blockwise Truncated Optimization Algorithm AISTATS 2021