second-order optimization

Extra-Newton: A First Approach to Noise-Adaptive Accelerated Second-Order Methods NeurIPS 2022 FedNL: Making Newton-Type Methods Applicable to Federated Learning ICML 2022 Gradient Descent on Neurons and its Link to Approximate Second-order Optimization ICML 2022 Second-Order Sensitivity Analysis for Bilevel Optimization AISTATS 2022 SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning ICLR 2022 Symmetry Teleportation for Accelerated Optimization NeurIPS 2022 The Hessian Screening Rule NeurIPS 2022 The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022 ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning AAAI 2021 Analytic Insights into Structure and Rank of Neural Network Hessian Maps NeurIPS 2021 An Inertial Newton Algorithm for Deep Learning JMLR 2021 A Trace-restricted Kronecker-Factored Approximation to Natural Gradient AAAI 2021 LocalNewton: Reducing communication rounds for distributed learning UAI 2021 M-FAC: Efficient Matrix-Free Approximations of Second-Order Information NeurIPS 2021 Newton-LESS: Sparsification without Trade-offs for the Sketched Newton Update NeurIPS 2021 On the Variance of the Fisher Information for Deep Learning NeurIPS 2021 Second-Order Neural ODE Optimizer NeurIPS 2021 Second Order Techniques for Learning Time-series with Structural Breaks AAAI 2021 SKFAC: Training Neural Networks With Faster Kronecker-Factored Approximate Curvature CVPR 2021 SONIA: A Symmetric Blockwise Truncated Optimization Algorithm AISTATS 2021 Tensor Normal Training for Deep Learning Models NeurIPS 2021 THOR, Trace-based Hardware-driven Layer-Oriented Natural Gradient Descent Computation AAAI 2021 Tractable structured natural-gradient descent using local parameterizations ICML 2021 Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization ICML 2021 Curvature-corrected learning dynamics in deep neural networks ICML 2020