conftrace_

2022 AAAI AAAI 2022

A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract)

Abstract

Abstract Incorporating curvature information in stochastic methods has been a challenging task. This paper proposes a momentum accelerated BFGS quasi-Newton method in both its full and limited memory forms, for solving stochastic large scale non-convex optimization problems in neural networks (NN).

🌉 Interdisciplinary Bridge - Artificial Intelligence and Mathematics & Optimization

Authors

S. Indrapriyadarsini, Shahrzad Mahboubi, Hiroshi Ninomiya, Takeshi Kamio, Hideki Asai

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Machine Learning > Optimization & Theory > Optimization Deep Learning > Techniques > Model Architecture Mathematics & Optimization > Optimization > Optimization Deep Learning > Optimization & Theory > Neural Network Optimization Deep Learning > Optimization & Theory > Optimization

Keywords

stochastic optimization neural network training quasi-newton method momentum method momentum acceleration neural network bfgs algorithm limited memory optimization

More from AAAI 2022

All AAAI 2022 papers →

Class-Wise Adaptive Self Distillation for Federated Learning on Non-IID Data (Student Abstract)

Detecting Neighborhood Gentrification at Scale via Street Views and POIs (Student Abstract)

A Discriminative and Robust Feature Learning Approach for EEG-Based Motor Imagery Decoding (Student Abstract)

AsyncFL: Asynchronous Federated Learning Using Majority Voting with Quantized Model Updates (Student Abstract)

Code Representation Learning Using Prüfer Sequences (Student Abstract)

Gerrymandering under Uncertain Preferences (Student Abstract)