FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training

Philip Zmushko; Aleksandr Beznosikov; Martin Takac; Samuel Horváth

2025 ICML ICML 2025

FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Philip Zmushko, Aleksandr Beznosikov, Martin Takac, Samuel Horváth

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

low-rank adaptation gradient splitting memory efficient blockwise optimization sgd signsgd

Download PDF