Skip to content
conftrace_
2024 ICML ICML 2024

Flora: Low-Rank Adapters Are Secretly Gradient Compressors

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Yongchang Hao, Yanshuai Cao, Lili Mou

Topics

Machine Learning > Optimization & Theory > Neural Network Optimization Deep Learning > Architectures > Neural Networks

Keywords

gradient compression low-rank adaptation high-rank updates optimization states projection matrices sublinear space complexity
Download PDF