From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications

Ajay Kumar Jaiswal; Yifan Wang; Lu Yin; Shiwei Liu; Runjin Chen; Jiawei Zhao; Ananth Grama; Yuandong Tian; Zhangyang Wang

2025 ICML ICML 2025

From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Ajay Kumar Jaiswal, Yifan Wang, Lu Yin, Shiwei Liu, Runjin Chen, Jiawei Zhao, Ananth Grama, Yuandong Tian, Zhangyang Wang

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

hessian analysis fine tuning gradient subspace low rank compression welore

Download PDF