Sparse is Enough in Fine-tuning Pre-trained Large Language Models

Weixi Song; Zuchao Li; Lefei Zhang; Hai Zhao; Bo Du

2024 ICML ICML 2024

Sparse is Enough in Fine-tuning Pre-trained Large Language Models

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Weixi Song, Zuchao Li, Lefei Zhang, Hai Zhao, Bo Du

Topics

Machine Learning > Optimization & Theory > Bayesian Inference Machine Learning > Learning Types > Transfer Learning Artificial Intelligence > Core AI > Large Language Models

Keywords

sparse fine-tuning gradient based loss landscape oscillations pac-bayesian generalization error bound quasi-sparsity sift

Download PDF