LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning

Zihang Liu; Tianyu Pang; Oleg Balabanov; Chaoqun Yang; Tianjin Huang; Lu Yin; Yaoqing Yang; Shiwei Liu

2025 ICML ICML 2025

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Zihang Liu, Tianyu Pang, Oleg Balabanov, Chaoqun Yang, Tianjin Huang, Lu Yin, Yaoqing Yang, Shiwei Liu

Topics

Machine Learning > Optimization & Theory > Optimization Artificial Intelligence > Core AI > Large Language Models Artificial Intelligence > Core AI > Reasoning

Keywords

low-rank approximation sparse fine-tuning rank reduction lift magnitude-based sparse fine-tuning principal weights

Download PDF