Skip to content
conftrace_
2024 ICML ICML 2024

LoRAP: Transformer Sub-Layers Deserve Differentiated Structured Compression for Large Language Models

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Guangyan Li, Yongqiang Tang, Wensheng Zhang

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

feed-forward network multi-head self-attention structured pruning low-rank matrix approximation gradient-free structured channel pruning input activation weighted singular value decomposition
Download PDF