Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead

Rickard Brüel Gabrielsson; Jiacheng Zhu; Onkar Bhardwaj; Leshem Choshen; Kristjan Greenewald; Mikhail Yurochkin; Justin Solomon

2025 ICML ICML 2025

Compress then Serve: Serving Thousands of LoRA Adapters with Little Overhead

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Rickard Brüel Gabrielsson, Jiacheng Zhu, Onkar Bhardwaj, Leshem Choshen, Kristjan Greenewald, Mikhail Yurochkin, Justin Solomon

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

gpu memory compression loras throughput clusters scaling matrices

Download PDF