Skip to content
conftrace_
2024 ICML ICML 2024

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir braverman, Beidi Chen, Xia Hu

Topics

Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

quantization key-value cache batch size tuning free throughput 2bit
Download PDF