MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Zechun Liu; Changsheng Zhao; Forrest Iandola; Chen Lai; Yuandong Tian; Igor Fedorov; Yunyang Xiong; Ernie Chang; Yangyang Shi; Raghuraman Krishnamoorthi; Liangzhen Lai; Vikas Chandra

2024 ICML ICML 2024

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, Vikas Chandra

Topics

Machine Learning > Application Areas > Efficient Computing Natural Language Processing > Generation > Language Modeling Artificial Intelligence > Core AI > Large Language Models

Keywords

model architecture embedding sharing weight sharing grouped-query attention deep and thin architectures sub-billion parameter models

Download PDF