Skip to content
conftrace_
2024 ICML ICML 2024

A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Mohammed Nowaz Rabbani Chowdhury, Meng Wang, Kaoutar El Maghraoui, Naigang Wang, Pin-Yu Chen, Christopher Carothers

Topics

Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Neural Networks

Keywords

model size fine-tuned model sparse mixture-of-experts computational requirements pruning experts router l2 norm
Download PDF