conftrace_

expert load balancing

3 papers

Explore in graph

Co-occurring keywords

mixture of expert (640) model compression (3302) dynamic gating (5) model deployment (35) parallel computation (38) expert routing (44) model efficiency (206) flops reduction (50) neural network (6616) token dropping (10)

Papers

Toward Efficient Inference for Mixture of Experts NIPS 2024
Turn Waste into Worth: Rectifying Top-k Router of MoE EMNLP 2024
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Models EMNLP 2024