conftrace_

expert routing

44 papers

Explore in graph

Co-occurring keywords

mixture of expert (640) large language model (13587) model compression (3302) neural network (6616) parameter-efficient fine-tuning (736) load balancing (55) sparse activation (36) expert specialization (26) model efficiency (206) routing mechanism (17)

Papers

Advancing SMoE for Continuous Domain Adaptation of MLLMs: Adaptive Router and Domain-Specific Loss ACL 2025
HookMoE: A learnable performance compensation strategy of Mixture-of-Experts for LLM inference acceleration EMNLP 2025
EWoRA: Expert Weighted Low-Rank Adaptation for Heterogeneous Data AACL 2025
HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts ACL 2024
Mixture of Diverse Size Experts EMNLP 2024
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training EMNLP 2024
MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts NIPS 2024
LocMoE: A Low-overhead MoE for Large Language Model Training IJCAI 2024
Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models NAACL 2024
$\textit{Read-ME}$: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design NIPS 2024
MoVA: Adapting Mixture of Vision Experts to Multimodal Context NIPS 2024
Turn Waste into Worth: Rectifying Top-k Router of MoE EMNLP 2024
RepMode: Learning to Re-Parameterize Diverse Experts for Subcellular Structure Prediction CVPR 2023
Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks ICML 2023
On the Benefits of Learning to Route in Mixture-of-Experts Models EMNLP 2023
Collaboration of Experts: Achieving 80% Top-1 Accuracy on ImageNet with 100M FLOPs ICML 2022
Mixture-of-Experts with Expert Choice Routing NIPS 2022
A Mixture-of-Experts Model for Antonym-Synonym Discrimination ACL 2021
BASE Layers: Simplifying Training of Large, Sparse Models ICML 2021