conftrace_

expert specialization

26 papers

Explore in graph

Co-occurring keywords

mixture of expert (640) parameter efficiency (417) knowledge distillation (3725) model compression (3302) large language model (13587) neural network (6616) expert routing (44) instruction tuning (838) model architecture (324) load balancing (55)

Papers

Expert-Inspired Multi-Agent Coordination for Multi-Objective Molecular Optimization AAAI 2026
Specialization through Collaboration: Understanding Expert Interaction in Mixture-of-Expert Large Language Models EACL 2026
Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition ACL 2026
Hybrid Routing for a Mixture of LoRA Experts AAAI 2026
Understanding and Leveraging the Expert Specialization of Context Faithfulness in Mixture-of-Experts LLMs EMNLP 2025
HMoE: Heterogeneous Mixture of Experts for Language Modeling EMNLP 2025
Channel Merging: Preserving Specialization for Merged Experts AAAI 2025
Bagging-Expert Network for Multi-Task Learning: A Depolarization Solution in Multi-Gate Mixture-of-Experts AAAI 2025
DIVE into MoE: Diversity-Enhanced Reconstruction of Large Language Models from Dense into Mixture-of-Experts ACL 2025
Dynamic Expert Specialization: Towards Catastrophic Forgetting-Free Multi-Domain MoE Adaptation EMNLP 2025
Part-Of-Speech Sensitivity of Routers in Mixture of Experts Models COLING 2025
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging ACL 2025
BTS: Harmonizing Specialized Experts into a Generalist LLM EMNLP 2025
CoMoE: Contrastive Representation for Mixture-of-Experts in Parameter-Efficient Fine-tuning EMNLP 2025
Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models ACL 2025
Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis ACL 2025
Nexus: Adaptive Upcycling to Efficiently Pretrain Mixture of Experts EMNLP 2025
Bag of Tricks for Sparse Mixture-of-Experts: A Benchmark Across Reasoning, Efficiency, and Safety EMNLP 2025
Mixture-of-Clustered-Experts: Advancing Expert Specialization and Generalization in Instruction Tuning EMNLP 2025
A Lightweight Mixture-of-Experts Neural Machine Translation Model with Stage-wise Training Strategy NAACL 2024
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models ACL 2024
MoDE: A Mixture-of-Experts Model with Mutual Distillation among the Experts AAAI 2024
Let the Expert Stick to His Last: Expert-Specialized Fine-Tuning for Sparse Architectural Large Language Models EMNLP 2024
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization NIPS 2024
Mod-Squad: Designing Mixtures of Experts As Modular Multi-Task Learners CVPR 2023