conftrace_

mixture of expert

640 papers

Explore in graph

Also known as

MOE

Co-occurring keywords

large language model (13587) model compression (3302) multi-task learning (2575) multimodal learning (4645) neural network (6616) knowledge distillation (3725) low-rank adaptation (560) parameter efficiency (417) transfer learning (5449) expert routing (44)

Papers

Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT EMNLP 2022
Learning Mixture of Domain-Specific Experts via Disentangled Factors for Autonomous Driving AAAI 2022
Go Wider Instead of Deeper AAAI 2022
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR 2022
MoESR: Blind Super-Resolution Using Kernel-Aware Mixture of Experts WACV 2022
How Can Cross-lingual Knowledge Contribute Better to Fine-Grained Entity Typing? ACL 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation NAACL 2022
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts EMNLP 2022
Eliciting and Understanding Cross-task Skills with Task-level Mixture-of-Experts EMNLP 2022
Mixture of Attention Heads: Selecting Attention Heads Per Token EMNLP 2022
DEMix Layers: Disentangling Domains for Modular Language Modeling NAACL 2022
GraphDIVE: Graph Classification by Mixture of Diverse Experts IJCAI 2022
Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production EMNLP 2022
Using Deep Mixture-of-Experts to Detect Word Meaning Shift for TempoWiC EMNLP 2022
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale ICML 2022
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts NAACL 2022
Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers ICML 2022
BMCook: A Task-agnostic Compression Toolkit for Big Models EMNLP 2022
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning EMNLP 2022
Spatial Mixture-of-Experts NIPS 2022
On the Representation Collapse of Sparse Mixture of Experts NIPS 2022
A Theoretical View on Sparsely Activated Networks NIPS 2022
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training NIPS 2022
Diversifying Content Generation for Commonsense Reasoning with Mixture of Knowledge Graph Experts ACL 2022
StableMoE: Stable Routing Strategy for Mixture of Experts ACL 2022