conftrace_

mixture of expert

640 papers

Explore in graph

Also known as

MOE

Co-occurring keywords

large language model (13587) model compression (3302) multi-task learning (2575) multimodal learning (4645) neural network (6616) knowledge distillation (3725) low-rank adaptation (560) parameter efficiency (417) transfer learning (5449) expert routing (44)

Papers

Mixture of Tokens: Continuous MoE through Cross-Example Aggregation NIPS 2024
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts NIPS 2024
Mixture of Experts Meets Prompt-Based Continual Learning NIPS 2024
Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrast NIPS 2024
$\textit{Read-ME}$: Refactorizing LLMs as Router-Decoupled Mixture of Experts with System Co-Design NIPS 2024
AutoMix: Automatically Mixing Language Models NIPS 2024
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts CVPR 2024
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models ACL 2024
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts NIPS 2024
Multi-Task Dense Prediction via Mixture of Low-Rank Experts CVPR 2024
Neural Experts: Mixture of Experts for Implicit Neural Representations NIPS 2024
PEMT: Multi-Task Correlation Guided Mixture-of-Experts Enables Parameter-Efficient Transfer Learning ACL 2024
XFT: Unlocking the Power of Code Instruction Tuning by Simply Merging Upcycled Mixture-of-Experts ACL 2024
MoExtend: Tuning New Experts for Modality and Task Extension ACL 2024
SciMind: A Multimodal Mixture-of-Experts Model for Advancing Pharmaceutical Sciences ACL 2024
Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising NIPS 2024
Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts NIPS 2024
MoVA: Adapting Mixture of Vision Experts to Multimodal Context NIPS 2024
Customizing Language Models with Instance-wise LoRA for Sequential Recommendation NIPS 2024
Werkzeug at SemEval-2024 Task 8: LLM-Generated Text Detection via Gated Mixture-of-Experts Fine-Tuning SEMEVAL 2024
Model Decides How to Tokenize: Adaptive DNA Sequence Tokenization with MxDNA NIPS 2024
Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts AISTATS 2024
Graph Mixture of Experts: Learning on Large-Scale Graphs with Explicit Diversity Modeling NIPS 2023
Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation ACL 2023
Domain-Agnostic Neural Architecture for Class Incremental Continual Learning in Document Processing Platform ACL 2023