conftrace_

mixture of expert

640 papers

Explore in graph

Also known as

MOE

Co-occurring keywords

large language model (13587) model compression (3302) multi-task learning (2575) multimodal learning (4645) neural network (6616) knowledge distillation (3725) low-rank adaptation (560) parameter efficiency (417) transfer learning (5449) expert routing (44)

Papers

On Parameter Estimation in Deviated Gaussian Mixture of Experts AISTATS 2024
LocMoE: A Low-overhead MoE for Large Language Model Training IJCAI 2024
SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR INTERSPEECH 2024
MoEUT: Mixture-of-Experts Universal Transformers NIPS 2024
MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models NIPS 2024
AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality EMNLP 2024
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention NIPS 2024
LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-Training EMNLP 2024
Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models EMNLP 2024
Towards One-to-Many Visual Question Answering EMNLP 2024
BiMediX: Bilingual Medical Mixture of Experts LLM EMNLP 2024
MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning EMNLP 2024
Enhancing Legal Expertise in Large Language Models through Composite Model Integration: The Development and Evaluation of Law-Neo EMNLP 2024
How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers NIPS 2024
Mixture of In-Context Experts Enhance LLMs' Long Context Awareness NIPS 2024
MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition EMNLP 2024
Efficient and Interpretable Grammatical Error Correction with Mixture of Experts EMNLP 2024
FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion NIPS 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NIPS 2024
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization NIPS 2024
Variational Distillation of Diffusion Policies into Mixture of Experts NIPS 2024
MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks NIPS 2024
Transforming Vision Transformer: Towards Efficient Multi-Task Asynchronous Learner NIPS 2024
Mixture of Nested Experts: Adaptive Processing of Visual Tokens NIPS 2024
Multi-Head Mixture-of-Experts NIPS 2024