conftrace_

model upcycling

5 papers

Explore in graph

Co-occurring keywords · papers mentioning both

mixture of expert (3) expert specialization (2) parameter efficiency (2) parameter reuse (1) checkpoint reuse (1) sparsely activated model (1) parameter alignment (1) neural network (1) expert diversity (1) dense checkpoint (1)

Papers

Symphony-MoE: Harmonizing Disparate Pre-trained Models into a Coherent Mixture-of-Experts AAAI 2026 DeRS: Towards Extremely Efficient Upcycled Mixture-of-Experts Models CVPR 2025 Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization ICLR 2025 Nexus: Adaptive Upcycling to Efficiently Pretrain Mixture of Experts EMNLP 2025 Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints ICLR 2023