Skip to content
conftrace_

dense model

5 papers

Explore in graph

Co-occurring keywords · papers mentioning both

mixture of expert (4) large language model (3) scaling law (2) language model (2) knowledge distillation (2) model compression (2) inference cost (1) model initialization (1) memory constraint (1) autoregressive language model (1)

Papers

Joint MoE Scaling Laws: Mixture of Experts Can Be Memory Efficient ICML 2025 BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NeurIPS 2024 Breaking ReLU Barrier: Generalized MoEfication for Dense Pretrained Models EMNLP 2024 Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models EMNLP 2024 GPT-NeoX-20B: An Open-Source Autoregressive Language Model ACL 2022