computational requirements

pruning experts (1) router l2 norm (1) pre-trained initializations (1) sparse mixture-of-experts (1) network growth (1) backward error analysis (1) fine-tuned model (1) model size (1) generalization performance (1) training dynamics (1)