sparse activation

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models ACL 2024 Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning AAAI 2024 Enhancing Multiple Dimensions of Trustworthiness in LLMs via Sparse Activation Control NeurIPS 2024 Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models AAAI 2024 Learning Neural Networks with Sparse Activations COLT 2024 Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy ICLR 2024 Mixture of Diverse Size Experts EMNLP 2024 Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK JMLR 2024 Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning CoRL 2024 Unraveling Babel: Exploring Multilingual Activation Patterns of LLMs and Their Applications EMNLP 2024 AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation ACL 2023 Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference NeurIPS 2023 On the Benefits of Learning to Route in Mixture-of-Experts Models EMNLP 2023 On the Compositional Generalization in Versatile Open-domain Dialogue ACL 2023 Towards Being Parameter-Efficient: A Stratified Sparsely Activated Transformer with Dynamic Capacity EMNLP 2023 Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers ICML 2022 GLaM: Efficient Scaling of Language Models with Mixture-of-Experts ICML 2022 M³ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design NeurIPS 2022 Mixture-of-Experts with Expert Choice Routing NeurIPS 2022 MoEfication: Transformer Feed-forward Layers are Mixtures of Experts ACL 2022 Sparse Instance Activation for Real-Time Instance Segmentation CVPR 2022 Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR 2022 Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks ICML 2020 The Tree Ensemble Layer: Differentiability meets Conditional Computation ICML 2020 Why We Go Where We Go: Profiling User Decisions on Choosing POIs IJCAI 2020