conftrace_

sparse activation

36 papers

Explore in graph

Co-occurring keywords

mixture of expert (640) model compression (3302) parameter efficiency (417) large language model (13587) neural network (6616) language model (4599) expert routing (44) feed-forward network (79) model scaling (135) convolutional neural network (4226)

Papers

Gating Dropout: Communication-efficient Regularization for Sparsely Activated Transformers ICML 2022
Sparse Instance Activation for Real-Time Instance Segmentation CVPR 2022
M³ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design NIPS 2022
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR 2022
MoEfication: Transformer Feed-forward Layers are Mixtures of Experts ACL 2022
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts ICML 2022
Mixture-of-Experts with Expert Choice Routing NIPS 2022
Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks ICML 2020
Why We Go Where We Go: Profiling User Decisions on Choosing POIs IJCAI 2020
The Tree Ensemble Layer: Differentiability meets Conditional Computation ICML 2020
SplineNets: Continuous Neural Decision Graphs NIPS 2018