parameter efficiency

Sparse MLP for Image Recognition: Is Self-Attention Really Necessary? AAAI 2022 Streaming Automatic Speech Recognition with Re-blocking Processing Based on Integrated Voice Activity Detection Interspeech 2022 Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR 2022 Taming Sparsely Activated Transformer with Stochastic Experts ICLR 2022 The State of Sparse Training in Deep Reinforcement Learning ICML 2022 To update or not to update? Neurons at equilibrium in deep models NeurIPS 2022 Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure EMNLP 2022 Towards Simple and Efficient Task-Adaptive Pre-training for Text Classification AACL 2022 Unified Scaling Laws for Routed Language Models ICML 2022 AdapterDrop: On the Efficiency of Adapters in Transformers EMNLP 2021 Are wider nets better given the same number of parameters? ICLR 2021 Benchmarking a transformer-FREE model for ad-hoc retrieval EACL 2021 BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks EMNLP 2021 Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference EMNLP 2021 Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with $1/n$ Parameters ICLR 2021 CAM-GAN: Continual Adaptation Modules for Generative Adversarial Networks NeurIPS 2021 Cascaded Head-colliding Attention ACL 2021 Channel Permutations for N:M Sparsity NeurIPS 2021 Chasing Sparsity in Vision Transformers: An End-to-End Exploration NeurIPS 2021 Compressing 1D Time-Channel Separable Convolutions Using Sparse Random Ternary Matrices Interspeech 2021 Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data ICLR 2021 Continuous Self-Attention Models with Neural ODE Networks AAAI 2021 Counter-Interference Adapter for Multilingual Machine Translation EMNLP 2021 Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation EMNLP 2021 Deep Continuous Networks ICML 2021