inference efficiency

SlowBERT: Slow-down Attacks on Input-adaptive Multi-exit BERT ACL 2023 Speculative Decoding with Big Little Decoder NeurIPS 2023 TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction EMNLP 2023 The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment EMNLP 2023 Towards Fully Quantized Neural Networks For Speech Enhancement Interspeech 2023 Towards Inference Efficient Deep Ensemble Learning AAAI 2023 You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model CVPR 2023 ABC: Attention with Bounded-memory Control ACL 2022 A-ViT: Adaptive Tokens for Efficient Vision Transformer CVPR 2022 ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference ACL 2022 ELMER: A Non-Autoregressive Pre-trained Language Model for Efficient and Effective Text Generation EMNLP 2022 FP8 Quantization: The Power of the Exponent NeurIPS 2022 Latency-aware Spatial-wise Dynamic Networks NeurIPS 2022 LayerConnect: Hypernetwork-Assisted Inter-Layer Connector to Enhance Parameter Efficiency COLING 2022 Linearizing Transformer with Key-Value Memory EMNLP 2022 Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems EMNLP 2022 Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time ICML 2022 MoEfication: Transformer Feed-forward Layers are Mixtures of Experts ACL 2022 Non-autoregressive Translation with Layer-Wise Prediction and Deep Supervision AAAI 2022 SkipBERT: Efficient Inference with Shallow Layer Skipping ACL 2022 Speeding up Transformer Decoding via an Attention Refinement Network COLING 2022 AdapterDrop: On the Efficiency of Adapters in Transformers EMNLP 2021 Almost Free Semantic Draft for Neural Machine Translation NAACL 2021 A User-Adaptive Layer Selection Framework for Very Deep Sequential Recommender Models AAAI 2021 Bag of Tricks for Optimizing Transformer Efficiency EMNLP 2021