inference optimization

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale ICML 2022 Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers NSDI 2022 FalCon: Fine-Grained Feature Map Sparsity Computing With Decomposed Convolutions for Inference Optimization WACV 2022 Fast Vocabulary Transfer for Language Model Compression EMNLP 2022 MLink: Linking Black-Box Models for Collaborative Multi-Model Inference AAAI 2022 Sparsifying Transformer Models with Trainable Representation Pooling ACL 2022 Who Says Elephants Can’t Run: Bringing Large Scale MoE Models into Cloud Scale Production EMNLP 2022 A Root of a Problem: Optimizing Single-Root Dependency Parsing EMNLP 2021 Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference EMNLP 2021 Efficient Machine Translation with Model Pruning and Quantization EMNLP 2021 FastSeq: Make Sequence Generation Faster ACL 2021 HW-TSC’s Participation in the WMT 2021 Efficiency Shared Task EMNLP 2021 IrEne: Interpretable Energy Prediction for Transformers ACL 2021 LeeBERT: Learned Early Exit for BERT with cross-level optimization ACL 2021 LightSeq: A High Performance Inference Library for Transformers NAACL 2021 RepVGG: Making VGG-Style ConvNets Great Again CVPR 2021 TenTrans High-Performance Inference Toolkit for WMT2021 Efficiency Task EMNLP 2021 Early Exiting BERT for Efficient Document Ranking EMNLP 2020 Efficient Inference For Neural Machine Translation EMNLP 2020 High Performance Depthwise and Pointwise Convolutions on Mobile Devices AAAI 2020 Inference Stage Optimization for Cross-scenario 3D Human Pose Estimation NeurIPS 2020 Learning Fast and Robust Target Models for Video Object Segmentation CVPR 2020 Losing Heads in the Lottery: Pruning Transformer Attention in Neural Machine Translation EMNLP 2020 Neural Speed Reading Audited EMNLP 2020 Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning NeurIPS 2020