inference optimization

Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis CVPR 2024 RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control JMLR 2024 Selective Perception: Learning Concise State Descriptions for Language Model Actors NAACL 2024 SGLang: Efficient Execution of Structured Language Model Programs NeurIPS 2024 Sing it, Narrate it: Quality Musical Lyrics Translation EMNLP 2024 SLiM: Speculative Decoding with Hypothesis Reduction NAACL 2024 SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model NeurIPS 2024 SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding EMNLP 2024 Sprout: Green Generative AI with Carbon-Efficient LLM Inference EMNLP 2024 Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024 TensorOpera Router: A Multi-Model Router for Efficient LLM Inference EMNLP 2024 Toward Efficient Inference for Mixture of Experts NeurIPS 2024 Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators ICLR 2024 Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters EMNLP 2024 Tree Cross Attention ICLR 2024 Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding ACL 2024 VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models EMNLP 2024 When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models EMNLP 2024 Xinference: Making Large Model Serving Easy EMNLP 2024 Accelerating Transformer Inference for Translation via Parallel Decoding ACL 2023 Accurate and Structured Pruning for Efficient Automatic Speech Recognition Interspeech 2023 A Survey on Model Compression and Acceleration for Pretrained Language Models AAAI 2023 AutoDiffusion: Training-Free Optimization of Time Steps and Architectures for Automated Diffusion Model Acceleration ICCV 2023 BADGE: Speeding Up BERT Inference after Deployment via Block-wise Bypasses and Divergence-based Early Exiting ACL 2023 Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference AutoML 2023