inference optimization

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time ICML 2023 Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model JMLR 2023 Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding EMNLP 2023 FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference ACL 2023 Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings ACL 2023 Fixed Input Parameterization for Efficient Prompting ACL 2023 FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU ICML 2023 GPTCache: An Open-Source Semantic Cache for LLM Applications Enabling Faster Answers and Cost Savings EMNLP 2023 Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs EMNLP 2023 Multi-word Tokenization for Sequence Compression EMNLP 2023 MUX-PLMs: Pre-training Language Models with Data Multiplexing ACL 2023 Optimizing Dynamic Neural Networks with Brainstorm OSDI 2023 Predictive Exit: Prediction of Fine-Grained Early Exits for Computation- and Energy-Efficient Inference AAAI 2023 Rapid Diffusion: Building Domain-Specific Text-to-Image Synthesizers with Fast Inference Speed ACL 2023 Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference? EMNLP 2023 RIFormer: Keep Your Vision Backbone Effective but Removing Token Mixer CVPR 2023 Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time NeurIPS 2023 Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation EMNLP 2023 The Benefits of Bad Advice: Autocontrastive Decoding across Model Layers ACL 2023 The case for 4-bit precision: k-bit Inference Scaling Laws ICML 2023 Towards Optimal Caching and Model Selection for Large Model Inference NeurIPS 2023 Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases ICML 2023 ZipLM: Inference-Aware Structured Pruning of Language Models NeurIPS 2023 AdapLeR: Speeding up Inference by Adaptive Length Reduction ACL 2022 Better Intermediates Improve CTC Inference Interspeech 2022