speculative decoding

Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training EMNLP 2024 MatFormer: Nested Transformer for Elastic Inference NeurIPS 2024 Nearest Neighbor Speculative Decoding for LLM Generation and Attribution NeurIPS 2024 Online Speculative Decoding ICML 2024 Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding EMNLP 2024 REST: Retrieval-Based Speculative Decoding NAACL 2024 Sequoia: Scalable and Robust Speculative Decoding NeurIPS 2024 SLiM: Speculative Decoding with Hypothesis Reduction NAACL 2024 SpecExec: Massively Parallel Speculative Decoding For Interactive LLM Inference on Consumer Devices NeurIPS 2024 SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding EMNLP 2024 Speculative Contrastive Decoding ACL 2024 Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism ACL 2024 Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration NeurIPS 2024 Tandem Transformers for Inference Efficient LLMs ICML 2024 Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024 The Mamba in the Llama: Distilling and Accelerating Hybrid Models NeurIPS 2024 Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters EMNLP 2024 Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding ACL 2024 When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models ICML 2024 Fast Inference from Transformers via Speculative Decoding ICML 2023 SpecTr: Fast Speculative Decoding via Optimal Transport NeurIPS 2023 Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation EMNLP 2023 Speculative Decoding with Big Little Decoder NeurIPS 2023