conftrace_

speculative decoding

111 papers

Explore in graph

Also known as

SD

Co-occurring keywords

large language model (13587) inference acceleration (244) inference optimization (219) token generation (47) draft model (21) model compression (3302) model acceleration (80) language model (4599) autoregressive decoding (33) parallel decoding (74)

Papers

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding ACL 2024
REST: Retrieval-Based Speculative Decoding NAACL 2024
Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration NIPS 2024
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees EMNLP 2024
Graph-Structured Speculative Decoding ACL 2024
BASS: Batched Attention-optimized Speculative Sampling ACL 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding ACL 2024
Speculative Decoding with Big Little Decoder NIPS 2023
SpecTr: Fast Speculative Decoding via Optimal Transport NIPS 2023
Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation EMNLP 2023
Fast Inference from Transformers via Speculative Decoding ICML 2023