conftrace_

long-context generation

2 papers

Explore in graph

Also known as

LCG

Co-occurring keywords · papers mentioning both

latency-throughput tradeoff (1) sparse kv cache (1) batch size optimization (1) synthetic benchmark (1) performance degradation (1) draft model (1) speculative decoding (1) language model (1) benchmark evaluation (1)

Papers

MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding ICLR 2025 LongGenBench: Long-context Generation Benchmark EMNLP 2024