conftrace_

latency-throughput tradeoff

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

sparse kv cache (1) long-context generation (1) batch size optimization (1) draft model (1) speculative decoding (1)

Papers

MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding ICLR 2025