conftrace_

time to first token

4 papers

Explore in graph

Also known as

TTFT

Co-occurring keywords · papers mentioning both

key-value cache (2) chunked text (1) multi-device inference (1) context caching (1) extension phase (1) parallelization (1) prompt phase (1) key-value vectors (1) legolink algorithm (1) position-independent caching (1)

Papers

EPIC: Efficient Position-Independent Caching for Serving Large Language Models ICML 2025 TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text EMNLP 2025 Kraken: Inherently Parallel Transformers For Efficient Multi-Device Inference NeurIPS 2024 KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation ICML 2024