conftrace_

key-value cache

55 papers

Explore in graph

Also known as

KV CACHE KV-CACHE KV

Co-occurring keywords

large language model (13587) attention mechanism (4001) model compression (3302) inference efficiency (248) kv cache (74) inference optimization (219) retrieval-augmented generation (1592) memory efficiency (261) long context (121) memory optimization (114)

Papers

Context Compression for Auto-regressive Transformers with Sentinel Tokens EMNLP 2023
Segcache: a memory-efficient and scalable in-memory key-value cache for small objects NSDI 2021
Flashield: a Hybrid Key-value Cache that Controls Flash Write Amplification NSDI 2019
LHD: Improving Cache Hit Rate by Maximizing Hit Density NSDI 2018
A Simple Cache Model for Image Recognition NIPS 2018