conftrace_

kv cache compression

30 papers

Explore in graph

Also known as

KV CACHE KV

Co-occurring keywords

large language model (13587) attention mechanism (4001) memory efficiency (261) model compression (3302) kv cache (74) inference optimization (219) long context (121) inference efficiency (248) memory optimization (114) efficient inference (233)

Papers

SnapKV: LLM Knows What You are Looking for Before Generation NIPS 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches EMNLP 2024
A Simple and Effective L_2 Norm-Based Strategy for KV Cache Compression EMNLP 2024
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time NIPS 2023