conftrace_

kv cache

74 papers

Explore in graph

Also known as

KVCACHE KV CACHE KV

Co-occurring keywords

large language model (13587) model compression (3302) inference optimization (219) inference efficiency (248) inference acceleration (244) memory efficiency (261) attention mechanism (4001) long context (121) memory optimization (114) key-value cache (55)

Papers

Squeezed Attention: Accelerating Long Context Length LLM Inference ACL 2025
Value Residual Learning ACL 2025
InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model ACL 2025
Streamlining the Collaborative Chain of Models into A Single Forward Pass in Generation-Based Tasks ACL 2025
LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation EMNLP 2025
TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text EMNLP 2025
GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction EMNLP 2025
SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matching AACL 2025
SCA: Selective Compression Attention for Efficiently Extending the Context Window of Large Language Models EMNLP 2024
Block Transformer: Global-to-Local Language Modeling for Fast Inference NIPS 2024
DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion NIPS 2024
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization NIPS 2024
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact ACL 2024
Unlocking Data-free Low-bit Quantization with Matrix Decomposition for KV Cache Compression ACL 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression EMNLP 2024
InfiniPot: Infinite Context Processing on Memory-Constrained LLMs EMNLP 2024
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition ACL 2024
NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time ACL 2024
SnapKV: LLM Knows What You are Looking for Before Generation NIPS 2024
SGLang: Efficient Execution of Structured Language Model Programs NIPS 2024
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time NIPS 2023
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models NIPS 2023
$S^3$: Increasing GPU Utilization during Generative Inference for Higher Throughput NIPS 2023