conftrace_

key-value cache

55 papers

Explore in graph

Also known as

KV CACHE KV-CACHE KV

Co-occurring keywords

large language model (13587) attention mechanism (4001) model compression (3302) inference efficiency (248) kv cache (74) inference optimization (219) retrieval-augmented generation (1592) memory efficiency (261) long context (121) memory optimization (114)

Papers

CIFLEX: Contextual Instruction Flow for Sub-task Execution in Multi-Turn Interactions with a Single On-Device LLM EMNLP 2025
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization EMNLP 2025
CSR:Achieving 1 Bit Key-Value Cache via Sparse Representation AAAI 2025
Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query EMNLP 2025
EdgeInfinite: A Memory-Efficient Infinite-Context Transformer for Edge Devices ACL 2025
FlashBack: Efficient Retrieval-Augmented Language Modeling for Fast Inference ACL 2025
SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation ACL 2025
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts ACL 2025
Loki: Low-rank Keys for Efficient Sparse Attention NIPS 2024
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition ACL 2024
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention NIPS 2024
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference EMNLP 2024
SnapKV: LLM Knows What You are Looking for Before Generation NIPS 2024
Block Transformer: Global-to-Local Language Modeling for Fast Inference NIPS 2024
Efficient Test-Time Adaptation of Vision-Language Models CVPR 2024
You Only Cache Once: Decoder-Decoder Architectures for Language Models NIPS 2024
Layer-Condensed KV Cache for Efficient Inference of Large Language Models ACL 2024
RelayAttention for Efficient Large Language Model Serving with Long System Prompts ACL 2024
Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization EMNLP 2024
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling EMNLP 2024
InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management OSDI 2024
ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction NIPS 2024
MiniCache: KV Cache Compression in Depth Dimension for Large Language Models NIPS 2024
Focused Transformer: Contrastive Training for Context Scaling NIPS 2023
Long-Range Language Modeling with Selective Cache EMNLP 2023