kv cache

A Training-Free Sub-quadratic Cost Transformer Model Serving Framework with Hierarchically Pruned Attention ICLR 2025 CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferences ICLR 2025 DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference ICLR 2025 DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads ICLR 2025 DyCoke: Dynamic Compression of Tokens for Fast Video Large Language Models CVPR 2025 DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs EMNLP 2025 Efficient Beam Search for Large Language Models Using Trie-Based Decoding EMNLP 2025 FIER: Fine-Grained and Efficient KV Cache Retrieval for Long-context LLM Inference EMNLP 2025 GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction EMNLP 2025 IAM: Efficient Inference through Attention Mapping between Different-scale LLMs ACL 2025 InfiniSST: Simultaneous Translation of Unbounded Speech with Large Language Model ACL 2025 KV-Edit: Training-Free Image Editing for Precise Background Preservation ICCV 2025 KV-Latent: Dimensional-level KV Cache Reduction with Frequency-aware Rotary Positional Embedding ACL 2025 LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation EMNLP 2025 LeanK: Learnable K Cache Channel Pruning for Efficient Decoding EMNLP 2025 Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries ICML 2025 Lookahead Q-Cache: Achieving More Consistent KV Cache Eviction via Pseudo Query EMNLP 2025 MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference ACL 2025 MagicPIG: LSH Sampling for Efficient LLM Generation ICLR 2025 MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference NAACL 2025 MiniKV: Pushing the Limits of 2-Bit KV Cache via Compression and System Co-Design for Efficient Long Context Inference ACL 2025 MobiLoRA: Accelerating LoRA-based LLM Inference on Mobile Devices via Context-aware KV Cache Optimization ACL 2025 Multi-matrix Factorization Attention ACL 2025 Palu: KV-Cache Compression with Low-Rank Projection ICLR 2025 RefreshKV: Updating Small KV Cache During Long-form Generation ACL 2025