Skip to content
conftrace_
2026 EMNLP EMNLP 2026

TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning

AbstractLong-context inference with large language models (LLMs) is constrained by the linear growth of the key–value (KV) cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a “meta-query” during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.

More from EMNLP 2026

All EMNLP 2026 papers →