Skip to content
conftrace_
2026 EMNLP EMNLP 2026

TokenDecouple: Frequency-aware Token Merging and Pyramidal Text Decay

AbstractLarge Language Models (LLMs) have recently shown strong potential for time series (TS) analysis by integrating numerical signals with rich text prompts. In this process, although TS are encoded into token sequences and fed into LLMs in a manner similar to text prompts, TS tokens differ from prompt tokens in their underlying structure and semantics, a distinction that remains underexplored in existing LLM-based TS systems. In this work, we investigate both TS tokens and prompt tokens from a new perspective. First, we re-examine TS tokens in the frequency-domain and reveal that token-level information contribution is highly uneven: large groups of tokens share redundant spectral patterns, while a small subset acts as critical frequency anchors. Motivated by this observation, we introduce a frequency-based token merging strategy for TS tokens, which preserves essential information while eliminating redundancy. Second, we uncover a unexplored pyramidal decay effect in prompt tokens, showing that their influence rapidly diminishes with model depth. Building on these findings, we propose TokenDecouple, a unified framework that decouples the compression of TS tokens and prompt tokens. Extensive experiments across diverse TS tasks show that TokenDecouple achieves up to 7.68× inference speedup while delivering performance improvements in 78% of evaluated scenarios, establishing an efficient paradigm for LLM-based TS analysis.

More from EMNLP 2026

All EMNLP 2026 papers →