Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism

Aviv Bick; Eric Xing; Albert Gu

2025 ICML ICML 2025

Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Aviv Bick, Eric Xing, Albert Gu

Topics

Artificial Intelligence > Core AI > Foundation Models Machine Learning > Application Areas > Efficient Computing Natural Language Processing > Generation > Language Modeling

Keywords

state space model gsm8k mmlu gather-and-aggregate mechanism pruned llama-3.1-8b retrieval bottleneck

Download PDF