induction head

Attention with Markov: A Curious Case of Single-layer Transformers ICLR 2025 Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence ICML 2025 Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training NAACL 2025 Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning NAACL 2025 KV Shifting Attention Enhances Language Modeling ICML 2025 Language Models “Grok” to Copy NAACL 2025 Selective Induction Heads: How Transformers Select Causal Structures in Context ICLR 2025 The Missing Alignment Link of In-context Learning on Sequences ICML 2025 Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning IJCNLP 2025 Which Attention Heads Matter for In-Context Learning? ICML 2025 How Transformers Learn Causal Structure with Gradient Descent ICML 2024 Linking In-context Learning in Transformers to Human Episodic Memory NeurIPS 2024 Overthinking the Truth: Understanding how Language Models Process False Demonstrations ICLR 2024 The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains NeurIPS 2024 The mechanistic basis of data dependence and abrupt learning in an in-context classification task ICLR 2024 Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers NeurIPS 2024 What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation ICML 2024 Birth of a Transformer: A Memory Viewpoint NeurIPS 2023 Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale ACL 2023 Transformers Learn In-Context by Gradient Descent ICML 2023