conftrace_

induction head

11 papers

Explore in graph

Co-occurring keywords

in-context learning (1405) large language model (13587) language model (4599) attention head (146) repetition neuron (3) skill neuron (5) episodic memory (86) attention mechanism (4001) representation learning (6206) transformer architecture (1560)

Papers

Language Models “Grok” to Copy NAACL 2025
Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training NAACL 2025
Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning IJCNLP 2025
Induction Heads as an Essential Mechanism for Pattern Matching in In-context Learning NAACL 2025
Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning AACL 2025
Linking In-context Learning in Transformers to Human Episodic Memory NIPS 2024
Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers NIPS 2024
The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains NIPS 2024
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale ACL 2023
Birth of a Transformer: A Memory Viewpoint NIPS 2023
Transformers Learn In-Context by Gradient Descent ICML 2023