What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Aaditya K Singh; Ted Moskovitz; Felix Hill; Stephanie C.Y. Chan; Andrew M Saxe

2024 ICML ICML 2024

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

❓ The Questioner

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Aaditya K Singh, Ted Moskovitz, Felix Hill, Stephanie C.Y. Chan, Andrew M Saxe

Topics

Artificial Intelligence > Core AI > Interpretability Machine Learning > Optimization & Theory > Learning Theory Natural Language Processing > Resources & Methods > Large Language Models

Keywords

induction head phase change clamping activations match-and-copy operation optogenetics-inspired causal framework subcircuits

Download PDF