Position: Do pretrained Transformers Learn In-Context by Gradient Descent?

Lingfeng Shen; Aayush Mishra; Daniel Khashabi

2024 ICML ICML 2024

Position: Do pretrained Transformers Learn In-Context by Gradient Descent?

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

❓ The Questioner

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Lingfeng Shen, Aayush Mishra, Daniel Khashabi

Topics

Machine Learning > Optimization & Theory > Learning Theory Natural Language Processing > Generation > Language Modeling Artificial Intelligence > Core AI > Large Language Models

Keywords

in-context learning gradient descent output distribution pre-trained transformer demonstrations llama-7b

Download PDF