Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context

Xiang Cheng; Yuxin Chen; Suvrit Sra

2024 ICML ICML 2024

Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Xiang Cheng, Yuxin Chen, Suvrit Sra

Topics

Machine Learning > Optimization & Theory > Optimization Deep Learning > Architectures > Transformers Artificial Intelligence > Core AI > Large Language Models

Keywords

in-context learning transformer gradient descent nonlinear function activation class of functions

Download PDF