Skip to content
conftrace_
2024 ICML ICML 2024

Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Zixuan Wang, Stanley Wei, Daniel Hsu, Jason D. Lee

Topics

Machine Learning > Core Methods > Representation Learning Machine Learning > Optimization & Theory > Learning Theory Deep Learning > Architectures > Transformers

Keywords

transformer gradient descent fully-connected network algorithmic separation out-of-distribution length generalization sparse token selection
Download PDF