Fundamental limits of learning in sequence multi-index models and deep attention networks: high-dimensional asymptotics and sharp thresholds

Emanuele Troiani; Hugo Cui; Yatin Dandi; Florent Krzakala; Lenka Zdeborová

2025 ICML ICML 2025

Fundamental limits of learning in sequence multi-index models and deep attention networks: high-dimensional asymptotics and sharp thresholds

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Emanuele Troiani, Hugo Cui, Yatin Dandi, Florent Krzakala, Lenka Zdeborová

Topics

Machine Learning > Optimization & Theory > Learning Theory Deep Learning > Architectures > Transformers

Keywords

sharp threshold self-attention layer bayes-optimal learning deep attention networks low-rank weights sequence multi-index models

Download PDF