Looking Beyond the Top-1: Transformers Determine Top Tokens in Order

Daria Lioubashevski; Tomer M. Schlank; Gabriel Stanovsky; Ariel Goldstein

2025 ICML ICML 2025

Looking Beyond the Top-1: Transformers Determine Top Tokens in Order

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Daria Lioubashevski, Tomer M. Schlank, Gabriel Stanovsky, Ariel Goldstein

Topics

Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Transformers Natural Language Processing > Generation > Language Modeling

Keywords

language modeling transformer early-exit strategy hidden layer embedding saturation event top-k tokens

Download PDF