next-token prediction

Humanoid Locomotion as Next Token Prediction NeurIPS 2024 Implicit Optimization Bias of Next-token Prediction in Linear Models NeurIPS 2024 InternVideo2: Scaling Foundation Models for Multimodal Video Understanding ECCV 2024 Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions EMNLP 2024 Local to Global: Learning Dynamics and Effect of Initialization for Transformers NeurIPS 2024 Many-Shot In-Context Learning NeurIPS 2024 Mechanics of Next Token Prediction with Self-Attention AISTATS 2024 Non-asymptotic Convergence of Training Transformers for Next-token Prediction NeurIPS 2024 Object Recognition as Next Token Prediction CVPR 2024 On the Origins of Linear Representations in Large Language Models ICML 2024 Semformer: Transformer Language Models with Semantic Planning EMNLP 2024 Semiparametric Token-Sequence Co-Supervision ACL 2024 Sequential Modeling Enables Scalable Learning for Large Vision Models CVPR 2024 SMART: Scalable Multi-agent Real-time Motion Generation via Next-token Prediction NeurIPS 2024 Teaching Arithmetic to Small Transformers ICLR 2024 The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains NeurIPS 2024 The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More NeurIPS 2024 The Mystery of the Pathological Path-star Task for Language Models EMNLP 2024 The Pitfalls of Next-Token Prediction ICML 2024 Timer: Generative Pre-trained Transformers Are Large Time Series Models ICML 2024 Towards a theory of how the structure of language is acquired by deep neural networks NeurIPS 2024 Transformers Represent Belief State Geometry in their Residual Stream NeurIPS 2024 Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer NeurIPS 2023 Token Prediction as Implicit Classification to Identify LLM-Generated Text EMNLP 2023 On the Role of Bidirectionality in Language Model Pre-Training EMNLP 2022