Theory

Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? A Petroglyph Revisited ACL 2025 Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs ACL 2025 $\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling NeurIPS 2024 AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters ACL 2024 Abrupt Learning in Transformers: A Case Study on Matrix Completion NeurIPS 2024 A Canonicalization Perspective on Invariant and Equivariant Learning NeurIPS 2024 Achieving Domain-Independent Certified Robustness via Knowledge Continuity NeurIPS 2024 Algorithmic progress in language models NeurIPS 2024 Alpha Invariance: On Inverse Scaling Between Distance and Volume Density in Neural Radiance Fields CVPR 2024 An Analytic Solution to Covariance Propagation in Neural Networks AISTATS 2024 An Empirical Study of Scaling Law for Scene Text Recognition CVPR 2024 An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem NeurIPS 2024 A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks JMLR 2024 A Phase Transition between Positional and Semantic Learning in a Solvable Model of Dot-Product Attention NeurIPS 2024 Approximation Rate of the Transformer Architecture for Sequence Modeling NeurIPS 2024 A provable control of sensitivity of neural networks through a direct parameterization of the overall bi-Lipschitzness NeurIPS 2024 A two-scale Complexity Measure for Deep Learning Models NeurIPS 2024 Backward Lens: Projecting Language Model Gradients into the Vocabulary Space EMNLP 2024 Base of RoPE Bounds Context Length NeurIPS 2024 Batch Normalization Alleviates the Spectral Bias in Coordinate Networks CVPR 2024 Batch Normalization Is Blind to the First and Second Derivatives of the Loss AAAI 2024 Bayes-optimal learning of an extensive-width neural network from quadratically many samples NeurIPS 2024 B-cosification: Transforming Deep Neural Networks to be Inherently Interpretable NeurIPS 2024 Benign overfitting in leaky ReLU networks with moderate input dimension NeurIPS 2024 Beyond IID weights: sparse and low-rank deep Neural Networks are also Gaussian Processes ICLR 2024