Theory

HiRoPE: Length Extrapolation for Code Models Using Hierarchical Position ACL 2024 How does Architecture Influence the Base Capabilities of Pre-trained Language Models? A Case Study Based on FFN-Wider and MoE Transformers NeurIPS 2024 How does Gradient Descent Learn Features --- A Local Analysis for Regularized Two-Layer Neural Networks NeurIPS 2024 How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks NeurIPS 2024 How many classifiers do we need? NeurIPS 2024 Hypothesis Testing the Circuit Hypothesis in LLMs NeurIPS 2024 Identifying Equivalent Training Dynamics NeurIPS 2024 If You Want to Be Robust, Be Wary of Initialization NeurIPS 2024 Immunization against harmful fine-tuning attacks EMNLP 2024 Implicit bias of SGD in L_2-regularized linear DNNs: One-way jumps from high to low rank ICLR 2024 Implicit Optimization Bias of Next-token Prediction in Linear Models NeurIPS 2024 Implicit Regularization in Deep Tucker Factorization: Low-Rankness via Structured Sparsity AISTATS 2024 Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems NeurIPS 2024 Implicit Regularization Paths of Weighted Neural Representations NeurIPS 2024 Improved Sample Complexity Bounds for Diffusion Model Training NeurIPS 2024 Improving Adaptivity via Over-Parameterization in Sequence Models NeurIPS 2024 Improving Normalization With the James-Stein Estimator WACV 2024 In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization NeurIPS 2024 Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models NeurIPS 2024 Infinite-Dimensional Feature Interaction NeurIPS 2024 Infinite Limits of Multi-head Transformer Dynamics NeurIPS 2024 Input Margins Can Predict Generalization Too AAAI 2024 Investigating the Impact of Model Width and Density on Generalization in Presence of Label Noise UAI 2024 Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization NeurIPS 2024 Law of Large Numbers and Central Limit Theorem for Wide Two-layer Neural Networks: The Mini-Batch and Noisy Case JMLR 2024