Tengyu Ma

SAM 2: Segment Anything in Images and Videos ICLR 2025 Rethinking Reconstruction and Denoising in the Dark: New Perspective, General Architecture and Beyond CVPR 2025 STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving ICML 2025 Non-Asymptotic Length Generalization ICML 2025 Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View ICLR 2025 Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training ICLR 2024 Linguistic Calibration of Long-Form Generations ICML 2024 One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention ICLR 2024 Chain of Thought Empowers Transformers to Solve Inherently Serial Problems ICLR 2024 Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation AAAI 2024 Large Language Models as Tool Makers ICLR 2024 Symbol tuning improves in-context learning in language models EMNLP 2023 Toward L_∞Recovery of Nonlinear Functions: A Polynomial Sample Complexity Bound for Gaussian Random Fields COLT 2023 DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining NIPS 2023 Asymptotic Instance-Optimal Algorithms for Interactive Decision Making ICLR 2023 What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models NIPS 2023 Data Selection for Language Models via Importance Resampling NIPS 2023 Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time NIPS 2023 Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models ICML 2023 ​​What learning algorithm is in-context learning? Investigations with linear models ICLR 2023 Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization NIPS 2023 Max-Margin Works while Large Margin Fails: Generalization without Uniform Convergence ICLR 2023 How Sharpness-Aware Minimization Minimizes Sharpness? ICLR 2023 First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains ICLR 2023 A theoretical study of inductive biases in contrastive learning ICLR 2023 Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift UAI 2022 Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments NIPS 2022 Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers NIPS 2022 Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulations NIPS 2022 Sharp Bounds for Federated Averaging (Local SGD) and Continuous Perspective AISTATS 2022 Toward Fast, Flexible, and Robust Low-Light Image Enhancement CVPR 2022 An Explanation of In-context Learning as Implicit Bayesian Inference ICLR 2022 Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution ICLR 2022 DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization ICLR 2022 Self-supervised Learning is More Robust to Dataset Imbalance ICLR 2022 Near-Optimal Algorithms for Autonomous Exploration and Multi-Goal Stochastic Shortest Path ICML 2022 Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification ICML 2022 Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation ICML 2022 Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data ICLR 2021 Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss NIPS 2021 Safe Reinforcement Learning by Imagining the Near Future NIPS 2021 Entity and Evidence Guided Document-Level Relation Extraction IJCNLP 2021 Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap COLT 2021 Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations NIPS 2021 Entity and Evidence Guided Document-Level Relation Extraction ACL 2021 Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration NIPS 2021 Active Online Learning with Hidden Shifting Domains AISTATS 2021 Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning NIPS 2021 Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve Optimism, Embrace Virtual Curvature NIPS 2021 Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling AAAI 2021 Label Noise SGD Provably Prefers Flat Global Minimizers NIPS 2021 Variance-reduced First-order Meta-learning for Natural Language Processing Tasks NAACL 2021 Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for Improved Generalization ICML 2021 Shape Matters: Understanding the Implicit Bias of the Noise Covariance COLT 2021 In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness ICLR 2021 Optimal Regularization can Mitigate Double Descent ICLR 2021 Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization ICLR 2021 Improved Sample Complexities for Deep Neural Networks and Robust Classification via an All-Layer Margin ICLR 2020 Learning Over-Parametrized Two-Layer Neural Networks beyond NTK COLT 2020 Robust and On-the-fly Dataset Denoising for Image Classification ECCV 2020 Learning Self-Correctable Policies and Value Functions from Demonstrations with Negative Sampling ICLR 2020 Beyond Lazy Training for Over-parameterized Tensor Decomposition NIPS 2020 Self-training Avoids Using Spurious Features Under Domain Shift NIPS 2020 MOPO: Model-based Offline Policy Optimization NIPS 2020 Model-based Adversarial Meta-Reinforcement Learning NIPS 2020 Federated Accelerated Stochastic Gradient Descent NIPS 2020 On the Expressivity of Neural Networks for Deep Reinforcement Learning ICML 2020 Understanding Self-Training for Gradual Domain Adaptation ICML 2020 The Implicit and Explicit Regularization Effects of Dropout ICML 2020 Individual Calibration with Randomized Forecasting ICML 2020 Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks NIPS 2019 Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel NIPS 2019 Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss NIPS 2019 Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation NIPS 2019 On the Performance of Thompson Sampling on Logistic Bandits COLT 2019 Approximability of Discriminators Implies Diversity in GANs ICLR 2019 Fixup Initialization: Residual Learning Without Normalization ICLR 2019 Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees ICLR 2019 Verified Uncertainty Calibration NIPS 2019 A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors ACL 2018 Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations COLT 2018 Gradient Descent Learns Linear Dynamical Systems JMLR 2018 Learning One-hidden-layer Neural Networks with Landscape Design ICLR 2018 Generalization and Equilibrium in Generative Adversarial Nets (GANs) ICML 2017 On the Optimization Landscape of Tensor Decompositions NIPS 2017 On the Ability of Neural Nets to Express Distributions COLT 2017 Distributed Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement JMLR 2017 A Non-generative Framework and Convex Relaxations for Unsupervised Learning NIPS 2016 Matrix Completion has No Spurious Local Minimum NIPS 2016 Provable Algorithms for Inference in Topic Models ICML 2016 Online Learning of Eigenvectors ICML 2015 Simple, Efficient, and Neural Algorithms for Sparse Coding COLT 2015 Sum-of-Squares Lower Bounds for Sparse PCA NIPS 2015 Provable Bounds for Learning Some Deep Representations ICML 2014 On Communication Cost of Distributed Statistical Estimation and Dimensionality NIPS 2014