Theory

Why neural networks find simple solutions: The many regularizers of geometric complexity NeurIPS 2022 Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power NeurIPS 2022 Width is Less Important than Depth in ReLU Neural Networks COLT 2022 Your Transformer May Not be as Powerful as You Expect NeurIPS 2022 Zero Stability Well Predicts Performance of Convolutional Neural Networks AAAI 2022 A Central Limit Theorem for Differentially Private Query Answering NeurIPS 2021 A Deeper Look at the Hessian Eigenspectrum of Deep Neural Networks and its Applications to Regularization AAAI 2021 A Diffusion Theory For Deep Learning Dynamics: Stochastic Gradient Descent Exponentially Favors Flat Minima ICLR 2021 Adversarial Robustness Guarantees for Random Deep Neural Networks ICML 2021 A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks AISTATS 2021 A Law of Robustness for Two-Layers Neural Networks COLT 2021 A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks ICLR 2021 Analytic Insights into Structure and Rank of Neural Network Hessian Maps NeurIPS 2021 Analytic Study of Families of Spurious Minima in Two-Layer ReLU Neural Networks: A Tale of Symmetry II NeurIPS 2021 Analyzing the Expressive Power of Graph Neural Networks in a Spectral Perspective ICLR 2021 An Information-Theoretic Characterization of Morphological Fusion EMNLP 2021 Architectural Adversarial Robustness: The Case for Deep Pursuit CVPR 2021 A Recipe for Global Convergence Guarantee in Deep Neural Networks AAAI 2021 A Theoretical Analysis of Fine-tuning with Linear Teachers NeurIPS 2021 A Theoretical Analysis of the Repetition Problem in Text Generation AAAI 2021 A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions ICML 2021 Attention is not all you need: pure attention loses rank doubly exponentially with depth ICML 2021 A Unified Taylor Framework for Revisiting Attribution Methods AAAI 2021 Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networks AAAI 2021 Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information ICML 2021