Theory

Towards Interpretable Sequence Continuation: Analyzing Shared Circuits in Large Language Models EMNLP 2024 Towards Large Certified Radius in Randomized Smoothing Using Quasiconcave Optimization AAAI 2024 Towards the Dynamics of a DNN Learning Symbolic Interactions NeurIPS 2024 Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis EMNLP 2024 Training Compute-Optimal Protein Language Models NeurIPS 2024 Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality (extended abstract) COLT 2024 Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis NeurIPS 2024 Transformers Represent Belief State Geometry in their Residual Stream NeurIPS 2024 UMB: Understanding Model Behavior for Open-World Object Detection NeurIPS 2024 Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure NeurIPS 2024 Understanding Pathologies of Deep Heteroskedastic Regression UAI 2024 Understanding Representation of Deep Equilibrium Models from Neural Collapse Perspective NeurIPS 2024 Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data NeurIPS 2024 Understanding Surprising Generalization Phenomena in Deep Learning AAAI 2024 Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling NeurIPS 2024 Understanding Transformer Reasoning Capabilities via Graph Algorithms NeurIPS 2024 Universal In-Context Approximation By Prompting Fully Recurrent Models NeurIPS 2024 Unraveling the Gradient Descent Dynamics of Transformers NeurIPS 2024 Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers NeurIPS 2024 Unveiling Linguistic Regions in Large Language Models ACL 2024 Visual Concept Connectome (VCC): Open World Concept Discovery and their Interlayer Connections in Deep Models CVPR 2024 Weight decay induces low-rank attention layers NeurIPS 2024 What does automatic differentiation compute for neural networks? ICLR 2024 What does guidance do? A fine-grained analysis in a simple setting NeurIPS 2024 What do Graph Neural Networks learn? Insights from Tropical Geometry NeurIPS 2024