Neural Network Optimization

TrAct: Making First-layer Pre-Activations Trainable NeurIPS 2024 Trade-Offs of Diagonal Fisher Information Matrix Estimators NeurIPS 2024 Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality (extended abstract) COLT 2024 Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis NeurIPS 2024 Training Integrable Parameterizations of Deep Neural Networks in the Infinite-Width Limit JMLR 2024 Trustless Audits without Revealing Data or Models ICML 2024 TULIP: Transformer for Upsampling of LiDAR Point Clouds CVPR 2024 Uncertainty Aware Learning for Language Model Alignment ACL 2024 Understanding and Improving Optimization in Predictive Coding Networks AAAI 2024 Understanding and Leveraging the Learning Phases of Neural Networks AAAI 2024 Understanding and Minimising Outlier Features in Transformer Training NeurIPS 2024 Understanding Linear Probing then Fine-tuning Language Models from NTK Perspective NeurIPS 2024 Understanding Surprising Generalization Phenomena in Deep Learning AAAI 2024 Understanding the Generalization Benefits of Late Learning Rate Decay AISTATS 2024 United We Stand: Using Epoch-Wise Agreement of Ensembles to Combat Overfit AAAI 2024 Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction NeurIPS 2024 Universal Neural Functionals NeurIPS 2024 Universal Prompt Optimizer for Safe Text-to-Image Generation NAACL 2024 Unlocking Black-Box Prompt Tuning Efficiency via Zeroth-Order Optimization EMNLP 2024 Unlocking Memorization in Large Language Models with Dynamic Soft Prompting EMNLP 2024 Unraveling the Gradient Descent Dynamics of Transformers NeurIPS 2024 Unveiling LoRA Intrinsic Ranks via Salience Analysis NeurIPS 2024 Unveiling The Matthew Effect Across Channels: Assessing Layer Width Sufficiency via Weight Norm Variance NeurIPS 2024 Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers ICML 2024 UWB at WASSA-2024 Shared Task 2: Cross-lingual Emotion Detection ACL 2024