neural network optimization

The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter NeurIPS 2023 The Geometry of Neural Nets' Parameter Spaces Under Reparametrization NeurIPS 2023 The Hessian perspective into the Nature of Convolutional Neural Networks ICML 2023 The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent ICML 2023 The Inside Story: Towards Better Understanding of Machine Translation Neural Evaluation Metrics ACL 2023 The Transient Nature of Emergent In-Context Learning in Transformers NeurIPS 2023 TIPS: Topologically Important Path Sampling for Anytime Neural Networks ICML 2023 TLM: Token-Level Masking for Transformers EMNLP 2023 Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask? NeurIPS 2023 Towards Higher Pareto Frontier in Multilingual Machine Translation ACL 2023 Towards Understanding Generalization of Graph Neural Networks ICML 2023 Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models NeurIPS 2023 Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete NeurIPS 2023 Training Trajectories of Language Models Across Scales ACL 2023 Transformer-Based Learned Optimization CVPR 2023 Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars NeurIPS 2023 Transformers learn through gradual rank increase NeurIPS 2023 Understand and Modularize Generator Optimization in ELECTRA-style Pretraining ICML 2023 Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias ICML 2023 Understanding Self-attention Mechanism via Dynamical System Perspective ICCV 2023 Understanding weight-magnitude hyperparameters in training binary networks ICLR 2023 Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning ICCV 2023 What can a Single Attention Layer Learn? A Study Through the Random Features Lens NeurIPS 2023 What functions can Graph Neural Networks compute on random graphs? The role of Positional Encoding NeurIPS 2023 Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers ACL 2023