Optimization

Wasserstein Distance Rivals Kullback-Leibler Divergence for Knowledge Distillation NeurIPS 2024 Weight Conditioning for Smooth Optimization of Neural Networks ECCV 2024 Weight decay induces low-rank attention layers NeurIPS 2024 What does guidance do? A fine-grained analysis in a simple setting NeurIPS 2024 White-Box Transformers via Sparse Rate Reduction: Compression Is All There Is? JMLR 2024 Why Do We Need Weight Decay in Modern Deep Learning? NeurIPS 2024 Wino Vidi Vici: Conquering Numerical Instability of 8-Bit Winograd Convolution for Accurate Inference Acceleration on Edge WACV 2024 Win: Weight-Decay-Integrated Nesterov Acceleration for Faster Network Training JMLR 2024 Worst-Case VCG Redistribution Mechanism Design Based on the Lottery Ticket Hypothesis AAAI 2024 ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-Order Optimization AAAI 2024 AAAI New Faculty Highlights: General and Scalable Optimization for Robust AI AAAI 2023 Accelerated On-Device Forward Neural Network Training with Module-Wise Descending Asynchronism NeurIPS 2023 Accelerating the Training of Video Super-resolution Models AAAI 2023 Accelerating Transformer Inference for Translation via Parallel Decoding ACL 2023 A Characteristic Function-Based Method for Bottom-Up Human Pose Estimation CVPR 2023 Achieving O(ε^-1.5) Complexity in Hessian/Jacobian-free Stochastic Bilevel Optimization NeurIPS 2023 Achieving High Accuracy with PINNs via Energy Natural Gradient Descent ICML 2023 ACL-SPC: Adaptive Closed-Loop System for Self-Supervised Point Cloud Completion CVPR 2023 Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction AAAI 2023 Adaptive Perturbation-Based Gradient Estimation for Discrete Latent Variable Models AAAI 2023 Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction NeurIPS 2023 A fast heuristic to optimize time-space tradeoff for large models NeurIPS 2023 AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix NeurIPS 2023 A Hyperparameter Optimization Toolkit for Neural Machine Translation Research ACL 2023 A Learnable Radial Basis Positional Embedding for Coordinate-MLPs AAAI 2023