conftrace_

neural network optimization

1293 papers

Explore in graph

Co-occurring keywords

model compression (3302) neural network (6616) convolutional neural network (4226) gradient descent (1144) stochastic gradient descent (1091) efficient computing (781) deep neural network (1803) large language model (13587) knowledge distillation (3725) representation learning (6206)

Papers

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers ACL 2023
B2T Connection: Serving Stability and Performance in Deep Transformers ACL 2023
Transformer-Based Learned Optimization CVPR 2023
FouriDown: Factoring Down-Sampling into Shuffling and Superposing NIPS 2023
Temporal Graph Neural Networks for Irregular Data AISTATS 2023
Balanced Training for Sparse GANs NIPS 2023
What can a Single Attention Layer Learn? A Study Through the Random Features Lens NIPS 2023
Efficient Activation Function Optimization through Surrogate Modeling NIPS 2023
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective NIPS 2023
RLEKF: An Optimizer for Deep Potential with Ab Initio Accuracy AAAI 2023
Feature-Learning Networks Are Consistent Across Widths At Realistic Scales NIPS 2023
AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix NIPS 2023
Making Scalable Meta Learning Practical NIPS 2023
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers NIPS 2023
Block Low-Rank Preconditioner with Shared Basis for Stochastic Optimization NIPS 2023
Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals NIPS 2023
A Guide Through the Zoo of Biased SGD NIPS 2023
GradMA: A Gradient-Memory-Based Accelerated Federated Learning With Alleviated Catastrophic Forgetting CVPR 2023
ActMAD: Activation Matching To Align Distributions for Test-Time-Training CVPR 2023
Network Expansion for Practical Training Acceleration CVPR 2023
YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors CVPR 2023
Understanding Self-attention Mechanism via Dynamical System Perspective ICCV 2023
Pruning Parameterization With Bi-Level Optimization for Efficient Semantic Segmentation on the Edge CVPR 2023
Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers CVPR 2023
DeepMAD: Mathematical Architecture Design for Deep Convolutional Neural Network CVPR 2023