conftrace_

neural network optimization

1293 papers

Explore in graph

Co-occurring keywords

model compression (3302) neural network (6616) convolutional neural network (4226) gradient descent (1144) stochastic gradient descent (1091) efficient computing (781) deep neural network (1803) large language model (13587) knowledge distillation (3725) representation learning (6206)

Papers

Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems ICML 2023
Equilibrium Point Learning ACML 2023
Stabilizing Transformer Training by Preventing Attention Entropy Collapse ICML 2023
Acceleration of Large Transformer Model Training by Sensitivity-Based Layer Dropping AAAI 2023
A Survey on Model Compression and Acceleration for Pretrained Language Models AAAI 2023
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning ICML 2023
Resilient Binary Neural Network AAAI 2023
Making Scalable Meta Learning Practical NIPS 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization ICML 2023
The Hessian perspective into the Nature of Convolutional Neural Networks ICML 2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient ICML 2023
Benign Overfitting in Two-layer ReLU Convolutional Neural Networks ICML 2023
Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks ICML 2023
Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons ICML 2023
Width and Depth Limits Commute in Residual Networks ICML 2023
Learning to Generalize Provably in Learning to Optimize AISTATS 2023
Hyperblock Floating Point: Generalised Quantization Scheme for Gradient and Inference Computation WACV 2023
Block Low-Rank Preconditioner with Shared Basis for Stochastic Optimization NIPS 2023
AutoGraph: Optimizing DNN Computation Graph for Parallel GPU Kernel Execution AAAI 2023
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets WACV 2023
Leveraging the two-timescale regime to demonstrate convergence of neural networks NIPS 2023
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks NIPS 2023
Mitigating Over-smoothing in Transformers via Regularized Nonlocal Functionals NIPS 2023
Lookaround Optimizer: $k$ steps around, 1 step average NIPS 2023
Train Faster, Perform Better: Modular Adaptive Training in Over-Parameterized Models NIPS 2023