conftrace_

neural network optimization

1293 papers

Explore in graph

Co-occurring keywords

model compression (3302) neural network (6616) convolutional neural network (4226) gradient descent (1144) stochastic gradient descent (1091) efficient computing (781) deep neural network (1803) large language model (13587) knowledge distillation (3725) representation learning (6206)

Papers

Acceleration of Large Transformer Model Training by Sensitivity-Based Layer Dropping AAAI 2023
Block Low-Rank Preconditioner with Shared Basis for Stochastic Optimization NIPS 2023
Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers NIPS 2023
Neural Architecture Search for Wide Spectrum Adversarial Robustness AAAI 2023
DMIS: Dynamic Mesh-Based Importance Sampling for Training Physics-Informed Neural Networks AAAI 2023
A Survey on Model Compression and Acceleration for Pretrained Language Models AAAI 2023
Mimetic Initialization of Self-Attention Layers ICML 2023
A Modern Look at the Relationship between Sharpness and Generalization ICML 2023
SERF: Towards Better Training of Deep Neural Networks Using Log-Softplus ERror Activation Function WACV 2023
DoG is SGD’s Best Friend: A Parameter-Free Dynamic Step Size Schedule ICML 2023
Resilient Binary Neural Network AAAI 2023
A Guide Through the Zoo of Biased SGD NIPS 2023
Learnable Surrogate Gradient for Direct Training Spiking Neural Networks IJCAI 2023
Lon-eå at SemEval-2023 Task 11: A Comparison of Activation Functions for Soft and Hard Label Prediction SEMEVAL 2023
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers EACL 2023
Deterministic Nonsmooth Nonconvex Optimization COLT 2023
Fed-LAMB: Layer-wise and Dimension-wise Locally Adaptive Federated Learning UAI 2023
Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask? NIPS 2023
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets WACV 2023
Continual Learning with Scaled Gradient Projection AAAI 2023
ModelKeeper: Accelerating DNN Training via Automated Training Warmup NSDI 2023
AAAI New Faculty Highlights: General and Scalable Optimization for Robust AI AAAI 2023
Regret Guarantees for Online Deep Control L4DC 2023
Probabilistic Weight Fixing: Large-scale training of neural network weight uncertainties for quantisation. NIPS 2023
Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems ICML 2023