conftrace_

batch size

34 papers

Explore in graph

Co-occurring keywords

stochastic gradient descent (1091) learning rate (192) large language model (13587) neural network (6616) gradient variance (31) neural network optimization (1293) convergence analysis (395) neural network training (309) memory optimization (114) language model (4599)

Papers

Breaking the Memory Barrier of Contrastive Loss via Tile-Based Strategy CVPR 2025
Exploring smaller batch sizes for a high-performing BabyLM model architecture EMNLP 2025
Training with Fewer Bits: Unlocking Edge LLMs Training with Stochastic Rounding EMNLP 2025
Exact Mean Square Linear Stability Analysis for SGD COLT 2024
How Two-Layer Neural Networks Learn, One (Giant) Step at a Time JMLR 2024
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NIPS 2024
How Do Large Language Models Acquire Factual Knowledge During Pretraining? NIPS 2024
Resolving Discrepancies in Compute-Optimal Scaling of Language Models NIPS 2024
Sigmoid Loss for Language Image Pre-Training ICCV 2023
Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions ICML 2023
Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule ICML 2023
Simplifying Neural Network Training Under Class Imbalance NIPS 2023
Small batch deep reinforcement learning NIPS 2023
Learning Rates as a Function of Batch Size: A Random Matrix Theory Approach to Neural Network Training JMLR 2022
The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NIPS 2022
Batch size-invariance for policy optimization NIPS 2022
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NIPS 2022
Remember More by Recalling Less: Investigating the Role of Batch Size in Continual Learning with Experience Replay (Student Abstract) AAAI 2021
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates AISTATS 2021
SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement NIPS 2021
Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup ACL 2021
Understanding Gradient Clipping In Incremental Gradient Methods AISTATS 2021
An Empirical Study of Stochastic Gradient Descent with Structured Covariance Noise AISTATS 2020
Understanding the Effects of Batching in Online Active Learning AISTATS 2020
Stochastic Optimization with Laggard Data Pipelines NIPS 2020