conftrace_

training dynamics

118 papers

Explore in graph

Also known as

TD

Co-occurring keywords

neural network optimization (1293) neural network (6616) neural tangent kernel (205) language model (4599) gradient descent (1144) stochastic gradient descent (1091) large language model (13587) representation learning (6206) model scaling (135) convergence analysis (395)

Papers

Soft ascent-descent as a stable and flexible alternative to flooding NIPS 2024
An exactly solvable model for emergence and scaling laws in the multitask sparse parity problem NIPS 2024
Sparse maximal update parameterization: A holistic approach to sparse training dynamics NIPS 2024
Spanning Training Progress: Temporal Dual-Depth Scoring (TDDS) for Enhanced Dataset Pruning CVPR 2024
Evaluating the design space of diffusion-based generative models NIPS 2024
How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics EMNLP 2024
Abrupt Learning in Transformers: A Case Study on Matrix Completion NIPS 2024
Analyzing and Improving the Training Dynamics of Diffusion Models CVPR 2024
Post-Hoc Reversal: Are We Selecting Models Prematurely? NIPS 2024
Tending Towards Stability: Convergence Challenges in Small Language Models EMNLP 2024
Learning, Forgetting, Remembering: Insights From Tracking LLM Memorization During Training EMNLP 2024
Understanding Surprising Generalization Phenomena in Deep Learning AAAI 2024
On Training Data Influence of GPT Models EMNLP 2024
Understanding the Generalization Benefits of Late Learning Rate Decay AISTATS 2024
Why Do We Need Weight Decay in Modern Deep Learning? NIPS 2024
Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models EMNLP 2024
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics NAACL 2024
Transformer verbatim in-context retrieval across time and scale CONLL 2024
Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis NIPS 2024
Investigation into Training Dynamics of Learned Optimizers (Student Abstract) AAAI 2024
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics NIPS 2024
Causal Estimation of Memorisation Profiles ACL 2024
Towards the Dynamics of a DNN Learning Symbolic Interactions NIPS 2024
Identifying Equivalent Training Dynamics NIPS 2024
Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning ACL 2024