conftrace_

pretraining loss

7 papers

Explore in graph

Co-occurring keywords · papers mentioning both

language model (3) learning rate schedule (2) meta analysis (1) loss curve (1) hessian trace (1) power law (1) scaling properties (1) semantic ability (1) syntactic ability (1) speech language model (1)

Papers

A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules ICLR 2025 Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check EMNLP 2025 Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View ICLR 2025 Scaling Properties of Speech Language Models EMNLP 2024 Understanding Emergent Abilities of Language Models from the Loss Perspective NeurIPS 2024 Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models ICML 2023 Code-switched inspired losses for spoken dialog representations EMNLP 2021