conftrace_

orthogonal initialization

4 papers

Explore in graph

Co-occurring keywords · papers mentioning both

layer similarity (1) initialization scheme (1) large language model (1) infinite width (1) dynamical isometry (1) cosine similarity (1) weight matrix (1) deep learning optimization (1) training dynamics (1) linear network (1)

Papers

DOCS: Quantifying Weight Similarity for Deeper Insights into Large Language Models ICLR 2025 On the Neural Tangent Kernel of Deep Networks with Orthogonal Initialization IJCAI 2021 Information Geometry of Orthogonal Initializations and Training ICLR 2020 Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks ICLR 2020