conftrace_

singular value distribution

3 papers

Explore in graph

Co-occurring keywords

mixed-precision quantization (40) delta compression (5) dynamical isometry (8) free probability theory (4) gradient propagation (34) sigmoid activation (6) large language model (13587) dale's law (1) excitatory inhibitory (1) weight quantization (137)

Papers

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models NIPS 2024
Learning better with Dale’s Law: A Spectral Perspective NIPS 2023
Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice NIPS 2017