conftrace_

bigram distribution

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

unigram distribution (1) softmax optimization (1) noise distribution (1) knowledge retrieval (1) softmax approximation (1) noise contrastive estimation (1) induction head (1) associative memory (1) neural network optimization (1) language model training (1)

Papers

Birth of a Transformer: A Memory Viewpoint NeurIPS 2023 An experimental analysis of Noise-Contrastive Estimation: the noise distribution matters EACL 2017