Nathan Godey
4 papers · 2022–2024 · 4 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓
🌍
Conference Polyglot
(4)
🌉
Interdisciplinary Bridge
🧭
Keyword Pioneer
🐝
Cross-Pollinator
(15)
Conferences
COLING (1)
EACL (1)
EMNLP (1)
ICLR (1)
Top co-authors
Keywords
hidden representation
(2)
language model
(1)
scaling law
(1)
model scaling
(1)
end-to-end training
(1)
cross-entropy loss
(1)
subword tokenization
(1)
geographical bia
(1)
representation degeneration
(1)
geographical representation
(1)
angular distance
(1)
character perturbation
(1)
differentiable tokenization
(1)
robust language model
(1)
transformer architecture
(1)
self-attention anisotropy
(1)
Papers
On the Scaling Laws of Geographical Representation in Language Models
COLING 2024
Anisotropy Is Inherent to Self-Attention in Transformers
EACL 2024
Headless Language Models: Learning without Predicting with Contrastive Weight Tying
ICLR 2024
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling
EMNLP 2022