Nathan Godey
4 papers · 2022–2024 · 4 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓
🌍
Conference Polyglot · Bronze
🌉
Interdisciplinary Bridge · Bronze
Venues
Top co-authors
Research topics
Keywords
hidden representation
(2)
contrastive learning
(1)
self-supervised learning
(1)
language modeling
(1)
language model
(1)
scaling law
(1)
model scaling
(1)
end-to-end training
(1)
cross-entropy loss
(1)
subword tokenization
(1)
weight tying
(1)
geographical bia
(1)
representation degeneration
(1)
geographical representation
(1)
angular distance
(1)
character perturbation
(1)
differentiable tokenization
(1)
robust language model
(1)
transformer architecture
(1)
self-attention anisotropy
(1)
Papers
Anisotropy Is Inherent to Self-Attention in Transformers
EACL 2024
Headless Language Models: Learning without Predicting with Contrastive Weight Tying
ICLR 2024
On the Scaling Laws of Geographical Representation in Language Models
COLING 2024
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling
EMNLP 2022