Skip to content
conftrace_

byte token

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

multilingual language model (1) byte-level tokenization (1) adaptive tokenization (1) non-latin script (1) language fairness (1) gradient-based tokenization (1) subword tokenization (1) multilingual fairness (1) language model (1) multilingual nlp (1)

Papers

MAGNET: Improving the Multilingual Fairness of Language Models with Adaptive Gradient-Based Tokenization NeurIPS 2024