conftrace_

multilingual tokenizer

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

language plasticity (1) language-aware pre-tokenization (1) large language model (1) zipf law (1) language adaptation (1) tokenizer evaluation (1) scaling consistency (1) multilingual tokenization (1) intrinsic metric (1) byte pair encoding (1)

Papers

MUTANT: A Recipe for Multilingual Tokenizer Design ACL 2026 One Tokenizer To Rule Them All: Emergent Language Plasticity via Multilingual Tokenizers ACL 2026 Beyond Text Compression: Evaluating Tokenizers Across Scales ACL 2025