conftrace_

subword tokenizer

6 papers

Explore in graph

Co-occurring keywords · papers mentioning both

subword tokenization (2) language model (2) cross-lingual transfer (2) embedding reparameterization (1) trie structure (1) token index (1) linguistic relationship (1) multilingual tokenization (1) phonological probing (1) language encoding (1)

Papers

Phonemes to the Rescue: Multilingual Tokenization Based on International Phonetic Alphabet ACL 2026 BabyLM’s First Words: Word Segmentation as a Phonological Probing Task CoNLL 2025 Flexible and Efficient Grammar-Constrained Decoding ICML 2025 TriEmbed: Bridge the Gap between Text and Token Indices with Embedding Reparameterization ACL 2025 Language Model Tokenizers Introduce Unfairness Between Languages NeurIPS 2023 Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages ACL 2023