subword tokenization

AdaptBPE: From General Purpose to Specialized Tokenizers EACL 2026 Contextual morphologically-guided tokenization for Latin encoder models EACL 2026 Corpus-Dependent Subcharacter Encoding via HMM-Guided Code Assignment ACL 2026 Evaluating Morphological Plausibility of Subword Tokenization via Statistical Alignment with Morpho-Syntactic Features EACL 2026 How Tokenization Limits Phonological Knowledge Representation in Language Models and How to Improve Them ACL 2026 Modelling the Morphology of Verbal Paradigms: A Case Study in the Tokenization of Turkish and Hebrew EACL 2026 Morpheme Matters: Morpheme-Based Subword Tokenization for Korean Language Models EACL 2026 MUTANT: A Recipe for Multilingual Tokenizer Design ACL 2026 ReBPE: Iteratively Improving the Internal Structure of a Structured Tokeniser by Mining its Internal Structure EACL 2026 Stop Taking Tokenizers for Granted: They Are Core Design Decisions in Large Language Models EACL 2026 Taien at SemEval-2026 Task 9: Multilingual Polarization Detection Using Transformer-based Models SemEval 2026 TokDrift: When LLM Speaks in Subwords but Code Speaks in Grammar ACL 2026 Tokenization and Morphological Fidelity in Uralic NLP: A Cross-Lingual Evaluation EACL 2026 Tokenization Cost, Retention, and Orthography Robustness for Ladin and Italian Varieties EACL 2026 When Multilingual Evaluation Assumptions Fail: Tokenization Effects Across Scripts EACL 2026 Word predictability estimates from language models are not robust to tokenizer vocabulary CoNLL 2026 Adapting Large Language Models for Character-based Augmentative and Alternative Communication EMNLP 2025 Adversarial Tokenization ACL 2025 A Morpheme-Aware Child-Inspired Language Model EMNLP 2025 Annotating and Inferring Compositional Structures in Numeral Systems Across Languages ACL 2025 Byte Pair Encoding Is All You Need For Automatic Bengali Speech Recognition AACL 2025 Can Perplexity Predict Finetuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali ACL 2025 Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries ACL 2025 Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learning ACL 2025 Exploring morphology-aware tokenization: A case study on Spanish language modeling EMNLP 2025