conftrace_

subword fertility

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

tokenization efficiency (1) tokenizer optimization (1) embedding layer (1) multilingual adaptation (1) language adaptation (1) byte pair encoding (1) inference speed (1) subword tokenization (1) low-resource language (1) continued pretraining (1)

Papers

An Information-Theoretic Approach to Reducing Fertility in LLMs for Manipuri Machine Translation IJCNLP 2025 R-BPE: Improving BPE-Tokenizers with Token Reuse EMNLP 2025