conftrace_

bpe tokenizer

5 papers

Explore in graph

Co-occurring keywords · papers mentioning both

subword tokenization (2) large language model (1) bpe tokenization (1) pretraining data (1) surprisal estimate (1) cross-lingual vocabulary transfer (1) data mixture inference (1) date fragmentation (1) token frequency analysis (1) word predictability (1)

Papers

Word predictability estimates from language models are not robust to tokenizer vocabulary CoNLL 2026 Date Fragments: A Hidden Bottleneck of Tokenization for Temporal Reasoning EMNLP 2025 Dictionaries to the Rescue: Cross-Lingual Vocabulary Transfer for Low-Resource Languages Using Bilingual Dictionaries ACL 2025 Data Mixture Inference Attack: BPE Tokenizers Reveal Training Data Compositions NeurIPS 2024 Optimized Tokenization for Transcribed Error Correction EMNLP 2023