conftrace_

tokenization parity

2 papers

Explore in graph

Also known as

TP

Co-occurring keywords · papers mentioning both

tokenizer fertility (1) information parity (1) compression efficiency (1) non-standard orthography (1) large language model (1) multilingual capabilities (1) multilingual language model (1) text compression (1) low-resource language (1) language model evaluation (1)

Papers

Crowdsourcing Piedmontese to Test LLMs on Non-Standard Orthography EACL 2026 Information Parity: Measuring and Predicting the Multilingual Capabilities of Language Models EMNLP 2024