Skip to content
conftrace_
2026 EMNLP EMNLP 2026

TALIM: A Benchmark Dataset for Tajik Academic Language Intelligence Measurement

AbstractWe introduce TALIM (Tajik Academic Language Intelligence Measurement), a large-scale academic benchmark for evaluating academic reasoning and language understanding in Tajik. TALIM spans nine subjects and three question formats, multiple-choice, matching, and free-response, sourced natively from the Tajik national curriculum rather than translated from existing benchmarks. We evaluate nine LLMs, both proprietary and open-weight models, finding that proprietary models substantially outperform open-weight models across all subjects and question types. Grammar is the weakest subject across all models, while physics is consistently the strongest. English translation substantially improves open-weight performance while Farsi transliteration yields only marginal gains, indicating that failures stem from data scarcity rather than script unfamiliarity. Chain-of-thought analysis reveals several systematic failure modes: hallucination, reasoning process failures, and grammatical category confusion. We hope TALIM will serve as a resource for assessing Tajik language capabilities of frontier models and motivate similar native benchmark construction efforts for other low-resource languages.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →