Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

AbstractIn this paper, we present 1020 Group’s submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low-cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →