Tom Kocmi

AI-Assisted Human Evaluation of Machine Translation NAACL 2025 Command-A-Translate: Raising the Bar of Machine Translation with Difficulty Filtering EMNLP 2025 Estimating Machine Translation Difficulty EMNLP 2025 Findings of the WMT25 General Machine Translation Shared Task: Time to Stop Evaluating on Easy Test Sets EMNLP 2025 Findings of the WMT25 Multilingual Instruction Shared Task: Persistent Hurdles in Reasoning, Generation, and Evaluation EMNLP 2025 Findings of the WMT25 Shared Task on Automated Translation Evaluation Systems: Linguistic Diversity is Challenging and References Still Help EMNLP 2025 Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task EMNLP 2024 Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models ACL 2024 Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation EMNLP 2024 Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet EMNLP 2024 Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies ACL 2024 Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References NAACL 2024 SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window NAACL 2024 Cometoid: Distilling Strong Reference-based Machine Translation Metrics into Even Stronger Quality Estimation Metrics EMNLP 2023 eBLEU: Unexpectedly Good Machine Translation Evaluation Using Simple Word Embeddings EMNLP 2023 Evaluating Metrics for Document-context Evaluation in Machine Translation EMNLP 2023 Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet EMNLP 2023 Findings of the WMT 2023 Shared Task on Machine Translation with Terminologies EMNLP 2023 GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4 EMNLP 2023 Poor Man’s Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference EACL 2023 Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent EMNLP 2023 Findings of the 2022 Conference on Machine Translation (WMT22) EMNLP 2022 MS-COMET: More and Better Human Judgements Improve Metric Performance EMNLP 2022 NTREX-128 – News Test References for MT Evaluation of 128 Languages AACL 2022 Results of WMT22 Metrics Shared Task: Stop Using BLEU – Neural Metrics Are Better and More Robust EMNLP 2022 Searching for a Higher Power in the Human Evaluation of MT EMNLP 2022 Findings of the 2021 Conference on Machine Translation (WMT21) EMNLP 2021 On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs EACL 2021 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation EMNLP 2021 CUNI Submission for the Inuktitut Language in WMT News 2020 EMNLP 2020 CUNI Systems for the Unsupervised and Very Low Resource Translation Task in WMT20 EMNLP 2020 Findings of the 2020 Conference on Machine Translation (WMT20) EMNLP 2020 Gender Coreference and Bias Evaluation at WMT 2020 EMNLP 2020 CUNI Submission for Low-Resource Languages in WMT News 2019 ACL 2019 CUNI Basque-to-English Submission in IWSLT18 EMNLP 2018 CUNI Submissions in WMT18 EMNLP 2018 Trivial Transfer Learning for Low-Resource Neural Machine Translation EMNLP 2018 LanideNN: Multilingual Language Identification on Character Window EACL 2017