Skip to content
conftrace_
2020 EMNLP EMNLP 2020

Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions

Abstract

AbstractThis paper presents the description of our submission to WMT20 sentence filtering task. We combine scores from custom LASER built for each source language, a classifier built to distinguish positive and negative pairs and the original scores provided with the task. For the mBART setup, provided by the organizers, our method shows 7% and 5% relative improvement, over the baseline, in sacreBLEU score on the test set for Pashto and Khmer respectively.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing
🐣 Hot Topic Early Bird - low-resource language

Authors

Muhammad ElNokrashy, Amr Hendy, Mohamed Abdelghaffar, Mohamed Afify, Ahmed Tawfik, Hany Hassan Awadalla

Topics

Machine Learning > Core Methods > Classification Machine Learning > Application Areas > Domain Adaptation Machine Learning > Application Areas > Efficient Computing Natural Language Processing > Applications > Machine Translation Machine Learning > Learning Types > Ensemble Learning

Keywords

machine translation neural machine translation cross-lingual embedding low-resource language sentence embedding sentence alignment parallel corpus filtering low-resource machine translation sentence filtering laser embedding score combination
Download PDF

More from EMNLP 2020

All EMNLP 2020 papers →
Towards a Better Evaluation of Metrics for Machine Translation
Incorporate Semantic Structures into Machine Translation Evaluation via UCCA
Filtering Noisy Parallel Corpus using Transformers with Proxy Task Learning
Bicleaner at WMT 2020: Universitat d’Alacant-Prompsit’s submission to the parallel corpus filtering shared task
An exploratory approach to the Parallel Corpus Filtering shared task WMT20
Dual Conditional Cross Entropy Scores and LASER Similarity Scores for the WMT20 Parallel Corpus Filtering Shared Task