conftrace_

2020 EMNLP EMNLP 2020

An exploratory approach to the Parallel Corpus Filtering shared task WMT20

Abstract

AbstractIn this document we describe our submission to the parallel corpus filtering task using multilingual word embedding, language models and an ensemble of pre and post filtering rules. We use the norms of embedding and the perplexities of language models along with pre/post filtering rules to complement the LASER baseline scores and in the end get an improvement on the dev set in both language pairs.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Ankur Kejriwal, Philipp Koehn

Topics

Machine Learning > Core Methods > Embedding Learning Machine Learning > Application Areas > Data Augmentation Natural Language Processing > Applications > Machine Translation Natural Language Processing > Resources & Methods > Multilingual NLP Machine Learning > Learning Types > Ensemble Learning

Keywords

machine translation parallel corpus ensemble method language model parallel corpus filtering corpus filtering multilingual word embedding language model perplexity word embedding norm

More from EMNLP 2020

All EMNLP 2020 papers →

Filtering Noisy Parallel Corpus using Transformers with Proxy Task Learning

Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions

Bicleaner at WMT 2020: Universitat d’Alacant-Prompsit’s submission to the parallel corpus filtering shared task

Dual Conditional Cross Entropy Scores and LASER Similarity Scores for the WMT20 Parallel Corpus Filtering Shared Task

Improving Parallel Data Identification using Iteratively Refined Sentence Alignments and Bilingual Mappings of Pre-trained Language Models

Alibaba Submission to the WMT20 Parallel Corpus Filtering Task