Scaling pseudo-labeling data for end-to-end low-resource speech translation (the case of Kurdish language)

In this paper we propose a pseudo-labeling pipeline to generate End-to-End Speech to Text Translation (E2E S2TT) data for low-resource languages. This pipeline allows us to achieve very promising results on S2TT task without having any parallel speech corpora. The proposed pipeline is composed of a speech segmentation, followed by a speech recognition system and a machine translation system. Our study is performed on Kurdish language which doesn't have resources for E2E S2TT. In our study, we firstly fine-tune and evaluate the ASR and MT systems to achieve a degree of reliability on the these components. The pipeline is used to pseudo-label 3,200 hours of Kurdish speech aligned with English translation. The pseudo-labeled data is extensively evaluated with different E2E S2TT systems such as Seq2Seq Transfomers and Whisper model. Achieving a 20.68 BLEU score on Fleurs benchmark shows the effectiveness of the our approach and its potential for other low-resource languages. The parallel pseudo-labeled data can be retrieved from: https://lium.univ-lemans.fr/en/ckbens2tt/