Self-Supervised Models of Speech Processing for Haitian Creole

We develop tailored speech processing models for Haitian Creole positioning it as a high-resource language in terms of SSL models of speech processing. We do so by pretraining monolingual Wav2Vec2-Base, Wav2Vec2-Large and Data2Vec-Audio-Base models from scratch, which are then finetuned on an Automatic Speech Recognition task. We compare the performance of these models with models finetuned from larger multilingual (XLSR-53, XLSR2-300m, MMS-1B) and monolingual French-based models (LeBenchmark 1 to 7K). Our results highlight the effectiveness of pretraining monolingual models from scratch, demonstrating that they can achieve performance comparable to or even surpassing larger models derived from large-scale multilingual pretraining. Our work provides essential resources for robust Haitian Creole speech recognition and offers pretrained models that can be adapted to other French-based Caribbean Creoles, opening new avenues for linguistic research and practical applications in the region.