Skip to content
conftrace_
2021 EACL EACL 2021

SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identification

Abstract

AbstractIn this paper we present our submission for the EACL 2021-Shared Task on Offensive Language Identification in Dravidian languages. Our final system is an ensemble of mBERT and XLM-RoBERTa models which leverage task-adaptive pre-training of multilingual BERT models with a masked language modeling objective. Our system was ranked 1st for Kannada, 2nd for Malayalam and 3rd for Tamil.

Authors

Sai Muralidhar Jayanthi, Akshat Gupta

Topics

Natural Language Processing > Applications > Text Classification Natural Language Processing > Resources & Methods > Large Language Models

Keywords

ensemble learning offensive language detection masked language modeling multilingual bert task-adaptive pre-training
Download PDF

More from EACL 2021

All EACL 2021 papers →
Simon @ DravidianLangTech-EACL2021: Meme Classification for Tamil with BERT
Bitions@DravidianLangTech-EACL2021: Ensemble of Multilingual Language Models with Pseudo Labeling for offence Detection in Dravidian Languages
NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes
SSNCSE_NLP@DravidianLangTech-EACL2021: Offensive Language Identification on Multilingual Code Mixing Text
JUNLP@DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Langauges
MUCS@DravidianLangTech-EACL2021:COOLI-Code-Mixing Offensive Language Identification