Skip to content
conftrace_
2023 EMNLP EMNLP 2023

Abed at KSAA-RD Shared Task: Enhancing Arabic Word Embedding with Modified BERT Multilingual

Abstract

AbstractThis paper presents a novel approach to the Arabic Reverse Dictionary Shared Task at WANLP 2023 by leveraging the BERT Multilingual model and introducing modifications augmentation and using a multi attention head. The proposed method aims to enhance the performance of the model in understanding and generating word embeddings for Arabic definitions, both in monolingual and cross-lingual contexts. It achieved good results compared to benchmark and other models in the shared task 1 and 2.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Abdelrahim Qaddoumi

Topics

Deep Learning > Architectures > Transformers Natural Language Processing > Resources & Methods > Large Language Models Natural Language Processing > Resources & Methods > Lexical Semantics Machine Learning > Learning Types > Transfer Learning Natural Language Processing > Applications > Natural Language Understanding Deep Learning > Models > Language Models

Keywords

transfer learning attention mechanism word embedding multilingual bert arabic language bert multilingual reverse dictionary cross-lingual context arabic definition
Download PDF

More from EMNLP 2023

All EMNLP 2023 papers →
KSAA-RD Shared Task: Arabic Reverse Dictionary
UWB at Arabic Reverse Dictionary shared task: Computing the meaning of a gloss
Qamosy at Arabic Reverse Dictionary shared task: Semi Decoder Architecture for Reverse Dictionary with SBERT Encoder
Rosetta Stone at KSAA-RD Shared Task: A Hop From Language Modeling To Word–Definition Alignment
ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text
DetectiveRedasers at ArAIEval Shared Task: Leveraging Transformer Ensembles for Arabic Deception Detection