Transformer-based Arabic language lemmatization

Abstract

Traditional morphological methods have long been at the heart of Arabic language processing. They exploit the complex morphological structure of the language, characterized by roots, prefixes, infixes, and suffixes used for analyzing and generating words. These methods have been effective for tasks such as lemmatization, and rooting. However, they can be resource-intensive and do not always take semantics into account. Recent advancements in deep learning, particularly transformer-based models, offer new prospects for Arabic language processing. Despite their success in languages such as Turkish, Latin, English, and Slavic, to our knowledge, no transformer-based method has been implemented for the Arabic lemmatization task. In this study, we propose a transformer-based lemmatizer using T5, specifically AraT5, MT5, and BERT. We evaluate the models using the ArabicPADT UD Treebank corpus and benchmark their performance against standard metrics, including accuracy, precision, recall, and F1 score. Our results show an accuracy of $88.58 \%$, demonstrating competitive performance and highlighting the potential of transformer-based models for advancing Arabic lemmatization.