HistoryQuest: Arabic Question Answering in Egyptian History with LLM Fine-Tuning and Transformer Models

Abstract

Question answering (QA) in Egyptian history presents a unique and complex challenge for Arabic natural language processing (NLP). This study aims to explore and assess how large language models (LLMs) can enhance the accuracy and performance of Arabic question answering (QA), specifically in this domain. To conduct this investigation, we utilize two comprehensive datasets: the Arabic History-QA dataset and the Contextual Articles Dataset, which cover pivotal historical periods. We evaluate transformer-based models, including AraBERTv2, BERT-large-Arabic with Retrieval-Augmented Generation (RAG), fine-tuned LLaMa-2, and zero-shot LLaMa-3 with Retrieval-Augmented Generation (RAG). Through a rigorous and detailed evaluation process, we analyze how these models address various questions related to Egyptian history. This research contributes valuable insights into advancing the capabilities of Arabic NLP in specialized domains such as historical question answering. Our best results, summarized as the superiority of LLMs, beat those with transformers; additionally, the RAG significantly raised the performance level overall.