Enhancing Arabic healthcare fake news detection with data augmentation and multi-metric analysis using large language models

Abstract

The spread of fake news about healthcare can result in a global health crisis, as it is easy to mislead the public. Detection of fake Arabic news in the healthcare sector is crucial for identifying disinformation, especially in regions where Arabic is the predominant language. Various deep learning and machine learning methods have been proposed to categorize false Arabic news related to healthcare. However, the linguistic diversity of Arabic complicates the development of effective models. Furthermore, the lack of domain-specific high-quality data makes it difficult to build accurate and reliable models. Data augmentation (DA) techniques have shown great promise in addressing these challenges. This study presents a novel technique for expanding Arabic healthcare data by conducting a multi-metric analysis to comprehensively evaluate the quality of the augmented data based on several key aspects, including label preservation, novelty, diversity, and semantic similarity. In the initial phase of our research, we investigated the impact of various data augmentation techniques on widely used classification algorithms. Additionally, similarity thresholds are systematically examined to determine their effect on the classification task. Cosine and Jaccard distances are employed to evaluate the generated sentences in terms of semantics, diversity, novelty, and label preservation. Finally, we propose a novel ensemble augmentation approach that combines multiple DA techniques to generate more varied data. Based on the overall experimental results, the proposed methodology significantly improves the classification of Arabic fake news using AraBERT, with an accuracy increase of 12.1%. In comparison, Random Forest achieved an improvement of 14.7%.