Arabic Sign Language Recognition using MediaPipe and Deep Neural Network

Abstract

Individuals with hearing impairments who use Arabic Sign Language (ArSL) often face communication barriers in daily life due to societal unfamiliarity with sign language, limiting their social and professional integration. This paper introduces a high-accuracy ArSL recognition system leveraging MediaPipe for feature extraction and a Deep Neural Network (DNN) for classification. To improve performance and reduce computational complexity, only the most relevant features were collected. By capturing only 21 hand landmark coordinates via MediaPipe, a personally collected dataset of 7,010 samples covering 31 Arabic signs was created. New geometric features (means, angles and distances) have been created to improve spatial understanding using feature engineering techniques, which produce new, more informative features and capture better geometric relationships between hand landmarks. The proposed DNN model achieved a test accuracy of 99.71%, demonstrating near-perfect recognition capability. These results validate the effectiveness of combining MediaPipe for efficient feature extraction with DNN modeling and strategic engineered feature. Furthermore, the proposed research advances accessible communication tools for Hearing-impaired Arabic individuals and provides a scalable methodology applicable to other sign languages.