Arabic Sign Language Detection Using MediaPipe and Machine Learning Techniques

Abstract

Arabic Sign Language (ArSL) detection represents an important area of research in assistive communication technology for sign language users. This study investigates a machine learning (ML) approach for ArSL detection. A personally collected dataset containing 7010 samples representing 31 distinct signs is used. Hand coordinates were extracted from video frames using MediaPipe framework and subsequently transformed for ML analysis. Feature engineering techniques generate novel features capturing crucial geometric relationships between hand landmarks. Six ML algorithms (Random Forest Classifier, Decision Tree Classifier, Support Vector Machine, K-Nearest Neighbors Classifier, Multi-layer Perceptron Classifier, and AdaBoost Classifier) are evaluated across various preprocessing methods. The Random Forest Classifier, optimized by feature importance analysis, consistently achieves the highest performance metrics, demonstrating its effectiveness for ArSL sign classification. While the learning curve analysis suggests potential benefits from a larger dataset, this investigation paves the way for further exploration of advanced techniques and feature engineering to enhance the accuracy and robustness of future ArSL detection systems.