English sign language corpus with real-time mobile recognition and NLP for phrase interpretation

Abstract

Background Recognizing sign language has emerged as a crucial method for closing communication barriers between the general public and hearing and speech-impaired individuals. Purpose In order to facilitate instantaneous recognition in mobile apps, this article presents an extensive collection of sign language in English. Research Design In contrast to earlier datasets, our collection includes a large number of signers, a wide range of hand gestures, and a diversity of environmental variables, guaranteeing reliable performance in a range of real-world situations. Data Analysis Our method improves real-time language of signs recognition and transforms gestures to speech with low latency by utilizing portable optimization approaches. High-resolution data, a specific English dataset, and dataset variety all work together to greatly enhance model generalization by eliminating errors imposed on by changes in user demographics, illumination, and hand placement as well as phrase interpretation using Natural Language Processing model. Conclusions By creating more accessible and inclusive mobile applications, our research opens the door for smooth sign-to-speech translation across linguistic and cultural borders.