Development of a Computer-Based Sign Language Finger-Spelling Translator Using Images

Abstract

Even with advancements in technology, bridging the communication gap between the deaf and non-deaf is not yet perfect. Steps have been taken with the help of Machine Learning, Natural Language Processing (NLP), and Deep Learning to bridge the communication gap between deaf and non-deaf individuals. Machine Learning is used to recognize signs in already defined image libraries. NLP is used to process and translate written or spoken languages, working in conjunction with sign recognition methods. CNN is a deep learning method which is specifically designed for image recognition. The system developed in this work did not use any of these approaches; rather a pre-built image library is used to translate text to sign language. This work focuses on developing a low-cost web-based system for text-to-sign language translation utilizing a pre-built image library. The system leverages a pre-existing database of sign language images. The work explored the development of direct image matching to efficiently translate text to corresponding signs within the pre-built library. This method aims to overcome limitations associated with traditional sign recognition techniques, such as difficulties in handling variations in signing styles and background complexity. Additionally, a web-based system which capitalizes on the widespread availability of internet and mobile devices, promoting accessibility and user-friendliness was developed.