ML-Based Real-Time Gesture Recognition Platform : Mouse Replacement as a Case-Study

Abstract

This work proposes a novel neural network-based cursor control system, using hand gestures captured from a webcam. The system will allow the user to navigate the computer cursor using their hand and cursor functions, such as right and left clicks. This will be performed using different hand gestures. The proposed system uses nothing more than a low-resolution webcam and it is able to track the user's hand in two dimensions and can recognize up to five hand gestures, which are interpreted as mouse functions. The proposed algorithm provides an accuracy of 94 % for the detection and 82% for the classification. Our work is a real-time application with 0.4ms detection time and 300ms for the classification on a low-end machine.