TinyML-Based Hand Gesture Recognition for Device Control

Abstract

Hand gesture recognition enables intuitive and contactless device control by translating specific hand movements into commands. This allows users to effortlessly operate smart home systems, consumer electronics, and industrial machinery through natural gestures, enhancing accessibility and user experience without physical interfaces. This paper presents a robust TinyML-based framework for real-time hand gesture recognition, designed for efficient deployment on resource-constrained embedded devices. The system translates eight distinct hand gestures into device control commands using a vision-based approach. A dataset of 8,000 images was collected under varied lighting and angles to enhance generalization. The preprocessing pipeline incorporates hand isolation via MediaPipe, resizing, selective brightness enhancement, and normalization to optimize input data. Three lightweight convolutional neural networks, MobileNetV3Small, NASNetMobile, and DenseNet121, were evaluated and optimized using pruning and full integer quantization. Results show that all models retained high accuracy above $97 \%$ post-compression, with MobileNetV3Small achieving the best performance: $98.7 \%$ accuracy, an inference time of 4 ms, and a model size of 195 KB. This work demonstrates the viability of TinyML for enabling low-latency, privacy-preserving, and energy-efficient gesture-controlled interfaces on microcontrollers.