Efficient Temporal Modeling for Real-World Sign Language Recognition: A Comparative Study Under Data-Constrained Scenarios

Abstract

Designing effective temporal modeling strategies for video-based sign language recognition (SLR) remains challenging, particularly in low-resource settings where the behavior of modern architectures is not fully understood. In this study, we present a controlled comparative evaluation of temporal models, including recurrent architectures (RNN, LSTM, GRU) and a Transformer encoder, within a unified spatio-temporal framework based on a shared MobileNetV2 feature extractor. All models are trained and evaluated under identical conditions on a curated subset of the WLASL dataset (37 classes), ensuring a fair and reproducible comparison. The results show that recurrent models consistently achieve higher performance than the Transformer-based approach in data-constrained scenarios, with the CNN–LSTM model reaching an accuracy of 90.02%. In contrast, the Transformer model exhibits lower generalization capability, which may be attributed to its higher data requirements. Additionally, increasing architectural complexity through hybrid temporal designs does not result in performance improvements. These findings suggest that simpler recurrent architectures remain effective for temporal modeling in limited data settings and highlight the importance of aligning model complexity with data availability for practical SLR applications.