Walking stability prediction for pedestrians using gait energy images and hybrid deep and few-shot learning models

Abstract

The prediction and recognition of unstable human walking patterns are of high importance for active video surveillance, smart environments, and assistive healthcare, particularly for fall detection in the elderly. This research investigates the utility of Gait Energy Images (GEIs) combined with deep learning, vision transformers, and few-shot learning architectures to enhance the classification of stable and unstable pedestrian walking patterns. We evaluate and compare twelve methodologies: six classical or feature-based machine learning models (Linear SVM, HOG + SVM, LBP + RBF-SVM, Random Forest, XGBoost, and an adapted GaitSet baseline), three deep learning models (MobileNet, Vision Transformer (ViT), and YOLOv8-cls), and three episodic few-shot learning techniques (Prototypical, Matching, and Relation Networks) under data-scarcity regimes. To facilitate this evaluation, we introduce the Unstable and Stable Walking Pedestrian (USWP) dataset, constructed by fusing and harmonizing sequences from seven public action recognition databases, containing 3250 unique GEIs with a subject-independent evaluation protocol to prevent identity-based domain leakage. Our experiments demonstrate that the YOLOv8-cls model achieves an overall accuracy of 96.92% (97.14% on Loss-of-Balance anomalies and 94.67% on Active Motion anomalies), significantly outperforming the conventional Linear SVM baseline (75.38%) and MobileNet (91.08%). Conversely, Relation Networks exhibit lower few-shot performance (71.08%) due to optimization complexities in learning similarity metrics from sparse data. Leave-One-Dataset-Out (LODO) cross-validation reveals an average generalization accuracy of 83.28%, indicating significant domain bias across source databases and underscoring that within-dataset evaluations overestimate real-world generalization. Computational complexity analysis shows that MobileNet provides an optimal trade-off for real-time edge deployment (4.2 ms latency), while preprocessing ablation studies demonstrate that integrating the Segment Anything Model (SAM) with MediaPipe-derived Regions of Interest (ROIs) yields a 12.50% absolute improvement in accuracy by eliminating background noise.