GAPSI - genetic algorithm approach addressing feature drift using PSI

Abstract

Abstract Feature drift, characterized by temporal shifts in the statistical and contextual properties of input features, presents a persistent challenge in data such as Twitter (X). In these environments, evolving language patterns and user sentiment can significantly degrade the performance of sentiment classification models. This study introduces a novel framework that integrates the population stability index (PSI) for drift detection with a genetic algorithm-based feature selection mechanism (GAPSI) for adaptive reconfiguration of the model’s feature set. The proposed approach achieves a competitive classification accuracy of 91.2% on Twitter data (using Tweepy) while outperforming conventional machine learning models, deep learning architectures, and transformer-based models, such as BERT. Unlike traditional techniques that require continuous retraining, the GAPSI method dynamically reselects the most relevant features in response to drift, allowing for consistent performance with minimal computational overhead. Experimental results across four benchmark datasets demonstrate the robustness, scalability, and adaptability of the framework in real-world sentiment analysis tasks, offering a lightweight yet effective solution for evolving textual data environments.