Using convolutional neural networks to detect learner's personality based on the Five Factor Model

Abstract

Aiming at the detection of learners' personalities which can help us to enhance the educational learning process, we trained three Convolutional Neural Networks (CNNs) architectures (ResNet50, VGG16, AlexNet) with different datasets for predicting the Five Factor Model (FFM) of personality (Neuroticism, Openness to experience, Extraversion, Conscientiousness and Agreeableness) from the analysis of facial features using Facial Action Coding System (FACS). As well, we compared the three CNNs model results by using multiple evaluation metrics: accuracy, loss, confusion matrix and Reciever Operator Charactetristic- Area under the ROC Curve (ROC-AUC), to decide the most useful model for our use case. Our proposed methodology is based on three steps: The base step where the face’ characteristics are detected and cropped from each video frame using a facial landmark detection algorithm. The second step aims to detect in real time face Action Units (AUs) traits which figured in each frame and compute the highest AU probability appeared on frames sets. The third step is used to decide based on detected AUs combinations personalities according to FFM by using a pre-trained decision tree algorithm. Detected personalities are designed to be stored in a database as a dataset to be exploited and studied in multiple contexts particularly in the analysis of student personality in learning platforms.