The impact of dynamic feedback on AI tutor performance

Abstract

Introduction . Behavioural adaptive AI systems demonstrate significant potential in providing timely learning support. However, a key aspect of their operation, dynamically switching behavioural patterns based on real-time analysis of streaming data from learners, remains an area requiring further research. Aim . The aim of this study is to assess the impact of a transparent switching policy, based on sentiment and response latency, on student engagement, trust, and academic outcomes, as well as to examine its effect on response latency and expressed sentiment. Methodology and research methods . The authors conducted a randomised, real-world study involving 80 students during a 45-minute session. The experiment compared a dynamic-persona tutor with a fixed-persona baseline tutor. To evaluate the results, the following measures were used: a five-item engagement scale, a five-item trust scale, a curriculum-aligned ten-item pre- and post-knowledge test, log-level tutor-to-learner response latency, and message-level sentiment analysis mapped by a transformer classifier onto a polarity scale ranging from -1 to +1. The role-change algorithm operated as follows: if the rolling mean of sentiment was at or below -0.30, the tutor adopted the role of Empathic Coach; if response latency exceeded ten seconds, the tutor assumed the role of Rational Guide; in all other cases, the tutor remained a Neutral Instructor. Following a role change, there was a one-turn “cooldown” period, and a return to the neutral role occurred after two consecutive stable interactions. Results and scientific novelty . The authors developed a testable role-switching algorithm that selects the optimal interaction strategy in real-time by analysing the learner’s emotional state and response latency. The efficacy of this approach was confirmed in a real-world educational setting. Practical significance . This approach provides a ready-made solution for implementing adaptive learning in real-world educational settings. Its advantages include simple rules, low computational costs, and a transparent auditing system.