Optimizing Student Learning Outcomes in Virtual Classrooms Using Hierarchical Reinforcement Learning (HRL)

Abstract

Virtual learning environments have come with unique problems related to personalizing, optimizing, and tailoring learning experiences to the needs of the learner. Traditionally, there have been no methods for personalization because existing electronic learning systems use fixed instructional methods that do not consider the changing behavioral, cognitive, and motivational states of the learner. This research presents a Hierarchical Reinforcement Learning (HRL) framework dubbed HRL-VCO (Hierarchical Reinforcement Learning for Virtual Classroom Optimization) for optimizing the learning experience of students in virtual classroom settings by adapting instruction, difficulty, and engagement levels over different timescales. The HRL-VCO framework is made up of two hierarchical layers: a top-layer meta-controller responsible for deciding on the high-level teaching goals (session goals, topic scheduling), and a bottom-layer sub-policy network for implementing micro-level actions (quiz difficulty, session pacing, hint provision). The HRL-VCO model was developed and tested using the EdNet Dataset, which contained 131+ million interactions collected from 784,309 users. The experiments showed that HRL-VCO achieved 91.4% accuracy in predicting user activity, significantly outperforming other models, such as DQN, with its 83.2%, PPO, which gave 85.7%, and a supervised learning method, which yielded 78.6%. In addition, HRL-VCO showed an F1-score of 89.3%, a precision of 90.1%, and a recall of 88.6%, with a mean reward increase by 34.2% in comparison with flat RL baseline models. It has been proven that the policy hierarchical approach helped improve reward accumulation by 12.7% compared to flat RL methods. The obtained results clearly show that hierarchical reinforcement learning can bring a real revolution in the field of adaptive learning systems.