Evaluating the Effects of BKT-LSTM on Students' Learning Performance

Today, machine learning models and Deep Neural Networks (DNNs) are prevalent in various areas. Also, educational Artificial Intelligence (AI) is drawing increasing attention

with the rapid development of online learning platforms. Researchers explore different types of educational AI to improve students’ learning performance and experience in online classes. Educational AIs can be categorized into “interactive” and “predictive.” Interactive AIs answer simple course questions for students, such as the due day of homework and the final project’s minimum page requirement. Predictive educational AIs play a role in predicting students’ learning states. Instructors can adjust the learning content based on the students’ learning states. However, most AIs are not evaluated in an actual class setting. Therefore, we want to evaluate the effects of a state-of-the-art educational AI model, BKT (Bayesian Knowledge Tracing)-LSTM(Long Short-Term Memory), on students’ learning performance in an actual class setting. Data came from the course CNIT 25501, a large introductory Java program?ming class at Purdue University. Participants were randomly separated into the control and experimental groups (AI-group). Weekly quizzes measured participants’ learning performance. Pre-quiz and base quizzes estimated participants’ prior knowledge levels. Using BKT-LSTM, participants in the experimental group had questions from the knowledge that they were most lacking. However, participants in the control group had questions from randomly picked knowledge. The results suggested that both the experimental and control groups had lower scores in review quizzes than in base quizzes. However, the score difference between base quizzes and review quizzes for the experimental group was more often significantly different (three quizzes) compared to the control group (two quizzes), demonstrating the predictive capability of BKT-LSTM to some extent. Initially, we expected that BKT-LSTM would enhance students’ learning performance. However, in post-quiz, participants in the control group had significantly higher scores than those in the experimental group. The result suggested that continuous complex questions may negatively affect students’ learning initiatives. On the contrary, relatively easy questions may improve their learning initiatives.