Modeling Student's Response Time in a Time Constrained Online Adaptive Testing Setting

<p>Many universities have now shifted courses to an online instruction format which may cause tremendous challenges for teaching and testing. Without the regulation of in-class testing, cheating is unavoidable, and students may take advantage of it to improve their test scores. Traditional item selection methods in Computerized Adaptive Testing (CAT) consider item usage efficiency and test security using test proctors, but seldom consider testing would occur at home, especially after the pandemic. Response Time (RT) is a process data type newly introduced into Computerized Adaptive Testing (CAT). Researchers found that RT often provides insight into students’ testing behavior, solution strategy, and cognitive demands of items. Most of the item selection methods in CAT with RT have primarily focused on the critical parts: measurement accuracy of examinees’ ability, test efficiency, and security. However, it is essential to include content balancing from a practical operation test aspect. Especially in the post-pandemic period, students study hybrid by taking online assessments for university courses. This dissertation consists of two different but rather closely related objectives: 1). Utilize response time to assist item selection method in a content constrained CAT and indirectly reduce students’ action of being dishonest. A time-constrained content balancing CAT item selection algorithm is proposed for a large-scale university course that can indirectly limit the instance and benefits of cheating. Students will rely more on their proficiency level rather than intend to cheat under the scenario of low benefits of cheating, and high risk of being caught. 2). Besides reducing students’ dishonest actions, cheat detection is a necessary process post-assessment. Machine Learning (ML) application is incorporated as a cheat detection method to compare with the traditional psychometric detection methods. Below is an outline of this dissertation.</p>