Development and Psychometric Validation of an Emotion Reading Interest Dataset for Higher-Education Learners: A Current-Data Audit and Revalidation Framework

Abstract

Emotion aware educational systems need not only technically accurate emotion classifiers, but also access to valid instruments for emotion in education to measure the relations between learners' emotions and the reading interest, the relevance of their recommendations, their engagement as well as their perceived learning outcomes. In this study, a 25-item questionnaire for measuring 5 constructs: Emotional State, Reading Interest, Emotion Based Recommendation, Cognitive Retention, Engagement and Motivation is audited and psychometrically evaluated. There were 500 and 1000 records in two files provided, which were comma-separated. The record-level comparison results showed that the initial file of 500 records from the 1,000-response file did not differ from the entire 500-response file. Therefore, the files were not analysed as independent data, but rather the 1,000-response file was used as one file, and then split into the exploratory subsample of 500 respondents, and the confirmatory holdout of 500 respondents. The dataset contained no missing values in Q5–Q29, no out-of-range responses, no straight-lined response vectors and no duplicate 25-item response patterns. However, critically low internal consistency in the proposed scales was observed. In the full dataset, the values for Cronbach's alpha were between −0.018 and 0.049, and the values for provisional McDonald's omega were between 0.060 and 0.238. Corrected item–total correlation values were clustered around zero. The forced five-factor exploratory factor analysis accounted for a small amount of total item variance (11.30%) and the Kaiser–Meyer–Olkin measure of sampling adequacy was 0.495, while only three items had primary loadings ≥ 0.40. CFI = 0.381 and TLI = 0.300 were obtained in the confirmatory analysis. The composite reliability was not higher than 0.176, while the average value of variance extracted was not higher than 0.084. Eight out of 10 of the HTMT comparisons were above 0.90. Content-validity ratings, cognitive-interview records, test–retest observations, and ethics documentation and item-level reverse-scoring keys were not available. This data set can thus not be presented as a validated five-construct emotion–reading interest data set. The study pinpoints specific measurement failures, and outlines an expert review, cognitive interviewing, pilot testing, ordinal EFA, independent CFA, and measurement-invariance and test–retest procedure for instrument revision and new data collection.