DEVELOPING AND VALIDATING A HIGHER-ORDER THINKING AND INNOVATION COMPETENCY ASSESSMENT: PSYCHOMETRIC EVIDENCE FOR A ROBUST ADAPTIVE TESTING ITEM BANK

Abstract

This study aimed to develop and validate an assessment tool for evaluating higher-order thinking and innovation competency among Thai upper secondary students, with the ultimate goal of creating a high-quality item bank for future Computerized Adaptive Testing (CAT) applications. The instrument was designed as a situational judgment test with four response options aligned with Bloom’s revised taxonomy. The development process included expert validation, pilot testing, and large-scale administration to a multi-stage random sample of 714 students from four provinces in Northeastern Thailand. Confirmatory Factor Analysis (CFA) was conducted to verify the construct validity, and Item Response Theory (IRT) with the Graded Response Model (GRM) was applied to assess item parameters and test information. Reliability was examined through Cronbach’s alpha analyses. The CFA indices met recommended thresholds (χ² = 37.343, df = 25, p = .0535; RMSEA = 0.025; SRMR = 0.022; CFI = 0.992; TLI = 0.975.), indicating good model fit. GRM analysis revealed high discrimination parameters (a = 0.82–2.15) and well-ordered threshold parameters. Cronbach’s alpha was 0.927. Competency cut scores were determined from Wright Map analysis of GRM threshold parameters, classifying students into four proficiency levels: (1) Insufficient competency (θ ≤ –1.31) ; (2) Emerging competency (–1.31 < θ ≤ –0.35) ; (3) Competent (–0.35 < θ ≤ 0.56); and (4) High competency (θ > 0.56)