skip to main content skip to footer

Statistical Report of Fall 2009 CBAL Reading Tests CBAL IRT

Author(s):
Fu, Jianbin; Wise, Maxwell; Chung, Seunghee
Publication Year:
2012
Report Number:
RM-12-12
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
51
Subject/Key Words:
Cognitively Based Assessment of, for, and as Learning (CBAL), Reading Assessment, Item Analysis, Item Response Theory (IRT), Statistical Reports

Abstract

In the Cognitively Based Assessment of, for, and as Learning (CBAL) research initiative, we develop innovative K-12 prototype tests based on cognitive competency models. In this report, we present the statistical results of 2 CBAL Grade 7 reading tests administered to students in 9 states in fall 2009. Specifically, classical item statistics including rater reliability for human scored items, item P+, item-total correlations, item missing rates, differential item functioning, interscore correlations, and reliabilities of subscores and total scores were reported. Using item response theory, tests were calibrated and scaled based on the generalized partial credit model. Results from the concurrent calibration and the separate calibration with anchor linking were very similar and interchangeable. In addition, t-tests, multiple comparisons, and mixed models were used to examine the factors influencing test scores including demographic group, test form, test-order, student, and school. The results show that these 2 tests, as well as the linking sets, performed reasonably well.

Read More