Variability in Percentage Above Cut Scores Due to Discreteness in Score Scale IRT

Author(s):

Lu, Ying

Publication Year:

2017

Report Number:

RR-17-32

Source:

ETS Research Report

Document Type:

Report

Page Count:

13

Subject/Key Words:

Cut Scores, Score Scale, Accountability, Growth, Elementary Secondary Education, Item Response Theory (IRT), Standards-Based Tests, Mastery Tests

Abstract

For standard- or criterion-based assessments, the use of cut scores to indicate mastery, nonmastery, or different levels of skill mastery is very common. As part of performance summary, it is of interest to examine the percentage of examinees at or above the cut scores (PAC) and how PAC evolves across administrations. This paper shows that discreteness in score scales can affect the PAC statistics considerably, especially when the test is short and when there are high examinee density and low item density near the cut score on the reporting score scale. This paper also includes recommendations on how to adjust the PAC statistics when they are used in trend analyses.

Read More

• Request Copy (specify title and report number, if any)
• https://doi.org/10.1002/ets2.12158