skip to main content skip to footer

Agreement of Teachers on Evaluating Assessments of Learning Progressions in English Language Arts and Mathematics ELA

van Rijn, Peter; Graf, Edith Aurora; Arieli-Attali, Meirav; Song, Yi
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Test Evaluation, Learning Progressions, English Language Arts (ELA), Mathematics, Summative Assessment, Teachers, Text Difficulty, Tasks, Human Raters, Agreement


In this study, we explored the extent to which teachers agree on the ordering and separation of levels of two different learning progressions (LPs) in English language arts (ELA) and mathematics. In a panel meeting akin to a standard‐setting procedure, we asked teachers to link the items and responses of summative educational assessments to LP levels. We evaluated four types of agreement among teachers, with test developers, and with trained raters. Although the results were quite mixed, agreement on the LP levels of mathematics items was generally better than that for ELA tasks. The agreement on the LP levels of ELA sample responses was generally better than that for mathematics. Implications of the results are discussed.

Read More