skip to main content skip to footer

Best Practices for Constructed-Response Scoring PRMSE

Author(s):
McCaffrey, Daniel F.; Casabianca, Jodi M.; Ricker-Pedley, Kathryn L.; Lawless, Rene; Wendler, Cathy
Publication Year:
2022
Report Number:
RR-22-17
Source:
ETS Research Report
Document Type:
Report
Page Count:
60
Subject/Key Words:
Constructed-Response Scoring, Human Raters, Validity, Automated Scoring Models, Proportional Reduction in Mean-Squared Error (PRMSE), Multimodal Processing

Abstract

This document describes a set of best practices for developing, implementing, and maintaining the critical process of scoring constructed-response tasks. These practices address both the use of human raters and automated scoring systems as part of the scoring process and cover the scoring of written, spoken, performance, or multimodal responses. Best Practices for Constructed-Response Scoring is designed not to act as an independent guide, but rather to be used in conjunction with other ETS publications, including the Guidelines for Constructed-Response and Other Performance Assessments, ETS Standards for Quality and Fairness, ETS Guidelines for Fair Tests and Communications, and the ETS Guidelines for Fairness.

Read More