skip to main content skip to footer

Psychometric Considerations and a General Scoring Strategy for Assessments of Collaborative Problem Solving CPS ECSAP CBA

Author(s):
Hao, Jiangang; Liu, Lei; Kyllonen, Patrick C.; Flor, Michael; von Davier, Alina A.
Publication Year:
2019
Report Number:
RR-19-41
Source:
ETS Research Report
Document Type:
Report
Page Count:
17
Subject/Key Words:
Collaborative Problem Solving (CPS), Psychometrics, Test Design, Test Scoring, ETS Collaborative Science Assessment Prototype (ECSAP), Simulation-Based Assessment, Computer Based Assessment (CBA), 21st Century Skills

Abstract

Collaborative problem solving (CPS) is an important 21st-century skill that is crucial for both career and academic success. However, developing a large-scale and standardized assessment of CPS that can be administered on a regular basis is very challenging. In this report, we introduce a set of psychometric considerations and a general scoring strategy around assessing CPS, summarized based on the results of the extensive empirical studies we conducted at Educational Testing Service (ETS) over the past 6 years. Using the ETS Collaborative Science Assessment Prototype as an example, we show how these psychometric considerations have been incorporated into the development of the assessment prototype and how the scoring strategy has been implemented.

Read More