skip to main content skip to footer

Assessing Complex Problem-Solving Performances CBT ECD

Bennett, Randy Elliot; Jenkins, Frank; Persky, Hilary R.; Weiss, Andrew R.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Performance Assessment, Computer-Based Testing (CBT), Evidence-Centered Design (ECD), Simulations


Computer-based simulations can give a more nuanced understanding of what students know and can do than traditional testing methods. These extended, integrated tasks, however, introduce particular problems, including producing an overwhelming amount of data, multidimensionality, and local dependence. In this paper, we describe an approach to understanding the data from complex performances based on Evidence-centered Design, a methodology for devising assessments and for using the evidence observed in complex student performances to make inferences about proficiency. We use as an illustration the NAEP Problem-Solving in Technology-Rich Environments Study, which is being conducted to exemplify how nontraditional skills might be assessed in a sample-based national survey. The paper focuses on the inferential uses of ECD, especially how features are extracted from student performance, how these extractions are evaluated, and how the evaluations are accumulated to make evaluative judgments.

Read More