skip to main content skip to footer

Providing Threshold Score Recommendations for the Second Phase of Assessments for the HEIghten Outcomes Assessment Suite: A Standard-Setting Study CCE ICD

Author(s):
Swiggett, Wanda D.
Publication Year:
2019
Report Number:
RM-19-06
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
45
Subject/Key Words:
HEIghten, Standard Setting, Angoff Method, Outcomes Assessment, Civic Competency and Engagement (CCE), Intercultural Competency and Diversity (ICD), Threshold Scores, Performance Level Descriptors, Borderline Students, Student Learning Outcomes, Modular Tests, Student Progress, Impact Data, College Students, Higher Education

Abstract

The HEIghten outcomes assessment suite is made up of 5 assessments designed to measure general student learning outcomes. These modular assessments are designed to provide information to students and institutions, such as student progress on each learning outcome. The standard-setting study for the 3 assessments that make up the first phase of the HEIghten suite (critical thinking, quantitative literacy, and written communication) was conducted in 2016. This report describes the standard-setting study conducted for the 2 remaining HEIghten assessments that make up the second phase of development (civic competency and engagement [CCE] and intercultural competency and diversity [ICD]). The standard-setting panel consisted of 12 college-level educators who teach undergraduate students the transferrable skills measured by these assessments. For 2 forms of each assessment, the panel recommended 2 threshold scores per form, which mark the beginning of the second and third performance categories. Using modified Angoff methods, the panel made 2 rounds of judgments, with feedback and discussion between the rounds. Evaluations were administered after training and at the conclusion of the study. The panelists indicated that their standard-setting training was clear, that the process was easy to follow, that they understood the purpose of the study, and that they supported all of the final recommended scores.

Read More