skip to main content skip to footer

Providing Threshold Score Recommendations for the First Three Tests of the HEIghten Outcomes Assessment Suite: A Standard-Setting Study

Author(s):
Swiggett, Wanda D.
Publication Year:
2017
Report Number:
RM-17-06
Source:
ETS Research Memorandum
Document Type:
Report
Page Count:
50
Subject/Key Words:
HEIghten, Standard Setting, Student Learning Outcomes, Angoff Method, Performance Level Descriptors, Impact Data, Critical Thinking, Problem Solving, Written Communication, Performance Level Descriptors, Test Familiarization, Quantitative Literacy

Abstract

The HEIghten outcomes assessment suite is made up of multiple assessments designed to assess general student learning outcomes for students exiting college. These assessments are modular and are designed to inform students as well as institutions of student progress on each learning outcome. A standard-setting study was conducted for the first 3 HEIghten assessments—critical thinking, quantitative literacy, and written communication. A panel of 11 college professors who teach undergraduate students participated in the study. These educators teach courses in which the general education skills assessed in the 3 HEIghten assessments can be attained. For each of the 3 assessments, the panel recommended 2 threshold scores, which mark the beginning of the proficient and advanced performance categories. Scores below the proficient threshold score classify students as developing. The panel provided recommendations for 2 forms of each assessment. The panelists made 2 rounds of item-level judgments, following the modified Angoff and extended Angoff methods, for the selected- and constructed-response items, respectively. Between each round, they discussed panelists’ judgments and impact data based on the operational administration of the assessments. Evaluations were administered after each training and at the conclusion of the study. The panelists indicated their training in standard setting was clear, that the process was easy to follow, and that they understood the purpose of the study. Additionally, the panelists indicated that they supported all the final recommended scores.

Read More