skip to main content skip to footer

A New Look at Formulating Hypotheses Items

Carlson, Sybil B.; Ward, William C.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Graduate Record Examinations Board, Cognitive Ability, Formulating Hypotheses (Test), Free Response Tests, Scoring, Test Construction


This project focused on the two major issues that need to be addressed in considering FH items for operational use: (1) the costs of scoring, and (2) rather than the conventional number-right scoring, the assignment of scores along a range of values. The first issue was addressed directly by seeking ways to increase the efficiency of scoring through computerized delivery and scoring. The second issue was addressed both directly and indirectly by recommending specific procedures for the computer recognition of responses and problem delivery that will be sufficiently reliable and well-rationalized to be acceptable to reasonable evaluators. This project involved collaboration with experts who are closely involved in confronting the issues involved in the computer recognition and evaluation of openended responses. After a series of analyses to explore the design and scoring of FH-type items for computer delivery, we arrived at specific recommendations for developing a system to deliver computerized problems of the FH type. When developed, the prototype also will serve as a computerized research tool to conduct further investigations of potential variations in these types of items.

Read More