skip to main content skip to footer

The Relationship of Constrained Free-Response to Multiple-Choice and Open-Ended Items

Bennett, Randy Elliot; Rock, Donald A.; Braun, Henry I.; Frye, Douglas; Spohrer, James C.; Soloway, Elliot
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Computer Science Tests, Free Response Tests, MicroPROUST, Test Construction, Test Items


This study examined the relationship of a machine-scorable, constrained free-response computer science item that required the student to debug a faulty program to two other types of items: (1) multiple-choice and (2) free response requiring production of a computer program. Confirmatory factor analysis was used to test the fit of a three-factor model to these data and to compare the fit of this model to three alternatives. These models were fit using two random-half samples, one given a faulty program containing one bug and the other a program with three bugs. A single-factor model best fit the data for the sample taking the one-bug constrained free response, and a two-factor model fit the data for the second sample. In addition, the factor intercorrelations showed this item type to be significantly related to both the free-response items and the multiple-choice measures. (42pp.)

Read More