skip to main content skip to footer

Toward Assessment of Design Skill in Engineering GREB GRE

Author(s):
Katz, Irvin R.; James, Carolyn M.
Publication Year:
1997
Report Number:
RR-97-16
Source:
ETS Research Report
Document Type:
Report
Page Count:
38
Subject/Key Words:
Graduate Record Examinations Board, Comparative Analysis, Constructed-Response Tests, Engineering Tests, Graduate Record Examinations (GRE), Test Design

Abstract

Design is an important part of engineering education, but cannot be assessed easily through current multiple-choice tests. In this report, we explore the possibility of including design problems in assessments of engineering students. We report a study comparing constructed-response design problems and constructed-response versions of typical engineering assessments, labeled "analysis" problems. Through analyses of verbal protocols, we argue that design problems can elicit qualitatively different behaviors compared with analysis problems, and therefore may assess different underlying constructs. While design problems may be a worthwhile addition to assessments of engineering students, such problems present several challenges, including issues of automatic scoring, the amount of information about candidate skill contained in responses, and the potential for construct-irrelevant variance. These challenges are surmountable, but require careful construction of design assessments to avoid potential pitfalls. This work represents one part of a research program aimed at informing the redesign of the GRE Engineering Subject Test.

Read More