Toward Assessment of Design Skill in Engineering

Katz, Irvin R.; James, Carolyn M.
Publication Year:
Report Number:
Document Type:
Subject/Key Words:
Comparative analysis constructed response tests engineering tests Graduate Record Examinations test design


Design is an important part of engineering education, but cannot be assessed easily through current multiple-choice tests. In this report, we explore the possibility of including design problems in assessments of engineering students. We report a study comparing constructed-response design problems and constructed-response versions of typical engineering assessments, labeled "analysis" problems. Through analyses of verbal protocols, we argue that design problems can elicit qualitatively different behaviors compared with analysis problems, and therefore may assess different underlying constructs. While design problems may be a worthwhile addition to assessments of engineering students, such problems present several challenges, including issues of automatic scoring, the amount of information about candidate skill contained in responses, and the potential for construct-irrelevant variance. These challenges are surmountable, but require careful construction of design assessments to avoid potential pitfalls. This work represents one part of a research program aimed at informing the redesign of the GRE Engineering Subject Test.

Read More


Find a Publication

Advanced Search

Closing the Achievement Gap

Closing the Achievement GapLearn more about ETS's commitment to closing the achievement gap through rigorous research, thought-provoking forums and more.