Detecting Construct-Irrelevant Variance in an Open-Ended, Computerized Mathematics Task

Gallagher, Ann; Bennett, Randy Elliot; Cahalan, Cara
Publication Year:
Report Number:
Document Type:
Subject/Key Words:
Computer-based testing item sets mathematics speededness


The purpose of this study was to evaluate whether variance due to computer-based presentation was associated with performance on a new constructed-response type--Mathematical Expression--that requires examinees to build mathematical expressions using a mouse and an on-screen tool palette. Participants took parallel computer-based and paper-based tests consisting of Mathematical Expression items, plus a test of their skill in entering and editing data using the computer interface. Comparisons of mean performance, reliability, speededness, and relations with external indicators were conducted across the paper-based and computer-based tests; also, computer-based math score was regressed on edit/entry score after controlling for paper-and- pencil math score and background information. Although no statistical evidence of construct-irrelevant variance was detected, some examinees reported mechanical difficulties in responding and indicated a preference for the paper-and-pencil test.

Read More


Find a Publication

Advanced Search

Closing the Achievement Gap

Closing the Achievement GapLearn more about ETS's commitment to closing the achievement gap through rigorous research, thought-provoking forums and more.