Detecting Construct-Irrelevant Variance in an Open-Ended, Computerized Mathematics Task
- Author(s):
- Gallagher, Ann; Bennett, Randy Elliot; Cahalan, Cara
- Publication Year:
- 2000
- Report Number:
- RR-00-18
- GREB-95-13P
- Source:
- Document Type:
- Subject/Key Words:
- Computer-based testing item sets mathematics speededness
Abstract
The purpose of this study was to evaluate whether variance due to computer-based presentation was associated with performance on a new constructed-response type--Mathematical Expression--that requires examinees to build mathematical expressions using a mouse and an on-screen tool palette. Participants took parallel computer-based and paper-based tests consisting of Mathematical Expression items, plus a test of their skill in entering and editing data using the computer interface. Comparisons of mean performance, reliability, speededness, and relations with external indicators were conducted across the paper-based and computer-based tests; also, computer-based math score was regressed on edit/entry score after controlling for paper-and- pencil math score and background information. Although no statistical evidence of construct-irrelevant variance was detected, some examinees reported mechanical difficulties in responding and indicated a preference for the paper-and-pencil test.