Detecting Construct-Irrelevant Variance in an Open-Ended, Computerized Mathematics Task

Author(s):
Gallagher, Ann; Bennett, Randy Elliot; Cahalan, Cara
Publication Year:
2000
Report Number:
RR-00-18
GREB-95-13P
Source:
Document Type:
Subject/Key Words:
Computer-based testing item sets mathematics speededness

Abstract

The purpose of this study was to evaluate whether variance due to computer-based presentation was associated with performance on a new constructed-response type--Mathematical Expression--that requires examinees to build mathematical expressions using a mouse and an on-screen tool palette. Participants took parallel computer-based and paper-based tests consisting of Mathematical Expression items, plus a test of their skill in entering and editing data using the computer interface. Comparisons of mean performance, reliability, speededness, and relations with external indicators were conducted across the paper-based and computer-based tests; also, computer-based math score was regressed on edit/entry score after controlling for paper-and- pencil math score and background information. Although no statistical evidence of construct-irrelevant variance was detected, some examinees reported mechanical difficulties in responding and indicated a preference for the paper-and-pencil test.

Read More