skip to main content skip to footer

Does It Matter If I Take My Mathematics Test on Computer? A Second Empirical Study of Mode Effects in NAEP CBT NAEP MOL

Bennett, Randy Elliot; Braswell, James S.; Oranje, Andreas; Sandene, Brent A.; Kaplan, Bruce A.; Yan, Fred
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Computer-Based Testing (CBT), Performance Assessment, Comparability Studies, Mathematics Assessment, National Assessment of Educational Progress (NAEP)


This report describes selected results from the 2001 Math Online (MOL) study, 1 of 3 field investigations sponsored by the National Center for Education Statistics to explore the use of new technology for the National Assessment of Educational Progress (NAEP). Of particular interest in the MOL study was the comparability of scores from paper- and computer-based tests. A nationally representative sample of 8th-grade students was administered a computer-based mathematics test and a test of computer facility, among other measures. In addition, a randomly parallel group of students was administered a paper-based test containing the same math items as the computer-based test. Results showed that the computer-based mathematics test was significantly harder statistically than the paper-based test. In addition, computer facility predicted online mathematics test performance after controlling for performance on a paper-based mathematics test, suggesting that degree of familiarity with computers may matter when taking a computer-based mathematics test in NAEP.

Read More