This report describes selected results from the 2001 Math Online (MOL) study, 1 of 3 field investigations sponsored by the National Center for Education Statistics to explore the use of new technology for the National Assessment of Educational Progress (NAEP). Of particular interest in the MOL study was the comparability of scores from paper- and computer-based tests. A nationally representative sample of 8th-grade students was administered a computer-based mathematics test and a test of computer facility, among other measures. In addition, a randomly parallel group of students was administered a paper-based test containing the same math items as the computer-based test. Results showed that the computer-based mathematics test was significantly harder statistically than the paper-based test. In addition, computer facility predicted online mathematics test performance after controlling for performance on a paper-based mathematics test, suggesting that degree of familiarity with computers may matter when taking a computer-based mathematics test in NAEP.