skip to main content skip to footer

Assessing Some of the Properties of Longer Blocks in the 1992 NAEP Reading Assessment NAEP

Author(s):
Donoghue, John R.; Mazzeo, John
Publication Year:
1995
Report Number:
RR-95-28
Source:
ETS Research Report
Document Type:
Report
Page Count:
27
Subject/Key Words:
Factor Analysis, National Assessment of Educational Progress (NAEP), Reading Tests, Test Equivalence, Test Length

Abstract

At Grades 8 and 12, the 1992 NAEP reading assessment contained a small number of 50-minute blocks, in addition to the usual 25-minute blocks. In order to determine whether to incorporate the 50-minute blocks into the operational scaling, this study sought to determine whether longer blocks measure a different construct from that assessed by the 25-minute blocks. Structural equation modeling tested the hypothesis that the structural parameters relating reading ability to demographic variables do not differ across block type. A multiple group analysis, where type of block (25- minute or 50-minute) defined the two groups, was used. The null hypothesis for this comparison was that the two types of blocks measure the same trait but may differ in observed mean and variance. Results of the main analysis did not reject the hypothesis of invariant structural parameters, and so the 50-minute blocks were incorporated in the 1992 NAEP scales. Sensitivity analyses indicated that this conclusion was moderately robust to assumptions made about missing data for items which were not reached, although Grade 8 results were more robust than those for Grade 12. Analyses using other measures of fit yielded the same pattern of results, although the magnitude of the x2 statistics was affected by the fit measure chosen, particularly the asymptotically distribution free method. Attempts to replicate the main analysis in independent samples yielded similar x2 values at Grade 8, but Grade 12 yielded x2 values which were substantially higher for some of the samples. The Grade 12 results raise questions as to the generalizability of the main analysis. Alternatives to the reliance on x2 measures are discussed for future research. (27pp.)

Read More