skip to main content skip to footer

Error Variance in Common Population Linking Bridge Studies

Author(s):
Jewsbury, Paul A.
Publication Year:
2019
Report Number:
RR-19-42
Source:
ETS Research Report
Document Type:
Report
Page Count:
31
Subject/Key Words:
Bridge Studies, Linking, Variance, Standard Error, Error of Measurement, Estimators, Score Comparability

Abstract

When an assessment undergoes changes to the administration or instrument, bridge studies are typically used to try to ensure comparability of scores before and after the change. Among the most common and powerful is the common population linking design, with the use of a linear transformation to link scores to the metric of the original assessment. In the common population linking design, randomly equivalent samples receive the new and previous administration or instrument. However, conventional procedures to estimate error variances are not appropriate for scores linked in a bridge study, because the procedures neglect variance due to linking. A convenient approach is to estimate a variance component associated with the linking to add to the conventionally estimated error variance. Equations for the variance components in this approach are derived, and the approximations inherently made in this approach are shown and discussed. Exact error variances of linked scores, accounting for both conventional sources of variance (e.g., sampling) and linking variance together, are derived and discussed. The consequences of how linking changes how certain errors are related is considered mathematically. Specifically, the impacts of linking on the error variance for the comparison of two linked estimates (e.g., comparing the mean score of boys to the mean score of girls, after linking), for the comparison of scores across the two samples (e.g., comparing the mean score of boys in the new administration or instrument to the mean score of boys in the old administration or instrument), and for aggregating scores across the two samples (e.g., the mean score of boys across both administrations or instruments) are derived and discussed. Finally, general methods to account for error variance in bridge studies by simultaneously accounting for both conventional and linking sources of error are recommended.

Read More