skip to main content skip to footer

A Simulation Study to Compare Nonequivalent Groups With Anchor Test Equating and Pseudo‐Equivalent Group Linking NEAT

Author(s):
Lu, Ru; Guo, Hongwen
Publication Year:
2018
Report Number:
RR-18-08
Source:
ETS Research Report
Document Type:
Report
Page Count:
18
Subject/Key Words:
Simulation, Anchor Test Equating, Pseudo-Equivalent Groups, Non-Equivalent-Groups Anchor Test (NEAT) Design, Population Distribution, Large-Scale Assessment, Linking, Variables, Background

Abstract

In this paper we compare the newly developed pseudo‐equivalent groups (PEG) linking method with the linking methods based on the traditional nonequivalent groups with anchor test (NEAT) design and illustrate how to use the PEG methods under imperfect equating conditions. To do this, we proposed a new method that combines the features of PEG linking and NEAT equating (referred as PEGAT) and compared it with NEAT and PEG. PEG mainly uses test takers' background variables to create PEG and then links scores on different forms whereas NEAT equating adjusts group differences in ability through the anchor test scores. The proposed method, PEGAT, uses background variables and anchor scores to adjust group ability differences. Using simulated data, these 3 linking methods were compared in 2 equating scenarios: small and large group difference in ability. The simulation design was based on real data on a test in operation. The test scores and the background variables were assumed to have a multivariate multinomial distribution. A log linear model was used to manipulate and produce simulated data. For PEG and PEGAT linking, 3 different sets of background variables were manipulated to study the impact of correlation strength of background variables to the total scores. Our results showed that NEAT linking outperformed PEG linking when the group ability difference was large, but that NEAT linking could be improved by incorporating the PEG adjustment procedure based on background variables. When the groups were similar in ability, PEG linking produced comparable results to NEAT. This finding justifies the use of PEG linking when a good anchor test is not available as well as the use of PEGAT when a good anchor is available but needs to be strengthened by background variables.

Read More