skip to main content skip to footer

Can Smoothing Help When Equating With Unrepresentative Small Samples?

Author(s):
Puhan, Gautam
Publication Year:
2011
Report Number:
RR-11-09
Source:
ETS Research Report
Document Type:
Report
Page Count:
27
Subject/Key Words:
Log-Linear Presmoothing, Test Equating, Small Sample, Random Equating Error, Systematic Equating Error

Abstract

The study evaluated the effectiveness of log-linear presmoothing (Holland & Thayer, 1987) on the accuracy of small sample chained equipercentile equatings under two conditions (i.e., using small samples that differed randomly in ability from the target population versus using small samples that were distinctly different from the target population). Results showed that equating with small samples (e.g., N < 50) using either raw or smoothed score distributions can result in a substantial amount of random equating error (although smoothing reduced random equating error). Even with samples sizes of 100, the random equating error was quite large (greater than the difference that matters or DTM) for almost all score points. Moreover, when the small samples were unrepresentative of the target population, which is quite likely for small samples, the amount of equating bias (in addition to random equating error) was considerably large for both the raw and smoothed equatings. It was concluded that although presmoothing helped reduce random equating error, it is unlikely to reduce equating bias caused by using an unrepresentative sample. Other alternatives to the small sample equating problem that focus more on improving data collection than on improving existing equating methods are discussed.

Read More