skip to main content skip to footer

An Evaluation of Item Fit Based on Generalized Residual Item Response Functions

Author(s):
Liao, Xiangyi; van Rijn, Peter; Sinharay, Sandip
Publication Year:
2025
Report Number:
RR-25-13
Source:
ETS Research Report
Document Type:
Report
Page Count:
19
Subject/Key Words:
Item Response Theory (IRT), Statistical Analysis, Wald Test, Generalized Residual, Item Fit Statistics, Marginal Maximum Likelihood Estimation, Simulation Studies, Type I Errors, 2-Parameter Logistic (2PL) Model, 3-Parameter Logistic Model (3PL)

Abstract

Evaluation of item fit for item response theory (IRT) models often involves a comparison of the observed and expected item response functions (IRFs). Several statistics have been suggested for evaluating item fit based on the discrepancy between IRFs, but the asymptotic distributions of the statistics under the null hypothesis are often not well established. Haberman et al. developed a method for evaluating the fit of IRFs based on generalized residuals. These residuals are functions of the latent proficiency variable in the IRT model and follow the standard normal distribution asymptotically. We develop a method to summarize these generalized residuals into a single summary statistic for each item and evaluate its asymptotic distribution. Kondratek suggested a similar Wald-type statistic, but without accounting for the uncertainty in the estimation of the item parameters. Our method combines the work of Haberman and Kondratek, resulting in a single fit statistic per item while accounting for estimation error. A series of simulations was carried out to investigate the performance of our statistic and compare it to several popular item fit statistics. Our method resulted in similar Type I errors as Kondratek’s statistic, with slightly better results in the case of small samples. Furthermore, the recovery was consistent across different levels of item difficulty, and power of the new item fit statistic was relatively low, except for problematic individual items, but this result was found with two competing statistics as well.

Read More