skip to main content skip to footer

Tools of Robustness for Item Response Theory IRT

Author(s):
Jones, Douglas H.
Publication Year:
1982
Report Number:
RR-82-41
Source:
ETS Research Report
Document Type:
Report
Page Count:
16
Subject/Key Words:
Computer Assisted Testing, Item Response Theory (IRT), Maximum Likelihood Statistics, Statistical Analysis, Test Reliability, Test Theory

Abstract

The technique for quantifying the impact of an outlying item response on the estimation of ability is developed. It is shown that for the typical estimation techniques, two-parameter, three-parameter, and Bayes, the outlying response has the most influence on the final estimator than any of the other responses. It is further shown by example that this impact of the outlier can seriously distort the estimate of ability. Formulas for predicting the type of behavior of an estimator in the presence of outliers are derived and an example of their use is given. These considerations are very important for computerized adaptive testing since in typical applications the computer presents items to the examinee based on the current estimate of the examinee's ability. An aberrant response could seriously affect the reliability of the computerized test. Modifications to the maximum likelihood estimator of ability are presented and shown to mollify the impact of aberrant responses. These modifications are compared to others presented by Bock and Mislevy. It is further shown how one may create new estimators tailored to reduce the impact of outliers and to possess other characteristics desirable to the practitioners. (16pp.)

Read More