Reducing Test Form Overlap of the GRE Subject Test in Mathematics Using IRT Triple-Part Equating

Author(s):
McKinley, Robert L.; Schaeffer, Gary A.
Publication Year:
1989
Report Number:
RR-89-08
GREB-86-14P
Source:
Document Type:
Subject/Key Words:
Aptitude tests equated scores item response theory (IRT) mathematics tests test construction

Abstract

A study was conducted to evaluate the feasibility of using item response theory (IRT) equating to reduce test form overlap of the GRE Subject Test in Mathematics. Monte-Carlo methods were employed to compare double-part equating with 20-item common item blocks to triple-part equating with 10-item common item blocks. The two methods were evaluated using a circular design that allowed a form to be equated to itself through a series of other forms. The design was replicated five times. Comparisons between scores on equated forms and scores on the base form indicated that triple-part equating did at least as well as double-part equating. This suggests that it may be reasonable to use IRT equating with the GRE Mathematics test with smaller common item blocks than are used with the linear equating procedures currently employed, as long as there are more of them. This would result in a substantial reduction in the advantage given to repeat examinees, and would significantly decrease the number of items on any one test form affected by compromised security. It was concluded that triple-part equating is promising, but it was pointed out that some issues need to be resolved before the procedure can be implemented. Among these are the effect of differing ability distributions across forms to be scaled, and the practicality of constructing six common item sets from each form. It was recommended that an additional study be conducted to examine the effect of differing ability distributions on the equating process, and to investigate the feasibility of combining common item sets into a single scaling.

Read More