skip to main content skip to footer

Using the Selection Variable for Matching or Equating SAT

Author(s):
Dorans, Neil J.; Wright, Nancy K.
Publication Year:
1993
Report Number:
RR-93-04
Source:
ETS Research Report
Document Type:
Report
Page Count:
30
Subject/Key Words:
Anchor Tests, Equated Scores, Sampling, Scholastic Aptitude Test (SAT), Test Taking Behavior

Abstract

(30pp.) This paper addresses whether we can improve equating results if we know the variable that accounts for all systematic differences between equating populations and use it as either an anchor in an anchor test design or as a variable on which to match equating samples. The sample invariant properties of four anchor test equating methods (Tucker and Levine equally reliable linear models, chained equipercentile and frequency estimation equipercentile models) under three sampling conditions, "representative," "matched on equating test," and "matched on selection variable" are examined. The "selection variable" is defined as the variable or set of variables along which subpopulations differ. In addition to being used for matching of subpopulations, the selection variable was used as an anchor for the four equating methods, and compared to equatings in which the equating test served as the anchor. All equatings were performed with either real Scholastic Aptitude Test (SAT) populations or with data drawn from simulated pseudo-populations which differed from their original real SAT populations on the basis of the selection variable. The tests used were the verbal and math portions of two forms of the SAT. The criteria for accuracy were equivalent-groups equipercentile equatings based on old and new form subpopulations of over 115,000 test takers. Results showed that matching on the selection variable improved accuracy over matching on the equating test for all methods. Compared with the representative sample equatings, Tucker and frequency estimation results improved with matching on the selection variable; chained equipercentile and Levine results were similar under these two sampling conditions. Results with the selection variable as an anchor were good for both the Tucker and frequency estimation methods; chained equipercentile and Levine results were quite unacceptable as anticipated since use of the selection variable--math scores for the verbal equatings and verbal scores for the math equatings--violated assumptions of these models. The positive results obtained for use of the selection variable as a matching variable or anchor test (for some methods) suggest that future research into the reasons test takers select certain test administrations may lead to improved test score equating practices.

Read More