Item Performance Across Native Language Groups on the Test of English as a Foreign Language™
- Alderman, Donald L.; Holland, Paul W.
- Publication Year:
- Report Number:
- RR-81-16, TOEFL-RR-09
- Document Type:
- Subject/Key Words:
- English tests item analysis language proficiency native speakers test validity
The Test of English as a Foreign Language™ (TOEFL®) was examined for instances in which the item performance of examinees with comparable scores differed according to their native languages. A chi-square procedure, sensitive to deviations of less than 10% from the expected frequencies of correct item responses across several language groups, revealed significant differences on seven-eights of the TOEFL items. Reviewers familiar with particular languages could attribute the relative advantage or disadvantage of those language groups on a specific item to linguistic similarities or dissimilarities with the English language. Reviewers could not, however, identify which items would exhibit differential performance across groups based upon inspection of a test form and answer key alone. These findings suggest that examinees' performance on given items in a test of proficiency in a second language will vary according to linguistic contrasts with their native language and that statistical procedures will be necessary for identifying items with exaggerated or unexplained differences across language groups. Four appendices provide a large number of statistical tables and graphs.