skip to main content skip to footer

Subject Matter Experts' Assessment of Item Statistics TSWE

Bejar, Isaac I.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
English Tests, Item Analysis, Statistical Analysis, Test Construction, Test of Standard Written English (TSWE)


The study was conducted to determine the degree to which subject matter experts could predict the difficulty and discrimination of items from the Test of Standard Written English. Two studies were conducted. The first study was meant to serve as a training exercise while the second study was meant to simulate the conditions under which ratings would be obtained in practice. It was concluded that despite the extended training period the raters did not approach a high level of accuracy, nor were they able to pinpoint the factors that contribute to difficulty and discrimination. Further research should attempt to uncover those factors by examining the items from a linguistic and psycholinguistic perspective. It was argued that by coupling linguistic features of the items with subject matter ratings it may be possible to attain more accurate predictions of difficulty and discrimination. (47pp.)

Read More