skip to main content skip to footer

The Utilization of Item Sampling in Evaluation Studies and Educational Research NICHD

Cahen, Leonard S.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
National Institute for Child Health and Human Development (NICHD), Educational Research, Evaluation Methods, Item Sampling, Mathematical Models


The practicality and problems of item, or matrix, sampling as a research and evaluation technique is considered. Previous notation and applications of the technique, dating back to 1932, are mentioned, and distinction is made between traditional (or person) sampling and matrix (or item) sampling and between ex-post facto sampling and a priori sampling. Logistical and statistical problems that must be dealt with in using matrix sampling are discussed. It is concluded that item sampling is efficient for two reasons: 1) statistical estimations seem to be fairly efficient when parameters of pupil sampling are compared with the results of item sampling, and 2) many different tests or measures can be administered via item sampling in the same time it takes to administer a long test to all pupils via pupil sampling. It is also concluded that much work must be done to develop statistical models to estimate generalizability coefficients and handle the problem of estimating the second moment (the variance). Additional work also needs to be done in the area of identifying interactions between the item-sampling technique, pupil abilities and other relevant variables. The item-sampling technique, it is concluded, is specially relevant on 1) evaluation studies where the major interest is in the estimation of group, rather than individual, performance, and 2) experimental studies with many dependent measures where time prevents administration of all items to all subjects.

Read More