skip to main content skip to footer

Using Bayesian Decision Theory to Design a Computerized Mastery Test IRT

Lewis, Charles; Sheehan, Kathleen M.
Publication Year:
Report Number:
ETS Research Report
Document Type:
Page Count:
Subject/Key Words:
Bayesian Statistics, Computerized Mastery Test, Decision Theory, Item Response Theory (IRT), Mastery Tests, Sequential Testing, Test Construction, Test Length, Variable Length Tests


Mastery testing is used in educational and certification contexts to decide, on the basis of test performance, whether or not an individual has attained a specified level of knowledge, or mastery, of a given subject. A theoretical framework for mastery testing, based on Item Response Theory and Bayesian decision theory, is described in this paper. In this framework, the idea of sequential testing is developed, with the goal of providing shorter tests for individuals who have clearly mastered (or clearly not mastered) a given subject, and providing longer tests for those individuals for whom the mastery decision is not as clear-cut. In a simulated application of the approach to a professional certification examination, it is shown that average test lengths can be reduced by half without sacrificing classification accuracy. (48pp.)

Read More