skip to main content skip to footer

A Tale of Two Models: Sources of Confusion in Achievement Testing CAT IRT

Author(s):
Reckase, Mark D.
Publication Year:
2017
Report Number:
RR-17-44
Source:
ETS Research Report
Document Type:
Report
Page Count:
17
Subject/Key Words:
Achievement Testing, Test Design, Test Development, Educational Psychology, Model-Based Measurement, Measurement Techniques, Psychology, Test Interpretation, Computerized Adaptive Testing (CAT), Item Response Theory (IRT), Angoff Memorial Lecture Series

Abstract

A common interpretation of achievement test results is that they provide measures of achievement that are much like other measures we commonly use for height, weight, or the cost of goods. In a limited sense, such interpretations are correct, but some nuances of these interpretations have important implications for the use of achievement test results. This paper will contrast two different theoretical underpinnings for the interpretation of educational assessments results as measurements. One of these theoretical views comes from initial attempts in psychology to measure the amount of a trait that was exhibited by a person. The other theoretical view comes from early work in education to measure how much of a desired curriculum was acquired by students. At times, these views conflict with each other and lead test developers and policy makers to ask for the impossible. After summarizing the two theoretical positions, the areas of conflict will be discussed. Finally, some recommendations will be given for what can be done to clarify the issues and minimize the problems that result from using conflicting theoretical frameworks. The 16th William H. Angoff Memorial Lecture was presented by Dr. Mark D. Reckase, University Distinguished Professor Emeritus, Michigan State University, at the National Press Club, Washington, D.C., on March 29, 2017. This paper is based on his lecture.

Read More