skip to main content skip to footer

Simulate to Understand Models, Not Nature

Author(s):
Dorans, Neil J.
Publication Year:
2014
Report Number:
RR-14-16
Source:
ETS Research Report
Document Type:
Report
Page Count:
9
Subject/Key Words:
Simulations, Empirical Investigations, Deductive Demonstrations, Attribute Substitution, Score Equating

Abstract

Simulations are widely used. Simulations produce numbers that are deductive demonstrations of what a model says will happen. They produce numerical results that are consistent with the premises of the model used to generate the numbers. These simulated numerical results are not empirical data that address aspects of the world that lies outside the model. In contrast, empirical data are central to the scientific method. When a simulation is substituted for the assessment of hypotheses with real data, a false sense of understanding can ensue and with it a biased perspective on the world. To illustrate the limitations of simulation and their proper role, examples are drawn from simulation studies about score equating.

Read More