skip to main content skip to footer

A Tree-Based Analysis of Items From an Assessment of Basic Mathematics Skills

Author(s):
Mislevy, Robert J.; Sheehan, Kathleen M.
Publication Year:
1994
Report Number:
RR-94-14
Source:
ETS Research Report
Document Type:
Report
Page Count:
31
Subject/Key Words:
Item Analysis, Item Response Models, Mathematics, Praxis Series, Regression (Statistics), Tree Structures (Graphs)

Abstract

(31pp.) The operating characteristics of 114 Mathematics pretest items from the Praxis I: Computer Based Test were analyzed in terms of item attributes and test developers' judgments of item difficulty. Item operating characteristics were defined as the difficulty, discrimination, and asymptote parameters of a three parameter logistic IRT model. Three types of item attributes were considered: surface features (for example, whether or not the item stem contained an equation); aspects of the solution process (for example, whether or not the solution required application of a standard formula); and response type (free-response or multiple-choice). Because the attribute set included large numbers of categorical variables, an approach based on binary regression trees (Breiman, Friedman, Olshen, & Stone, 1984) was implemented. The results were quite impressive for asymptote parameters (85% of variance explained), somewhat less so for difficulty parameters (36% of variance explained), and fairly unimpressive for discrimination parameters (only 12% of variance explained). In addition, the tree-based approach was found to be particularly useful for identifying important interaction effects and for developing graphical summaries of the modeling results.

Read More