skip to main content skip to footer

Formative Studies of Praxis III: Classroom Performance Assessments- Issues Identified in Qualitative Analysis of Record-of-Evidence Forms (Formative Study B6)

Camp, Roberta; Mandinach, Ellen B.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Classroom Observation Techniques, Performance Assessment, Praxis Series, Program Evaluation, Teacher Evaluation


The developers planned to use the results of the research to guide them in making informed changes in the criteria descriptions, the scoring rules, the assessment instruments, the procedural guidelines, and the assessor training program. This study, B6, reports on the issues identified in a qualitative analysis of the Record-of-Evidence forms completed during the pilot tests. Two researchers at ETS examined the Record-of-Evidence forms to determine whether assessors differed in the evidence they cited and whether records of evidence differed in their effectiveness as assessment documents. The researchers designed a matrix and coding scheme that they used in their independent examinations of the records of evidence. The completed matrices gave them an overview of differences in the Record-of- Evidence forms. They then compared and discussed their independent observations. The qualitative analysis based on the summary data and these observations suggested that four major issues be considered in assessors' preparation and in their use of the Record- of-Evidence forms: (a) the relationship of comments to rating; (b) the balance between interpretation and citing specific behaviors; (c) the handling of negative evidence, insufficient evidence and lack of evidence; and (d) the advantages of subject-matter knowledge and classroom experience. The study pinpointed specific questions for each issue and made recommendations for the assessor training program and assessors' practice.

Read More