Trends in U.S. educational policy drive a need to answer fundamental questions about effective classroom instruction, such as:
- How can we define quality in teaching and how can we measure it appropriately?
- How can teachers develop effective practices and how well do they apply these practices in the classroom?
ETS's research on teaching quality aims to:
- improve the conceptual frameworks and proficiency models that undergird assessments of teaching
- design, develop and study assessments of teaching quality
- advance scientific knowledge about the nature of teaching quality and how to measure it, concentrating on questions of validity and reliability
- better understand the measurement models behind assessments of teaching quality
Featured Publications
These are some recent or significant publications on the subject of understanding teaching quality:
2016
- Assessing Mathematical Knowledge for Teaching: The Role of Teaching Context
G. Phelps & H. Howell
Mathematics Enthusiast, Vol. 13, No. 1
In this article, the authors describe an analysis of 10 assessment tasks that target mathematical knowledge for teaching. Learn more about this publication -
Investigating the Relevance and Importance of High-Leverage Practices for Beginning Elementary School Teachers
M. P. Martin-Raugh, C. M. Reese, H. Howell, R. J. Tannenbaum, J. H. Steinberg, & J. Xu (2016)
ETS Research Memorandum No. RM-16-11The purpose of this study is to explore the validity evidence supporting the high-leverage practices (HLPs) of the ETS® National Observational Teaching Exam (NOTE) assessment series, a kindergarten through 6th grade teacher licensure assessment. Learn more about this publication >
-
Investigating the Relevance and Importance of Mathematical Content Knowledge Areas for Beginning Elementary School Teachers
M. P. Martin-Raugh, C. M. Reese, H. Howell, R. J. Tannenbaum, J. H. Steinberg, & J. Xu (2016)
ETS Research Memorandum No. RM-16-10The purpose of this report is to explore the content-related validity evidence supporting the mathematics components of the ETS® National Observational Teaching Exam (NOTE) assessment series, a kindergarten through 6th grade teacher licensure assessment. Learn more about this publication >
-
Leading a Classroom Discussion: Definition, Supporting Evidence, and Measurement of the ETS® National Observational Teaching Examination (NOTE) Assessment Series
M. S. Witherspoon, G. Sykes, & C. A. Bell (2016)
ETS Research Memorandum No. RM-16-09This paper provides a description and rationale for a performance assessment of a teaching practice—leading a classroom discussion (LCD)—included in the ETS® National Observational Teaching Examination (NOTE) assessment series. Learn more about this publication >
-
Investigating the Relevance and Importance of English Language Arts Content Knowledge Areas for Beginning Elementary School Teachers
M. P. Martin-Raugh, C. M. Reese, G. C. Phelps, R. J. Tannenbaum, J. H. Steinberg, & J. Xu (2016)
ETS Research Memorandum No. RM-16-08The purpose of this report is to explore the content-related validity evidence supporting the English language arts (ELA) components of the ETS® National Observational Teaching Exam (NOTE) assessment series, a kindergarten through 6th-grade teacher licensure assessment. Learn more about this publication >
-
Modeling and Explaining Content: Definition, Research Support, and Measurement of the ETS® National Observational Teaching Examination (NOTE) Assessment Series
L. M. Stickler & G. Sykes (2016)
ETS Research Memorandum No. RM-16-07This report reviews the scholarly and research evidence supporting the construct labeled modeling and explaining content (MEC), which is measured via a performance assessment in the ETS® National Observational Teaching Examination (NOTE) assessment series. Learn more about this publication >
-
Eliciting Student Thinking: Definition, Research Support, and Measurement of the ETS® National Observational Teaching Examination (NOTE) Assessment Series
Y. Qi & G. Sykes (2016)
ETS Research Memorandum No. RM-16-06This report describes and provides research and scholarly support for a core practice of teaching—eliciting student thinking (EST)—that is the target for a performance assessment contributing 1 component of the ETS® National Observational Teaching Examination (NOTE) assessment series. Learn more about this publication >
2015
- Inferring constructs of Effective Teaching from Classroom Observations: An Application of Bayesian Exploratory Factor Analysis Without Restrictions
J. R. Lockwood, T. D. Savitsky, & D. F. McCaffrey
The Annals of Applied Statistics, Vol 9, No. 3, pp. 1484-1509
In this article, the authors describe a study in which they used data from middle school mathematics and English-language arts teachers to inform research and practice on teacher performance measurement. Learn more about this publication - Understanding Consequential Assessment Systems of Teaching: Year 2 Final Report to Los Angeles Unified School District
C. Bell, N. Jones, J. Lewis, Y. Qi, D. Kirui, L. Stickler, & S. Liu (2015)
ETS Research Report Memorandum No. RM-15-12
This report examines the second year of data collection in the Understanding Consequential Assessment Systems of Teaching (UCAST) study. The report describes administrators’ learning during the first year that the Los Angeles Unified School District’s observations were part of a consequential teacher evaluation system. Learn more about this publication - Classroom Assessment Standards for PreK-12 Teachers: Joint Committee on Standards for Educational Evaluation
D. A. Klinger, P. J. McDivitt, B. B. Howard, M. Muñoz, W. T. Rogers,
& E. C. Wylie (2015)
Kindle book edition
The Classroom Assessment Standards for PreK-12 Teachers was released by the Joint Committee on Standards for Educational Evaluation, which is accredited by the American National Standards Institute (ANSI). The standards provide classroom teachers in pre-kindergarten through high school with research-based principles and guidelines for effective assessment of student learning. Learn more about this publication - Using Representational Tools to Learn About Complex Systems: A Tale of Two Classrooms
C. E. Hmelo-Silver, L. Liu, S. Gray, & R. Jordan (2015)
Journal of Research in Science Teaching, Vol. 52, No. 1, pp. 6–35
The authors present a study of physical and computer-based tools conducted in two different classroom settings. Learn more about this publication - How Teachers Teach: Mapping the Terrain of Practice
G. Sykes & S. Wilson
ETS White Paper
This paper represents an interpretive synthesis by the authors of main and contemporary currents in the research on teaching and learning. Learn more about this publication - The Impact of Measurement Error on the Accuracy of Individual and Aggregate SGP
D. F. McCaffrey, K. E. Castellano, & J. R. Lockwood
Educational Measurement: Issues and Practice, Vol. 34, No. 1, pp. 15–21
In this paper, the authors examine different student growth percentiles (SGP) to explain why test measurement error is detrimental to both student-level and aggregate-level SGP estimation in order to reduce potential bias. Learn more about this publication
2014
- Trends in Classroom Observation Scores
J. Casabianca, J. R. Lockwood, & D. F. McCaffrey
Educational and Psychological Measurement, Vol. 75, No. 2, pp. 311–337
The authors discuss trends in teacher observations and scoring. Both the quality of the teaching and of the raters tend to shift, which can have implications for the ability to draw valid inferences about teaching and for the design of such observations. The authors observed 458 middle school teachers over a two-year period using the Classroom Assessment Scoring System–Secondary (CLASS-S) protocol. Learn more about this publication - The Instructional Challenge in Improving Teaching Quality: Lessons From a Classroom Observation Protocol
D. Gitomer, C. Bell, Y. Qi, D. McCaffrey, B. Hamre, & R. Pianta
Teachers College Record, Vol. 116, No. 6, pp. 22–52
This article describes a study of 82 middle and high school Algebra I teachers. The study evaluates evidence of various aspects of instructional practice, score variations, the ability of observers to judge instructional practice, as well as teachers’ ability to self-evaluate. The authors discuss a major challenge to using teacher evaluation as a method of improving teaching and learning. Learn more about this publication - Correcting for Test Score Measurement Error in ANCOVA Models for Estimating Treatment Effects
J. R. Lockwood & D. McCaffrey
Journal of Educational and Behavioral Statistics, Vol. 39, No. 1, pp. 22–52
In this article, the authors discuss methods for estimating and correcting for measurement errors for treatment effects in observational studies. The authors describe extensions they developed for several methods of correcting for measurement error. Learn more about this publication - Cognitive Frameworks for Assessment, Teaching, and Learning: A Validity Perspective
M. T. Kane, & I. I. Bejar
Psicología Educativa, Vol. 20, No. 2, pp. 117-123
The authors provide an introduction to an issue of Psicología Educativa, which addresses questions about how it is possible to add value to educational assessments. They write that while grounding assessment design in cognitive theories and model-based methodologies is highly desirable, rigorous evaluation of the resulting scores is still necessary. Learn more about this publication - Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project
T. Kane, K. Kerr, & R. Pianta (Eds.)
Publisher: Wiley
This book describes research from the Measures of Effective Teaching project, to which ETS was a major contributor. ETS staff members authored or co-authored the following chapters in this edited volume- Grade-Level Variation in Observational Measures of Teacher Effectiveness (pp. 9–49)
K. Mihaly & D. McCaffrey - Improving Observational Score Quality: Challenges in Observer Thinking (pp. 50–97)
C. Bell, Y. Qi, A. Croft, D. Leusner, D. McCaffrey, D. Gitomer, & R. Pianta - Evaluating Efforts to Minimize Rater Bias in Scoring Classroom Observations
(pp. 383–414)
Y. S. Park, J. Chen, & S. Holtzman - Scoring Design Decisions: Reliability and the Length and Focus of Classroom Observations
(pp. 415–443)
J. Joe, C. McClellan, & S. Holtzman - Evidence on the Validity of Content Knowledge for Teaching Assessments
(pp. 493–529)
D. Gitomer, G. Phelps, B. Weren, H. Howell, & A. Croft - Optimizing Resources to Maximize Student Gains
(pp. 529–582)
C. McClellan, J. Donoghue, & R. Pianta
- Grade-Level Variation in Observational Measures of Teacher Effectiveness (pp. 9–49)
- Uncovering Multivariate Structure in Classroom Observations in the Presence of Rater Errors
D. F. McCaffrey, K. Yuan, T. D. Savitsky, J. R. Lockwood, & M. O. Edelen
Educational Measurement; Issues and Practice, 2014, Online first.
The authors examine the factor structure of scores from the CLASS-S protocol, as used in observations of middle school classroom teaching. Their results have implications not only for future studies to develop validity evidence and test theories of teaching, but also for practitioners who rely on the results of such studies to support their use and interpretation of observation scores. Learn more about this publication - Developing Content Knowledge for Teaching Assessments for the Measures of Effective Teaching Study
G. Phelps, B. Weren, A. Croft, & D. Gitomer
ETS Research Report No. RR-14-33
This paper documents the development of assessments of content knowledge for teaching (CKT) as part of a study funded by the Gates Foundation under the Measures of Effective Teaching (MET) project. The goal was to find a set of measures that could measure teaching effectiveness. Learn more about this publication - Teaching Content to English Learners in the Era of the Common Core Standards
S. Turkan, & M. Schramm-Possinger
R&D Connections, No. 23
The authors discuss teaching of content to English language learners in relation to the Common Core State Standards. They argue that teachers must facilitate ELLs' understanding and use of oral and written discourse for each discipline, which they refer to as "Disciplinary linguistic knowledge." Learn more about this publication - Proposing a Knowledge Base for Teaching Academic Content to English Language Learners: Disciplinary Linguistic Knowledge
S. Turkan, L. C. De Oliveira, O. Lee, & G. Phelps
Teachers College Record, Vol. 116 No. 3
The authors discuss the need for better knowledge about how teachers can address the linguistic and culture needs of a U.S. student population that is growing increasingly diverse. Learn more about this publication - Building Trust in Observations: A Blueprint for Improving Systems to Support Great Teaching
J. Wood, J. Joe, S. Cantrell, C. Tocci, S. Holtzman, & J. Archer
MET Project Policy and Practice Brief
This document, created as part of the Measures of Effective Teaching project, is designed to help states and districts to create plans for continual improvement of an observation system. It recommends action steps to improve observation rubrics, observer training, observer assessment and monitoring. View document.
2013
- Cognitive Interviews as a Tool for Investigating the Validity of Content Knowledge for Teaching Assessment
H. Howell, G. Phelps, A. Croft, D. Kirui, & D. Gitomer
ETS Research Report No. RR-13-19
This report describes a study based on 30 cognitive interviews with elementary mathematics teachers and 30 interviews of elementary English-language arts teachers. The report is intended both to provide information on the validity measures of content knowledge for teaching (CKT) and to provide guidance to researchers interested in replicating the design. The report articulates interpretive arguments that are central to the CKT measurement theory and then uses cognitive interview data to evaluate these arguments. Learn more about this publication - Effect of Observation Mode on Measures of Secondary Mathematics Teaching
J. Casabianca, D. McCaffrey, D. Gitomer, C. Bell, B. K. Hamre, & R. C. Pianta
Educational and Psychological Measurement, Vol. 73, No. 5, pp. 757–783
This article describes a study that investigated whether teacher observations made live in the classroom and those made from video recording of the same lessons yielded similar inferences about teaching. The researchers used scores on the Classroom Assessment Scoring System–Secondary (CLASS-S) from 82 algebra classrooms to explore the effect of observation mode. Learn more about this publication - Can Teacher Evaluation Improve Teaching?
L. Goe
Principal Leadership, Vol. 13, No. 7, pp. 24–29
This article discusses how new expectations for teacher observation emphasize the importance of principals giving teachers guidance and support that will help them to improve their teaching performance. View the article - Foundations of Observation: Considerations for Developing a Classroom Observation System That Helps Districts Achieve Constant and Accurate Scores
J. Joe, C. Tocci, S. Holtzman, & J. Williams
MET Project Policy and Practice Brief
This paper, written by members of the ETS team that built the MET project’s observation scoring system, offers guidance to school system leaders on the elements of observer training and assessment that can produce accurate and reliable results for teachers. Learn more about this publication - Reliability and Validity of Inferences About Teachers Based On Student Test Scores
E. H. Haertel
ETS William H. Angoff Memorial Lecture Series
In this report, the author argues that a systematic analysis of value-added models raises serious questions about their use as a tool for educational improvement. Learn more about this publication - An Argument Approach to Observation Protocol Validity
C. Bell, D. Gitomer, & D. McCaffrey
Educational Assessment, Vol. 17, No. 2–3, pp. 62–87
In this article, the authors develop a validity-argument approach for observation protocols that are used to assess teacher quality for high-stakes personnel and professional development decisions. Learn more about this publication - Including Students With Disabilities and English Learners in Measures of Educator Effectiveness
N. Jones, H. Buzick, & S. Turkan
Educational Researcher, Vol. 42, No. 4, pp. 234–241
This essay provides an overview of the challenges associated with accounting for students with disabilities and English learners in the evaluation of mainstream teachers. Learn more about this publication - Is Value-Added Accurate for Teachers of Students with Disabilities?
D. McCaffrey & H. Buzick
Publisher: Carnegie Knowledge Networks
The authors discuss challenges to the use of value-added models for evaluating teachers of students with disabilities and whether value-added measures for special education teachers are comparable with those of other teachers. View article - Will Teacher Value-Added Scores Change when Accountability Tests Change?
D. McCaffrey
Publisher: Carnegie Knowledge Networks
The author discusses the state of knowledge regarding value-added scores and how they are impacted by the type of test selected. View article
2012
- Assessing Quality in the Teaching of Content to English Language Learners
S. Turkan, A. Croft, J. Bicknell, & A. Barnes
ETS Research Report No. RR-12-10
This paper offers a chain of evidence collected to identify and verify the domain of teacher knowledge and skills required to teach ELLs content effectively in mathematics, science, reading/English language arts and social studies. Learn more about this publication
2011
- Substance and Show: Understanding Responses to NCATE Accreditation
C. Bell & P. Youngs
Teaching and Teacher Education, Vol. 27, No. 2, pp. 298–307
This article describes a case study of how universities in Connecticut responded to teacher education accreditation policy between 2002 and 2006. View publisher's page - Measuring Teachers' Contributions to Student Learning Growth for Nontested Grades and Subjects
L. Goe & L. R. Holdheide
National Comprehensive Center for Teacher Quality (NCCTQ)
This brief was developed to help states consider options for assessing student learning growth for the majority of teachers who teach content that is not assessed through standardized tests. Learn more about this publication
2010
- Recent Trends in Mean Scores and Characteristics of Test-Takers on Praxis II Licensure Tests
D. H. Gitomer & Y. Qi
U.S. Department of Education Office of Planning, Evaluation and Policy Development This study examines changes in teacher licensure scores from 1999 to 2006. The purpose of this study was to identify trends in Praxis® scores on a select number of tests across recent years and across as many states as possible. View report - Measuring the Effects of Professional Development on Teacher Knowledge: The Case of Developing Mathematical Ideas
C. Bell, S. Wilson, T. Higgins, & D. B. McCoach
Journal for Research in Mathematics Education, Vol. 41, No. 5, pp. 479–512
In this article, researchers describe a study that examined the impact of a teacher professional development program known as Developing Mathematical Ideas. The study considered how the program affected teachers’ knowledge in mathematics. View publisher's page
2009
- Models for Effective and Scalable Teacher Professional Development
M. Thompson & L. Goe
ETS Research Report No. RR-09-07
This paper presents the theoretical and empirical basis of an evolving program of professional development that employs school-embedded teacher learning communities as a central component, called Keeping Learning on Track. Learn more about this publication - Teacher Professional Development Focused on Formative Assessment: Changing Teachers, Changing Schools
E. C. Wylie, C. J. Lyon, & L. Goe
ETS Research Report No. RR-09-10
This paper outlines an approach to improving learning and teaching that combines two strong research bases: the research on formative assessment or assessment for learning, which provides information about what to change; and research on teacher learning communities, which guides decisions about how to change. Learn more about this publication - When Policy Instruments Combine to Promote Coherence: An Analysis of Connecticut’s Policies Related to Teacher Quality
P. Youngs & C. Bell
Journal of Education Policy, Vol. 24, No. 4, pp. 435–460
This paper explains how the Connecticut General Assembly and the Connecticut State Department of Education implemented a set of policies related to teaching and learning. View publisher's page
2008
- Approaches to Evaluating Teacher Effectiveness: A Research Synthesis
L. Goe, C. Bell, & O. Little
National Comprehensive Center for Teacher Quality (NCCTQ)
This research synthesis examines how teacher effectiveness is currently measured and provides practical guidance on how best to evaluate teacher effectiveness. The findings are presented along with related policy implications. Learn more about this publication
2007
- The Link Between Teacher Quality and Student Outcomes: A Research Synthesis
L. Goe
National Comprehensive Center for Teacher Quality (NCCTQ)
This research synthesis explores the evidence for the relationship between teacher quality and student outcomes in an effort to help identify which teacher qualifications and characteristics should be prioritized in educating and hiring those teachers who are most likely to have a positive impact on student learning. Learn more about this publication