What Do We Know About Good Teaching? Results of a Research Synthesis.

On-screen: [How Teachers Teach. A talk based on a project to map the terrain of teaching practice.]

Speakers: Gary Sykes of the Educational Testing Service, and Suzanne Wilson of the Neag School of Education, University of Connecticut

Gary Sykes, Principal Research Scientist, ETS - Good morning, everybody. We're happy that you can join us in the discussion today about teaching. This is based on work that Suzanne and I have been doing for the past year plus, and what we produced in this was something that we've called a map — a map of the terrain of teaching. So we'll talk a little more about the map metaphor in a moment.

The talk is based on the effort that we undertook, was to read pretty widely and deeply in the literature on teaching and try to use that as a basis for developing this framework of competencies.

On-screen: [On Today's Topic. The Report: Sykes, G., & Wilson, S. (2015). How Teachers Teach: Mapping the Terrain of Practice. Princeton, NJ: Educational Testing Service. Available at http://www.ets.org/research/topics/teaching/. Overview: Re-emergence of interest in teaching. Cultivating a practice of teaching. Building a strong teaching profession.]

So here's the report and the link to it. By way of background, just a few framing observations. We seem today — or at least my sense of it is — that we're at a new inflection point with the renewed interest among policy makers and educators in teaching. This may seem an odd statement, but much of the policy discourse over the past decade has focused rather on such things as standards; accountability; evaluation; data use; monitoring and sanction regimes; and incentives for various kinds to motivate students, teachers, and schools. Lost in this policy discourse has been a focus on what it means to teach well and how to support teachers in developing a practice of teaching.

Today though, I think there is a resurgence of interest in teaching and the teaching profession. Policy statements from the Center for American Progress and the National Education Policy Center (to name two new groups in the field that includes Teach Strong, Transform Teaching, and other initiatives) and shifting priorities on the part of the large foundations, such as Gates and others — all signal a renewed attention to teaching. Even as reports, such as The Mirage, suggest how difficult it is to develop professional development in support of good teaching.

So I'm just wondering, have you all noticed this change that's beginning to take shape in the field, just from your own observations and interactions with folks around Washington? Does this observation sort of roughly track with your sense of how the conversation is shifting, even as associated with the reauthorization of the Title I program and what's being built into that? That there's this new interest in teaching that I think is quite significant.

Running through this emerging policy theme is the idea that we need to cultivate a practice of teaching suitable to new goals and standards and a new catch word: "deep learning," that speaks to this interest. There's now interest in this concept, and connected to this theme of cultivation is a renewed interest in systemic elements that might support the development of a teaching profession. Certainly, the interest in teacher preparation counts, along with contentions about compensation, career pathways, new leadership roles, and others. The new round of reports and position statements set forth recommendations that touch on elements of this kind, shifting the focus from accountability to capability development. And this is the backdrop for our remarks today.

There are two main takeaways from our talk (we argue for two broad points today). The first is that teaching is complex work that never the less may be specified in useful ways but we must take care in how we represent teaching. Research proves an increasingly firm basis for this. The second is that we lack mechanisms or infrastructure that provides for integrated and synthetic work on teaching that brings together findings, insights, conceptual approaches, across the disparate literatures on teaching — the various traditions and research that have emerged in the field and really speak to one another. Our effort to burrow into the literature on teaching revealed many communities of scholars working on particular themes with relatively little discourse and connection across these emphases, leaving it up to teachers to work out how to respond and integrate the rising tide of guidance from research and development. We'll return to this point in a moment.

And now I'd like to turn things over to my colleague.

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut -  So the goal or vision that would underlie an idea for a framework for teaching is something that a lot of people have worked on, and we're certainly not the first or the last to try to tackle this. And our colleagues Pam Grossman and Morva McDonald spoke eloquently to the various ways in which this might serve sort some function and some of the challenges. So we'd just like to point out some of the things they've said.

Creating such a framework would require parsing that domain, figuring out how to carve it up, even though it's a moving target and quite complicated work. It would also require identifying an underlying grammar of practice, the sort of bones of what teachers do. And it would require the development of a common language for naming the constituent parts. This is something that we as a field in teacher education, at least, are painfully aware of the fact that we lack.

They also point out that in doing such work, such a framework could identify key components, those that are common across grade levels, across subject areas, across the context in which teachers work, across the different kinds of student groups that teachers work with, but also it could be sensitive to the subject-specific, student-specific, context nature of the work as well.

They note that — and we all know this from the standards movements from the 1980s and 1990s when people got in trouble tangling up issues having to do with statements about what students should learn with how teachers should teach — that a framework would have to be agnostic on various models or ideologies of teaching. And that it would have to work equally well to describe components common to both direct instruction and to more inquiry-oriented teaching, and everything in between, while offering the flexibility required to recognize the significant differences in how such components might be interpreted and enacted.

On-screen [A Framework for Teaching (cont'd). (Grossman, P. & McDonald, M. 2008. Back to the future: Directions for research in teaching and teacher education. American Educational Research Journal, 45(1), 184-205, p. 186).]

And finally, they conclude that this effort to parse teaching would need to be respectful of the difficulty of breaking apart such a complex system of activity and, they make the point, the dangers of doing irreparable harm to the integrity of the whole by making the incisions at the wrong places. One might not like the language, but the problem with carving up any kind of landscape is that people are very anxious to have categories and boxes to use in order to see and deliver programs of teacher preparation and teacher evaluation. And so the way that we carve up the landscape of teaching can help us say what's important about the complexity of the work, but it can also get us in trouble when people want to use that carving up to oversimplify. And they also say that such a framework would inform both research on teaching and improvement of professional education, which is a point that we will return to at the end.

So now Gary's going to talk about the purposes of our talk.

Gary Sykes, Principal Research Scientist, ETS - Just as an aside, another bit of work that Suzanne and I did was — in the interest to build this map — was to look at parallel cases across other professions. We looked at medicine. We looked at nursing. We looked at law, and so on. And one thing that was notable in these other cases, these other fields all have consensus competency models upon which many other things are based. They're there. They're understood. They've become institutionalized in the profession. And typically in the sorts of policy frameworks that guide the profession, we don't have an analog for that in the teaching field yet, but I think it's quite important.

So on the mapping metaphor: The guiding metaphor that we adopted here was the metaphor of a map. And, of course, if you think about maps, maps can represent territory at a lot of different sizes. You can have a map of the United States. You can have a map of a particular state and the map of a city, even maps of neighborhoods within cities. So maps are nested within one another. This is the image that we were using to try to portray not a geographical terrain but a conceptual terrain of teaching. It's a kind of a map.

On-screen: ["Mapping" the Terrain of Practice. On the mapping metaphor... Purposes of today's talk: 1. Present the results and highlight a small, critical set of issued prompted by this interpretive synthesis. 2. Invite discussion of these issues and their implications for teaching research, policy, and practice.]

And we want to begin by first just previewing the map for you; letting you know (sort of) how we organized the map at a fairly broad level. And noting the map, we have to call an interpretive synthesis. In interpretive synthesis we use wiggle room, because, in truth, when you look carefully into the research on teaching, you find, as Don Schon once wrote, "high, hard, ground and swampy ground." That is with to some aspects of teaching there is a high, hard ground of well-regarded technical knowledge that can be drawn on, but with respect to other aspects of teaching, it's swampier. It's less clear. There isn't a solid research base. There's descriptive work and other sorts of things, but the terrain here is very uneven.

So inevitably when you attempt an exercise like we did, you've got to use... there is an inevitable degree of just interpretation going on in how one constructs a synthesis that includes both the high ground and the low ground. So first take a look at the map, and then we want to focus the talk on just a few key aspects of this work that we'd like to draw to your attention that we think have interesting implications. The map ought to be regarded neither as the first nor the last word on this subject and we invite your thoughts, comments, critiques when we have a chance to open it up for discussion.

So the other thing we can notice is that ours is not the only effort to try to represent the terrain of teaching. There is, in fact, a sprawling landscape of related work with representations in the CAEP standards, the National Board standards, the InTASC standards, licensure examinations that require some kind of framing of teaching of different sorts, subject-area professional standards, teacher development and evaluation protocols, and Measures of Effective Teaching. So this report enters this sprawling landscape and, to some extent, operates in conversation with these other representations, all of which we looked at and which we drew on in the work that we produced here.

On-screen: [Mapping Teaching in the Field of Education. This report enters a sprawling landscape of related work: CAEP (Council for the Accreditation of Teacher Preparation); NBPTS (National Board for Professional Teaching Standards); InTASC (Interstate Teacher Assessment and Support Consortium); Licensure Examinations (EdTPA, Praxis, NOTE); Subject area professional association standards (e.g., NCTM, NCTE); Teacher development and evaluation protocols (e.g., Danielson, Marzano, CLASS); Measures of Effective Teaching.]

So onward.

Any effort of this kind requires some set of assumptions, and we articulated our assumptions in the report. And here, I just want to draw attention to a few of them that will help orient you to what we've produced. We pitched competence at a relatively high level of accomplishment, characterizing these as stretch goals for the profession. They map then not simply onto a stamp practice, but onto practices that are advocated by scholars and policy makers and that constitute the direction for improvement. In part, too, this choice oriented to new goals and standards for learning that policy makers are now advocating, not with one voice but in terms of a general tendency.

On-screen: [Assumptions Underlying the Synthesis. Stretch goals for the profession. New standards of learning. Principles of learning-role of student conceptions, prior knowledge, deep understanding of subject matter, meta cognitive aspects of learning (NRC, 2000). Some key aspects of learning. (National Research Council. 2000. How People Learn. Washington, DC: National Academy Press).]

Because teaching should attend to what is known about how people learn, we drew syntheses of research on learning and development — most widely and authoritatively provided in the national research report, How People Learn. And the title of our paper, called How Teachers Teach, was meant to invoke that widely cited report.

That report drew attention to three big principles of learning, and they included the fact that students bring preconceptions and prior knowledge to the classroom, which teachers have to access to promote understanding; the subject matter: if learning involves both a deep foundation of factual knowledge together with conceptual frames for knowledge retrieval and use; and finally, the theme of meta cognitions. The meta cognitive approaches help students assume control of their own learning and students must develop regulatory mechanisms that help them learn how to learn, to be aware of and to monitor their own thinking and their ways of learning.

And finally, we make a set of assumptions about teaching. As we will pursue in a moment, we draw attention to a conception of teaching as iterative and dynamic, as highly situated, as involving continuous adaptations to learners and learning, and as balancing general with subject-specific instructional guidance and practice.

Other of our assumptions are in the paper, and you can take a look at those when you look at the paper. Thank you. Suzanne...

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - So, the first, simplest parsing that we did was split the framework into two pieces, two domains. One having to do with everything related to instruction and what happens when teachers are in classrooms with students or getting ready to go into classrooms with students. And we call that the Instructional Domain. And then, the second domain, having to do with everything else that teachers are responsible for: their legal responsibilities, their ethical responsibilities (which all of these things also happening in classrooms), their relationships with parents and communities and with other professionals.

So our first parsing was just two simple domains. The first one takes up a lot more space in the paper, as you can imagine.

Then in each one of these domains, we then parsed it into categories that will be familiar to everyone. These are the categories that one finds on teacher development forms or categories in other national standards and efforts to describe teaching and provide frameworks for teaching. And we array them in a cycle, but this is a misrepresentation of what actually happens with teachers in some ways, because these are all in an iterative, interactive stream of work that someone does. And so in some ways, teachers are planning before they start a lesson, but sometimes their planning in the middle of the stream of instruction. And so we parsed out planning, the relational aspects of teaching, the social and academic life of classrooms, everything having to do with interactive (when I'm working with a group of students at a moment in time or I'm setting them up to work with one another) and then everything having to do with how I improve my practice. And, again, these are in a stream, and we do not understand them to be bi-directionally or linearly related to one another. This is related to the parsing and the problem with imposing a structure on things, because people could turn this into a linear model where they look for these things in some sort of an order. And we're sensitive to that being problematic.

Then we did a similar thing with the Professional Responsibilities Domain, and we identified four different areas in the same kind of relationship to one another: collaborating with other professionals, working with families and communities, fulfilling ones ethical obligations, and fulfilling one's legal responsibilities.

So now, Gary is going to present an example of... and so then in each one of these categories the framework consists of practices that we have identified within each one of these subdomains. We intentionally tried to pick up and take seriously the current interest in understanding what teachers actually have to do, not just reciting what they need to know, which has been a tradition in frameworks and in standards of teaching and teachers work. And that current emphasis is on trying to understand what it is we actually expect them to be able to do and what you would see if you notice them engaged in practice. So the report, within each one of these domains, focuses not exclusively on practice but tries to spend a lot of time talking about what does research say, about the behaviors that teachers enact in their work in these two big domains.

And now, Gary's going to give you an example of one.

On-screen: [An Example: Competence as Nested. Domain: Instruction. Sub-Domain: Interactive Teaching. Practice: Orchestrating Productive Discourses. Sub-Practices: 1. Employing multiple participation formats for classroom discourse. 2. Establishing norms and routines for classroom discourse oriented to particular subject matter domains. 3. Integrating student cultural practices into classroom discourse.]

Gary Sykes, Principal Research Scientist, ETS - Yeah, I mean, you know, you can imagine statements that might occur in standards that could begin at a really high level in this direction. Like, the competence would be enacting instruction effectively. And of course, something like that raises all kinds of questions, like "What do you mean by that?" So what you need to do is to begin to progressively drill down from higher levels of abstraction into teaching itself. And we'd just like to give you one example of that.

So if we go back to our picture, we're going to pick something out from here, "interactive teaching," and take a closer look at it.

On-screen: [An Example: Competence as Nested. Domain: Instruction. Sub-Domain: Interactive Teaching. Practice: Orchestrating Productive Discourses. Sub-Practices: 1. Employing multiple participation formats for classroom discourse. 2. Establishing norms and routines for classroom discourse oriented to particular subject matter domains. 3. Integrating student cultural practices into classroom discourse.]

So within interactive teaching, one of the practices that we already identified, we call: Orchestrating Productive Discourse. Teachers have to manage discourses, one element of instruction. And we further decompose this into a set of three broad practices. Now if we look at this one employing multiple participation formats for classroom discourse (the concept of participation format is in the literature on teaching), we can then take a look at a particular discourse format (which is now being very widely advocated in literature), which is: discussion-based teaching.

On-screen: [Nesting Practice: Leading Discussion. Prompting participation from all students. Steering discussion toward the learning goal: Uptake or linking talk, Revoicing, Press. Representing the content. Concluding the discussion.]

That is moving more of instruction away from lecture and initiation response (IRE) patterns toward more discussion-based teaching in which children are encouraged to offer their own thoughts and talk with one another to build collective and shared understanding. The teacher has to manage that kind of a pattern of discourse in a classroom. Most of the research, by the way, suggests this is a very complex practice for teachers to manage well, but it's very prominent in the literature as being tied into contemporary standards for learning the common core standards and others.

Within this discussion format are a series of further sub-practices. And for example, within this one, steering discussion toward the learning goal, the research has identified even finer grain practices: uptake or linking talk, where teachers take up what students say and link it to what other students are saying. It is a particularly important practice: Revoicing. Where teachers say, "I heard you just say that," so they revoice what the students have said and their engaged in this revoicing practice or press. Press refers to when teachers don't accept a partial or incomplete response from a student. They press them to elaborate, to support contentions with evidence — so their pressing student thinking. So these practices and sub-practices, then become even more fine-grained, and we can go, yet, a step further into what might be called particular mores of teaching.

And here... So here is just one example of a yet further decomposition of this high level practice down into much more descriptive, operational representations of what this looks like. And this case, from a study of elementary mathematics: Asking a student to explain someone else's solution: "Joey, can you explain what Natalia did?" Discussing differences between solutions: "Let's look again at what Dylan said. Dylan said it is a whole number. Stella, do you want to respond to that, given what you said to start with?" Making a suggestion to another student about his or her idea: "What is he going to have to do with that set of numbers? With 387? What does he have to do, Grayson?"

And this goes on to additional teaching moves, connecting students' ideas to other ideas: "Joaquin, can you see what Enrique is doing or what Natalia is doing and see if it looks like yours? Or if it's different?" Getting a student to create a solution with another student: "Griffin, why don't you sit down and work on the problem with Easton?" Using a solution that was shared with another student: "See how Paige counted? Could you take this problem and count like her?"

On-screen: [Teacher moves, cont'd. (Franke, M., Turrou, A., Webb, N., Ing, M., Wong, J., Shin, N. & Fernandez, C. 2015. Student engagement with others' mathematical ideas: The role of teacher invitation and support moves. The Elementary School Journal, 1161, 126-148)]

Do you get the idea? Now we've moved from a high level representation of a practice and just moved down, down, down, drilling further and further down into teaching. And so a broad claim in our paper on the structure is that teaching is a nested practice in this sense — that you get nests with increasingly finer grain sizes. And what good contemporary research is doing is operating at this level to identify exactly the sorts of teaching moves that effective teachers are using when; for example, they're trying to enact discussion as the participation format within their classroom.

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - So this kind of nesting is important in a number of different ways. The higher level nesting lets you collect particular sub-practices and micro-skills into bins that you can think about how to evaluate, how to insert in teacher preparation, and which ones (that whole conversation about high leverage practices and teacher preparation as being a part). A question about which of these practices is most important for beginning teachers to master and then what can be left until you get your feet under you and you begin to develop. This is a big question in teacher evaluation about the difference between someone observing you and saying that you can't do something and someone giving you feedback about what it means to then actually do it. And the lower level, more micro-analysis of what the practices look like and the insights into what this looks like in the contextualized, situated work of teaching is very important for teachers' development.

So these levels of organization in this nesting operates in lots of ways that can help training, evaluation, professional development, and assessment. This kind of nesting allows us to begin to create a common language and a conceptual approach that allows policy makers and teachers and practitioners and instructional leaders all to communicate with one another. It provides a basis for specific research practices that can be taught and learned. So this is... It also reveals holes in the literature where there isn't good guidance about what something looks like when you go into a classroom. Our field suffers, might I say, from a tendency to love high level rhetoric about everything teachers should know and be able to do. And I am a sinner and have written a number of those things. But when one reads that kind of rhetoric, it doesn't really help one (if you're interested in going into a classroom and doing it) to know what it means to do it. And so we need both. We need the higher level understanding of these things so that we can cluster things, but we also need this much more micro-analytic.

So that's an example of what the framework contains and, as we will discuss later on in the talk, each one of these sections is the result of a synthesis across lots of different domains of research and professional organizations' language about what teaching looks like. And so none of these is associated with one body of literature. Each one of these domains pulls on multiple bodies of literature.

So now we're going to turn to the second part of the talk. Gary and I have written a lot of documents that are sort of like this. We wrote... We helped the National Board generate the first version of the National Board for Professional Teaching Standard's standards. We have both worked with InTASC and with the Chief State School Officers on various projects. And so I think it's fair to say we're both a little bit surprised about how much we learned — and how much there's still yet to be learned — about trying to wrap your mind around what it means to synthesize everything that we actually know about instruction and the fundamental knowledge about learning and students that teachers need in order to do their work well.

So we have lots of things that we learned, but we picked three to focus on talk to you about now. So in the second part, we're going to talk about three things that we learned when looking across and trying to synthesize this literature. And the first one, which Gary will talk about, is about sequencing instruction.

Gary Sykes, Principal Research Scientist, ETS - Now this was a bit of an insight that struck me as we were doing this work. There's an aspect of teaching that we argue has not received the attention it deserves and that has to do with how teachers sequence instructional moves over varying spans of time. It's not true that it's received no attention at all. For example, most of you, many of you will know that Deborah Ball and her colleagues have identified a set of what they call high-leverage practices — high-leverage teaching practices — and one of these in this set is designing single lessons and sequences of lessons. And they describe it in these terms: "Carefully sequenced lessons help students develop deep understanding of content and sophisticated skills and practices. Teachers design the sequence lessons with an eye towards inviting opportunities for student inquiry and discovery, include opportunities for students to practice and master foundational concepts and skills before moving on to more advanced points. Effectively sequenced lessons maintain a coherent focus while keeping students engaged and also helps students achieve appreciation of what they've learned." So there's one representation of the importance of sequencing.

Three critical timespans require attention in the design and implementation of action sequences. Certainly sequencing occurs within lessons. That is, teachers are making moment-to-moment decisions about how a sequence moves in response to what students are saying to them. And these are all often on the spot and in response to what students are saying. So there's sequencing that teaches decisions teachers are making in action during lessons. Sequencing also occurs across lessons. And here's a little stylized example of this quoted from a report: "An introductory lesson for first graders on predicting events in a story or decoding words with a silent 'e' — tap and tape — might involve explanation, modeling, scaffolded practice, and gathering formative assessment information. The next lesson might involve review, re-teaching, or extension combined with additional practice and assessments of students' grasp of the new concepts. Follow-up lessons are likely to focus on practice and application, including small group activities or discussion. At the appropriate point, based on information from formative assessment activities, a summative evaluation should be administered. This might occur at different times for specific students." So you get this idea here that where effectiveness occurs is to some extent tied to teacher decisions as their sequencing their work over spans of time.

And then finally, of course, sequencing occurs over longer spans of time that involve how teachers start with planning and enact lessons then reflect on how well the lessons have gone. Maybe they'll have to go back and learn some things where they realized there were some errors in what they were doing or saying, and then teach it again, the next unit around. So there's these longer sequences of planning, action, reflection that teachers are engaged in. But I think the claim that we'd like to make about this is that the sequencing moves and adjustments teachers make over these phases of thought and action contribute directly to the quality of their work and the outcomes they achieve with students. But we hasten to add, this claim requires further conceptual and empirical work to fully substantiate.

Now, why should this attention to sequencing matter? Well, we think it's the case for several reasons. Sequences constitute naturally occurring units for inquiry. Studies need to attend to how teachers sequence their work and what constitutes it. Few studies to date have done this. Action sequences tend to be omitted from the measurement in assessment. Performance measures, such as ETS is developing in NOTE and in other cases, look at bounded instances and, likewise, observations attend to single classes.

Licensure measures, traditionally, have had attended to samples of knowledge used in teaching, absent any consideration of how such knowledge is deployed over units of time. Preparation programs, the third point, often require students to develop detailed lesson and unit plans, but then don't follow that into enactment in the classroom to look at how those plans are enacted and how teachers have to adjust those plans in response to what's actually going on when you actually try something out in a class. Children have a wonderful phrase for it. He called it the "vicissitudes of intentions," or to put it more colloquially, with teaching things always fall apart, things never go as expected. And so teachers are having constantly to make adjustments to that to figure out "what's my next move now, 'cause it didn't go quite the way I had it laid out in my plan."

We argue that, because one reading in the literature on teaching identifies action sequences over varying units of time. As a critical dimension, such units should enjoy greater place in research and preparation, and in evaluation. This is a tough contention, because it presents practical and logistical problems for studying these. But we commend attention to efforts that address these problems and launch experiments about them. We haven't had as much of this in the field. So... [phone alarm sounding]

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - That's his timer telling him he has to stop. That you might make the connection to the current work that's very interesting having to do with students learning trajectories in various fields. And that is directly addressing this particular problem. But there isn't as much of that in the field to inform how we would think about the preparation of initial early career teachers to know how to sequence things inside, in real time and over time, or what the difference is between the sophistication models of someone when they're earlier in their career and the sequencing they can do in later on in their career, and how they understand how they sequence things.

So our next example is adapting instruction, and this is something that almost everybody who pays attention to anything having to do with teaching talks about, in some way, but the language that we use is quite different. Some people talk about differentiation of instruction. Some people talk about making accommodations for students. Some people talk about being culturally relevant or responsive to different groups of students. People talk about making connections to students' funds of knowledge and their home lives. And learn the boundary between schools and classrooms, so as to make more connections to students and get them as engaged as possible in learning the subject matter, becoming part of the community of the classroom. And so there's lots and lots of literatures that are about adapting instruction and, unfortunately, they don't necessarily talk to one another.

We struggled in the report about how to talk about adapting (because we found ourselves), and this was an artifact of the problem of the parsing. You want to say a teacher has a plan and then she adapts it given her knowledge of the individual students in her classroom. But you actually don't want to say that, because teachers are always thinking about the students in their classrooms. And they don't have an idealized normative script of how they're going to teach something. They're always, even from day one, using their understanding of the children who are in their room to think about what makes sense. And so it's not a case that there is content and then we adapt it. There are students and there are things that we are teaching them about — subject matter and about how to get along with one another and how to develop social and academic skills — and all of that goes on in the teacher's head at the same time. And so this was an overarching issue that we thought about constantly when we were writing this report. Another literature that's really relevant here is the literature on formative assessment, because that whole literature is about giving teachers tools, so that they can adapt instruction, so that they can make accommodations, so that they can figure out in real time what's going on with their students and how to change course.

So for example, if I was going to work with a group of my students and I was going to be using small groups, there's lots of things I would think about. I would think about their background characteristics and what they're bringing to the classroom, because each group would be constituted with different experiences. So I might want to spread the knowledge of something from the community across various groups so that all the kids who knew something about a particular topic or about a particular community weren't in one group. I would think about who works well with others. If there are children in my class (I teach adults right now, but there are always troublemakers, no matter what ages that you're teaching), I am not going to put the troublemakers together in a group, unless that's going to serve my purposes. I'm going to spread that kind of wealth, richness, and opportunity as well. So when I'm building those small groups in my classroom, I'm thinking about all of these things, and then I'm constantly assessing whether or not those small groups are doing what I want them to do. And in the report it's these decisions that you make in your classroom are kind of like being in a hall of mirrors, because there's so many dimensions that you're thinking about. You're thinking about language. You're thinking about gender. You're thinking about race and culture and ethnicity. You're thinking about parents. You're thinking about the community. You're thinking about the subject matter. There are probably an infinite number of vectors that you could be thinking about. So any one description is meaningful in that context but we could not in the framework do a comprehensive listing of all such descriptions. That would be impossible.

So you can't describe everything, but you also have to be very careful that when you write things about adaptation, you don't "essentialize" students, because you don't. It is not particularly helpful to say "all children that come from this background have these tendencies." Because you can't essentialize them. We're all lots of different things. I'm a woman. I'm a nerd. I never had any friends. You don't need to know that. But, there are all sorts of things that I am, and if you put me in a box as people did when I was in school...I was the oldest daughter of an Irish Catholic family, and I grew up at a time when women were expected to be either nurses or secretaries. I was not planning to be a nurse or secretary. My father was a physicist. He had other ideas about me being an engineer, but the counselors in the school told me that really I needed to be thinking about secretarial programs. Because it was a time when that's what they told women to do. I was essentialized. And we do that to children all the time in the name of acknowledging difference. And so you're caught when you're writing frameworks like this. The one evil – the rock - being that you cannot possibly describe all the very specific decisions contextualized thickly as they are in classrooms, but on the other hand, you cannot run the risk of essentialization.

So this was a challenge in the report. It makes me think back to the map metaphor. All of us (I'm sure most of you), now have these smartphones, and we're all always using Google Maps or something. First you just have to get to a place and then you actually have to find where 1800 is on the street. And so you spread the screen out and it goes in deeper, right? And for a report like this, this point (about the nesting and about the sort of different levels of operation). The knowledge that we have about teaching operates quite differently at those different levels. It is very hard to find the stuff that gets you right to this building (if that metaphor makes sense), because that is the hardest stuff to gather really strong evidence about. It's one of the reasons why we need very good qualitative, interpretive work that takes place in context and describes those things thickly. Because, those are our version of getting to a very particular point and helping somebody see what it means to do good work as a teacher in that particular classroom.

So research has addressed this particular feature of teaching in a variety of different ways and provides (if push comes to shove) a lot of the language operates at a pretty general level. It's actually very heartening, because if you look at the language of accommodation and of differentiation and of culturally responsive, it resonates. There isn't a great deal of contradiction across those literatures, but the problem is it's because it's operating at a level of abstraction that doesn't necessarily help me figure out what to do with Enrique on that day when I'm teaching about the Civil War. So we need more research that systematically and strategically thinks about the kinds of knowledge that will be helpful for improving... helping to support the teaching workforce at those different levels.

In another vein of work, that we — Gary and I — just finished on is trying to review all the literature on instructional policy for the fifth Handbook of Research on Teaching. We looked at this relatively new phenomenon in the last 15 years of policy that's trying to get into classrooms and really shape instruction and what teachers do. And, pretty much, you know, there's been a lot of very interesting work but a core lesson from that work has been that policy — this is such a surprise —doesn't really help that much in the end. It's rather fickle or problematic. You'd prefer fickle, actually, over the problematic. But getting into classrooms and figuring out how to make a difference in teachers' work really depends on (as Gary has already mentioned) thinking a lot about how to build teachers' capabilities — that we need to have alignment. We need to have these tools in the system. But if we do not invest in thinking about how to build a capacity of teachers to make these professional decisions that are so contextualize and situated, then those policies actually can't make the difference that they want to. And this, it's heartening to us that the discourse has, we both think, turned a little bit more to the understanding that we need to talk about teaching as a profession. We need to talk about is how do we in put tools and systems in place that help professionals do the work, because they are essential to it. We are not going to figure this out by programming a bunch of robots to do the work of teaching.

So Gary is now going to talk about a third example which has to do with general and subject specific.

Gary Sykes, Principal Research Scientist, ETS - How are we doing on time now?

Person off frame: Hmm, we can hurry up a little. (some words not able to understand)

Gary Sykes, Principal Research Scientist, ETS - Okay. I may truncate these slides a little bit. But basically the theory is that, and again this is something that you probably already know, that to some extent the literature on teaching divides between work that is looking for general practices of teaching that would be applicable across subject areas and grade levels and so on, and research that anchors deeply in subject matter. So that there is both a general and a subject-specific literature on teaching that developed in the field with some uneasiness about just how those two literatures might fit together.

On-screen: [(3) Balancing General with Subject-specific Guidance. General: Time on Task, Homework, Managing small groups. Specific: Cognitively guided instruction, Reading like a historian, Teaching mathematical or scientific practices.]

One version of... Well here's a few just examples of some of the more general literature topics in that literature that don't address subjects particularly but that have produced important findings. And then examples of some other lines of research that absolutely rely on what teachers know about the subject and how to teach it to particular students.

So, you know, one version of this is represented as a Venn diagram. And ask, you know, how big is the overlap? How big are the two circles? Thinking of these two things as separate domains that stand in some kind of relation to one another, we're more inclined to think of a different depiction is better for this.

On-screen: [Portraying the General and the Subject-specific in Teaching. General Teaching Practices; Subject-specific Teaching Practices.]

Rather than a Venn, something like a figure/ground relationship is a better representation. So, here's a famous example. Did everybody see the goblet or the vase? If you look again, do you see two people facing each other? So, the famous figure/ground, and how your shifting back and forth between these. They're both there in the picture. And these we suggest is more like this subject-specific dimensions and the general dimensions. In any sort of teaching, they are both there. And so, to a certain extent, the research that picks up these things tends to be oriented to what the researchers still are looking for. That is to say findings tend to be an artifact of the methods used. So if you're Deborah Ball and you want to go into an elementary classroom and figure out how to teach math, you're really concentrated pretty heavily on teaching math. But on the other hand, if you are like Jacob Kounin, another famous researcher, and you want to understand what good classroom management is like — how do teachers manage behavior in classrooms? There's no particular reference to the subject matter there. You study it in math classes or English language arts or science. It's general. So both of those things are there and, as we said, to a certain extent are artifacts of subject matter.

We could give examples from the literature of these different bodies of research. They're all there, and I think our point would be that with respect to things like preparation, professional development, evaluation and so forth, both need to be represented. Both need to be present. But there are several other implications of this sort of basic point, and one is that we could still ask the question with respect to student outcomes, "Which is more important?" So for example, I could point out to you general guidance about how to lead a good discussion and that guidance might not have any reference at all to what it is that's being talked about. It just has to do with instructional moves that teachers make. And the question would be, "If a teacher enacted those moves in a math class or in a social studies class, would they get good results with kids? Or would it rather be the case that in order to get good results from the kids, what's essential in that discussion is the way in which mathematics is being brought in by the teacher? The way in which the social studies content is being brought in?"

So one could conduct research that would work on which of these two reviews might produce better results with children. And not much of that research has gone on. But our point is that we need balanced attention to these, certainly in preparation, and finally, probably that which of these two perspectives you use might depend on your purpose. Take teacher evaluation. Likelihood is, for practical reasons, districts are not going to be able to create different evaluative frameworks for each subject at the relevant grade levels. They don't have a choice but to develop a common framework — Danielson, whatever — and use that simply because they couldn't implement the others. So for evaluation we might say, "Okay," and the evaluation framework probably has to draw attention to general aspects of good teaching. But then, you might say, "We better pair that somewhere else in the system with attention to subject-specific aspects." And that might point to professional development, because if the only account a district has of good teaching is just general, the research says you're really missing an important part of what is involved in effective instruction. So there's got to be a balance and so it depends on purpose.

And I think, you know, we for the most part don't have that balance right yet in policy and practice. And for the most part, we're drawing attention to general aspects of good teaching, which is fine as far as they go, but we're under-representing, under-emphasizing the subject-specific aspects of teaching about which there is an accumulating and important body of research that suggests importance.

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - So why don't we go to the last closing observations?

Gary Sykes, Principal Research Scientist, ETS - So you want Concluding Observations.

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - So we're in part three now.

Gary Sykes, Principal Research Scientist, ETS - We have three, just three, concluding observations. And the first is just some text drawn from our report. We wrote, "Frameworks such as this one are daunting for they seem to project a practice that is unattainable by the typical practitioner working in the usual circumstances of teaching. Consider, though, that this mapping sets forth competencies for the profession as a whole, rather than as an account or description of what typical teachers do in the varied contexts of practice. Such an account is aspirational not least because the practices described here are difficult to enact and call for a high degree of knowledge, skill, and will." But the last thing I would say about this slide is if you're familiar with all the contemporary discourse on standards of learning, what we want children to be able to know and do when they graduate from high school, when they go through elementary school. Our argument would be, we're not going to get those results unless we can dramatically up the game with respect to instructional practice. And, hence, this framework is oriented to what upping the game might actually look like with respect to competency. That's the aspiration.

Second point is drawn from a recent study where the investigators concluded, "Effective teaching cannot be measured by either checking for a limited set of 'best practices' or rating global aspects of classroom process quality only. Contrary to popular belief, no single reductionist approach to effective teaching will be sufficient. Instead, both can enhance student learning to some extent, but best results may depend upon combining specific and global" practices.

On-screen: [Concluding Observations II. (Decristan, J., Kleime, E., Kunter, M., Hochweber, J., Buttner, G., Fauth, B., Hondrich, A. L., Rieser, S., Hertel, S. & Hardy, I. 2015. Embedded formative assessment and classroom process quality: How do they interact in promoting science understanding? American Educational Research Journal, 52(6), 1133-1159. P. 1156)]

In this particular study, the best practice they looked at was embedded formative assessment, and that's a discrete kind of a practice that's being advocated. They also had measures of general and global features of the classroom in which embedded assessment was used. These three features included cognitive activation, classroom management, and providing a positive climate for students. What they found was that the teachers using embedded assessment got better results. The teachers who were enacting the global features within the classrooms got better results. And the best results were achieved by teachers who were doing both together. And that ties in with the subject-specific and the global, that part of the complexity in teaching involves both best practices that are being advocated, but also the implementation of general and important strategies that characterize effective practice.

Suzanne M. Wilson, Neag Endowed Professor of Teacher Education, University of Connecticut - And our third observation is one that we've alluded to, that there's no lack of enthusiasm for studying teaching and there's a growing... there are growing bodies of research that inform the work of teachers, but unfortunately there's nothing really in our system that regularly integrates that knowledge and puts it together, even though that is actually what teachers need to do. They need to put things together. So if you're lucky, you find scholars who are working at the overlap of two circles, like they're interested in English language learning and in science teaching. But teachers work in an environment where they have to be thinking about the literature on social and emotional growth, on creating... on managing classrooms, on math instruction, or on some other subject matter instruction, on things having to do with generic teaching skills or the use of formative assessment. And, for the most part, we don't have an infrastructure in this country, nor do we regularly have a way of synthesizing both large scale quantitative work that gives us some insight and small scale qualitative interpretive work that gives us other very important insights for these sort of different levels of need that the system needs. And so teachers are left on their own to figure it out.

And that seems, given how hard teaching is anyway, really stupid. So Gary will close.

Gary Sykes, Principal Research Scientist, ETS - I think we might conclude simply by saying that this report and this talk today was meant rather more to provoke than to settle matters. And we hope that it will be a stimulus for you to ask us questions and talk to us about some of these aspects of teaching when we get back together after lunch. Thank you all for coming.

End of Video: How Teachers Teach

Video Duration: 59:21