Praxis® Performance Assessment for Teachers Overview Video

People in this video:

Narrators/commentators as noted below in text.

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS

Transcript Body

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: This recording provides a detailed overview of the PRAXIS Performance Assessment for Teachers. This product is designed by Educational Testing Service, a nonprofit organization founded in 1947. Our mission is to advance teacher quality and equity in education for all people worldwide.

On Screen: [Approximately 6,000 employees-major center for educational research. Develops, administers and scores more than 50 million assessments in 180+ countries, at more than 9,000 locations worldwide. Performance assessments: Praxis III, NBPTS, CalTPA, WA ProTeach, TLead, MET Study, MoPTA, PPAT]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: This is not the first time that we have developed performance assessments. You may be aware that we also developed a PRAXIS III Performance Assessment that included Classroom Observations, also National Board Certification with the National Board for Professional Teaching Standards, as well as the California Teacher Performance Assessment. We also developed tier two leadership assessments for the states of Washington and Kansas. We also participated in the Measuring Effectiveness of Teaching Study with the Gates Foundation. More recently we developed the Missouri Pre-Service Teacher Assessment, which has led to the development of the PRAXIS Performance Assessment for Teachers, also known as PPAT.

On Screen: [The Praxis® Continuum
Praxis Core Academic Skills for Educators
Praxis II Subject Area Assessments
Praxis II Principles of Learning and Teaching (PLT)
Praxis Performance Assessment for Teachers
School Leaders Licensure Assessment (SLLA)
School Superintendent Assessment (SSA)]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: PPAT fits very well within the entire continuum of PRAXIS offerings. We have teacher assessments for admission to teacher education programs, as well as content knowledge exams and assessments for school leadership and school superintendents, so the PRAXIS Performance Assessment for Teachers is another important piece of that continuum.

On Screen: [Research, Broaden the range of what we want to know about students and what we might see to give us evidence (Glaser, Lesgold, & Lajoie, 1987).
Capitalize on cognitive and educational psychology: how people learn, how they organize knowledge, how they put it to use (Greeno, Collins, & Resnick, 1997).
Put new technologies to use in assessment: to create new kinds of tasks, to bring them to life, to interact with examinees (Bennett, 1999). (Taken from Mislevy et al., Making Sense From Complex Assessments, 2002)]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: You may be familiar with research over time that has demanded assessments give us more evidence to pull upon how people learn and what they can do with what they learn, particularly in terms of how they put that knowledge to use. This demands that we create new assessments that require teacher candidates to perform new kinds of tasks.

On Screen: [Evaluating Teacher Effectiveness

Current measures for evaluating teachers are not often linked to their capacity to teach. Existing policies for measuring teacher quality either rely almost exclusively on classroom observations by principals ... teachers' course-taking records, and paper-and-pencil tests of basic academic skills and subject matter knowledge that are poor predictors of later effectiveness in the classroom. (Darling-Hammond, 2010)]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: This is parallel to recent research by Linda Darling-Hammond around evaluating teacher effectiveness. In that work she writes about how our measures for evaluating teachers must be more closely linked to their actual capacity to teach in the classroom. Classroom observations by principals, course-taking records of classes in college, even scores on pen and paper or computerized tests do not give us enough information about what teachers know and are able to do, so we need new assessments that will provide a prediction of how effective teachers will be on the job.

On Screen: [PPAT Overview
Developed by and for practitioners
Completed during student teaching (fall or spring)
Four tasks - one formative, three summative
Written responses to a series of prompts
Submission of artifacts/evidence
Sequential and developmental in approach
Includes a professional growth plan (related to teacher evaluation protocols and classroom observations)
Stored, submitted and scored online
Cost: $275]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: This assessment was developed by and for people who do this work every day. We were very careful to include classroom teachers and faculty members who work directly with student teachers in developing this work. PPAT was developed to be completed during the student teaching semester in the fall or the spring and it includes four important components. The first one is a largely formative component and the last three tasks are summative components. They involve written responses to a series of questions about what teachers are doing in their classroom and the claims they make about those choices, and they must provide the necessary artifacts and evidence to prove those claims.

PPAT is very sequential and developmental in approach in that it explains how teachers move through a continuum from Task 1, where they're just getting acclimated to the classroom, to Task four, where they're putting all the pieces of quality instruction together. PPAT also includes a Professional Growth Plan, which can easily be aligned to whatever teacher evaluation protocol is in place in a school or district. Everything is stored, submitted and scored in an online platform that ETS provides at a cost for $275.

On Screen: [National Development Team
ETS convened a panel of 26 educators from 17 states
Select faculty and cooperating teachers
Diverse panel of practitioners
Met in May and October 2013
Focused on standards, indicators and evidence
Created tasks, prompts and rubrics
Conducted small scale tryout of the process
Pilot began in January 2014
Visit www.ets.org/ppat]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: Last year ETS convened a panel of almost 30 educators from 17 different states across the country. We were very selective in making sure that we chose faculty members and cooperating teachers who work with student teachers on a regular basis. While we paid attention to ethnic and geographic diversity, we also paid attention to diversity across content areas and grade levels. The development team met twice in May and October of 2013 to look at the InTASC standards, as well as content area standards and Common Core standards, along with the indicators of how those standards might be met, and the evidence that teacher candidates would show to determine if they were meeting those standards. That team also created the tasks, prompts and the rubrics associated with PPAT and then conducted a small scale tryout of the entire process in their local school environments.

The national pilot began in January of 2014 and continued through April of 2015. More information is available on our website, www.ets.org/ppat.

On Screen: [Screen-shot of website titled ETS, Guiding your practice to promote learning and growth. ETS Performance Assessments. There are 3 menu tabs: For Test Takers, For States and Agencies, and For Educator Programs.]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: When you visit this website, you will see that there are three distinct sections. One is for state departments and agencies. Another section is specifically for faculty and educator preparation programs, and the third section is for individual candidates participating in the PPAT process.

On-screen: [The Website: www.ets.org/ppat, Resources and materials, Creating and submitting tasks, Task requirements and rubrics, Submission calendar, Understanding scores, Library of examples, Permissions]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: When you visit the website, you will find several resources and materials to help you complete the PPAT process. There is an entire section devoted to creating and submitting tasks that will help students navigate throughout the system. All of the task requirements and the rubrics for scoring each task are also available on the website. Both faculty and students may want to pay close attention to the submission calendar which provides detailed information about submission deadlines. There is also a section on understanding scores and how scores are reported. There is also a library of examples which provides both students and faculty samples from the actual submission system. And finally there is a section on permissions which students will need to review in order to get the appropriate consent from parents.

The first task is focused on the teacher candidate's knowledge of students and the learning environment. In this task they must demonstrate that they understand the classroom in regards to school climate and community and how contextual factors in this setting have an impact on instruction and student learning. They must show how they connect the context for school environment to the individual teaching strategies that they choose to promote student learning.

Task 1 is also an introduction to the online submission system and allows for beginner immersion into the process. This task is scored outside the online system, so the local faculty or supervising instructors may decide how they want to use Task 1 for assessing student development.

Task 2 focuses on assessment and data collection to measure student learning. In this task teacher candidates must demonstrate that they understand how to collect assessment data, how to analyze that data and to apply their findings to future instruction. In this task they focus on one major assessment within a broader plan, and they must show how the assessment is connected to the teaching strategies they use, the activities in which their students are engaged, the materials and resources that support those lessons, and their plan for next steps in terms of collecting more data about student learning. Task 2 also requires that the teacher candidate differentiate instruction around the needs of two focus students. ETS does not dictate who the focus students might be. It is the prerogative of the teacher candidate to determine two focus students for each of the Tasks, 2, 3 and 4.

Task 3 emphasizes instructional design for student learning. In this task the teacher candidate must show that he or she can develop instruction and use the appropriate technologies to facilitate student learning. The type of technology used is determined by what candidates have access to in a particular teaching environment, and candidates must choose the appropriate technology which will help them to focus on certain instructional strategies. They must show how they connect their instruction to the goals and to the prior knowledge of their students. They must also show how they adapt those goals, and the technology and the resources to meet individual student needs. They must demonstrate what their methods are for evaluating their impact on student achievement so that they might demonstrate not just what they did, but why they did it and how they know it's making a difference for student learning. They must also include a reflective analysis on how they will provide future instruction for the whole class. And as is the case with Task 2, they must also choose two focus students again and talk about how they will modify instruction around those students' individual needs.

Task 4 focuses on implementing and analyzing instruction to promote student learning. This is the task in which they have to put all the important pieces together. They must show how they plan and implement a lesson, how they incorporate research-based instructional strategies, how they adjust instruction for a whole-class environment as well as individual students with the focus on specific strategies that will engage students in academic language, higher-order thinking skills and literacy across the curriculum. Task 4 is the piece that includes a 15-mintue video. The video may be segmented to show different parts of the instructional practice, but not edited. Task 4 provides a 360-degree holistic analysis of the entire teaching cycle to show who a teacher is becoming as a classroom professional.

Task 4 will assess a range of standards, naturally with some overlap, because this is when the teacher candidate will show that they can put all the critical elements together –planning, instruction, guided practice, assessment, reflection and so on. Task 4 also requires a focus on two individual students in which the candidate must include their work samples and show how they as the instructor have had an impact on student learning and what that impact is. Task 4 is more heavily weighted in scoring. So Tasks 2, 3 and 4 are the tasks that are submitted in the online system and scored summatively.

On-screen: [Professional Growth Plan
Created at the end of clinical experience
Requires reflection on multiple observations by supervising instructor and cooperating teacher
Easily aligns with school-based teacher evaluation
Also involves a student survey
Reveals areas of growth to be addressed upon entering teaching
May be shared during job interview]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: A very important distinction for PPAT is emphasis on professional growth. Teacher candidates must complete a Professional Growth Plan at the end of the student teaching experience. This plan requires that they focus on multiple observations that they've received from their supervising instructor on campus, as well as the cooperating teacher in the K-12 environment. The growth plan is a template that may easily be aligned with whatever teacher evaluation protocol is in place in a particular school where the student teacher has been assigned. It also involves reflection on any surveys that they've conducted with their students to get some feedback on their performance. The Professional Growth Plan includes areas of growth that they may continue to work on when they move into the teaching profession. While the Professional Growth Plan is not scored, it is a learning tool to inform the teacher candidate’s work moving forward. It is their prerogative to share what they've learned about their practice in an interview with a potential employer.

On-screen: [Scoring
Standard-setting study to determine passing score
Trained raters who meet criteria for selection
Content-specific scoring (with deferral option)
Scoring at each step - such as planning, implementation, analysis, and reflection
Each task is scored by two different raters
Candidates get feedback within two weeks
Ongoing calibration to ensure inter-rater reliability]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: ETS conducted a multistate standard setting study in order to help states determine a recommended passing score. ETS is training raters to score each piece of the portfolio and raters must meet certain criteria. While PPAT is not content specific, it is content embedded in that teacher candidates must respond to all of the prompts through the lens of their particular content area, however we do ensure that the background of raters is content specific to the task that they score. If for some reason a rater does not feel comfortable with the content in a particular task, they have the option to defer that performance to be scored by someone else.

Scoring takes place at each step within the task – the planning, the implementation, the analysis, and the reflection. Each task is scored by two different raters, so every individual candidate will be scored by a minimum of six different experts across the field. As candidates move from one task to another, they will receive feedback to determine whether or not they're on the right track. And of course there will be ongoing calibration of raters to ensure inter-rater reliability.

On-screen: [Raters: May be program faculty, cooperating teachers. National Board Certified Teachers, and recently retired teachers (who meet rater criteria)
Trained on task requirements, scoring process and benchmark cases
Raters must qualify at each scoring session
Raters do not score their own students
Paid an hourly rate (approx. 45 -75 minutes per task)]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: Raters may be supervising instructors from teacher education programs, cooperating teachers from a K-12 environment, National Board Certified Teachers from K-12, or even recently retired teachers, so long as they meet the criteria for being selected as a rater. They are trained on what is required for each task, how the scoring process is implemented, and they will review various benchmark cases. Raters are paid at an hourly rate and we have anticipated that it takes between 45 minutes and 75 minutes to score each individual task. Raters must qualify each time they log into the system to score, but they will not score their own students. So no teacher candidate will be scored by supervising faculty on their own campus, but all of the performances are housed in the online system for centralized scoring and scored by raters all over the country.

On-screen: [Resubmission
Candidates may resubmit, one time, up to 3 tasks
One-time resubmission for $85 fee
Ongoing feedback guides decision to resubmit
Store work samples to create a library of artifacts and evidence to pull from
Final no-pass requires complete re-start
Students may appeal a final no-pass]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: There is a very comprehensive resubmission process in place with PPAT which allows candidates to resubmit up to three of their tasks. It is their option whether or not they want to resubmit Task 2, Tasks 2 and 3, or Tasks 2, 3, and 4. There is a one-time resubmission fee for $85 but the candidate has the prerogative to determine how many of the three tasks they would like to resubmit. Because candidates get ongoing feedback as each task is submitted, then it provides them guidance as to whether or not they will need to resubmit at the end of the semester. Candidates are able to store all of their work samples in an online library. So if they need to resubmit, they will already have artifacts and evidence to pull from through their online files. If after they have completed the entire process and resubmitted they still do not pass, then they will be required to repeat PPAT altogether. If for some reason a student is not comfortable with their final score, then there is an opportunity for them to appeal.

On-screen: [Added Benefits
What Educator Preparation Programs are already doing
Supported by research and designed by practitioners
Collaborative culture among higher education faculty
Available for all content areas
Transparent process fosters early introduction
Adaptable for alternate route candidates
Ongoing feedback loop as tasks are submitted
Opportunity to resubmit Tasks 2-4 for only $85]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: It is very much aligned with what teacher educator programs are already doing. Programs that are already addressing assessment, guided practice, reflective analysis, and differentiation around focus students' individual needs will find that PPAT fits very nicely within the course curriculum. It is supported by research and designed by practitioners. In many ways PPAT promotes collaboration among higher-ed faculty because those from the arts and sciences environment will be able to work closely with those in a school of education.

PPAT is available for all content areas not just high-volume areas, and this is particularly important for states that have a wealth of different certification areas. PPAT is a very transparent process because all of the prompts and rubrics are readily available on the website, so students may prepare in advance. PPAT does not have to be introduced solely in the senior year. Faculty members have the option to incorporate segments of PPAT in earlier course work before the student teaching semester so that teacher candidates will be well aware of what is going to be expected of them. It is also adaptable for alternate-route candidates so long as they have access to a classroom. Another added benefit is the ongoing feedback loop which gives teacher candidates some idea of how they are performing well before the end of the semester. And finally, PPAT affords them an opportunity to resubmit Tasks 2 through 4 for only $85.

On-screen: [Added Benefits
Centralized scoring - helps eliminate bias
Results stored in ETS's Data Manager
Professional learning for cooperating teachers
Solves reliability and validity issues for campus portfolios
Links to in-service professional growth and evaluation
Flexibility within the framework to address state and program needs
May replace Principles of Learning and Teaching tests which offsets cost to students.
ETS has done this before – many lessons learned!]

Narrator – Dr. Cathy Owens-Oliver, Client Relations Director, ETS: The centralized scoring process helps to eliminate both positive and negative bias. If your state is a Praxis user then performance on PPAT is included in Data Manager with all of your other Praxis information. This allows for longitudinal studies down the road. We have already received feedback from the K-12 environment that PPAT provides professional learning for the cooperating teachers who work with PPAT candidates in the classroom. It also solves reliability and validity issues for teacher education programs who have been conducting their own portfolios and may have run into issues with scoring and logistics. PPAT is also linked to in-service professional growth because of its connection to teacher evaluation protocols. There are certain flexibilities within the framework which allow teacher education programs or state departments to make certain adjustments. One may choose that one task be focused on a science lesson and another be focused on math. One may also choose to do Task 3 before Task 2. These kinds of flexibilities help to accommodate individual program needs.

States requiring the Principles of Teaching and Learning exam have the option to drop that assessment when they adopt PPAT. And last but not least, ETS has done this work many times before and has learned many lessons in terms of developing performance assessments.

ETS provides access to the submission and authoring site 24 hours a day, 7 days a week. The Call Center is available Monday through Friday, from 8 a.m. to 6 p.m. Eastern Standard Time. ETS provides technical support for students as they navigate through the system and a wealth of support materials to both candidates and faculty. ETS also provides webinars periodically and by request to both students and faculty across the country.

Thank you for listening to this recording. For more information you may visit our website at www.ets.org/ppat or write to us at ppat@ets.org.

[END OF WEBINAR]

Video duration: 18:48