ETS R&D Connections, No. 25. Conversation-Based Assessment

On-screen: [ETS R&D Connections, No. 25. Conversation-Based Assessment. Tanner Jackson. Diego Zapata-Rivera.]

Speaker: Tanner Jackson, Managing Research Scientist, R&D - My name is Tanner Jackson and I am a Research Scientist working in the Research & Development division at ETS.

Speaker: Diego Zapata-Rivera, Sr. Research Scientist, R&D - My name is Diego Zapata-Rivera and I am also a Research Scientist working in the Research & Development division.

Speaker: Tanner Jackson - In our article for R&D Connections we describe a new type of assessment called Conversation-Based Assessment. Within Conversation-Based Assessment humans interact and have a conversation with virtual computer characters. These are actual conversations that use everyday language and are designed to simulate what happens between a tutor and student.

Speaker: Diego Zapata-Rivera - In a normal human-to-human tutoring session, the tutor can talk with the student and ask questions that really pull out evidence of what the student knows or doesn’t know. Our work focuses on how to automate these interactions within a computer system. These automated conversations can adapt to what students say. For example, a virtual tutor can ask additional information from students who only give a partial answer or rephrase the question if the student did not understand it. So they can demonstrate what they know or can do.

Speaker: Tanner Jackson - An automated conversation with a computer is not necessarily a new idea. Conversation-based systems have been used in educational learning games, simulation-based training environments, and even intelligent tutoring systems. We are now leveraging this same technology and approach for use for assessment purposes. Since these interactions happen on a computer or even a mobile device, such assessments systems could potentially be scaled up and made available to many, many more students than would be feasible with humans.

Speaker: Diego Zapata-Rivera - By incorporating rich conversation information, we can complement existing types of assessment items like multiple-choice and open-ended responses. These interactive conversations can provide examinees with multiple opportunities to elaborate, can be used to elicit explanations about decisions that students make and can be embedded in a variety of assessment tasks, including simulations, scenario-based, and game-based assessments. We are evaluating the potential and limitations of conversation-based assessments. If we are successful in our efforts, future assessments may include conversations as a way of assessing students’ knowledge, skills and all attributes.

Speaker: Tanner Jackson - Conversation-based assessment represents some of the cutting edge research being conducted at ETS and we think it can have a positive impact on both teaching and learning. We hope you enjoy the article. Thank you!

On-screen: [R&D Connections. ets.org/research. ETS]

Speaker: Narrator - R&D Connections is a free publication, which you can download at no cost by visiting the Research section of ets.org.

End of Video: ETS R&D Connections, No. 25. Conversation-Based Assessment

Video duration: 0:03:09