ETS® Proficiency Profile Test Use Guidelines

Introduction

The ETS® Proficiency Profile was developed to assist in the assessment of the outcomes of general education programs in order to improve the quality of instruction and learning. It is a test of college-level skills in reading, writing, critical thinking and mathematics designed to measure the academic skills developed through general education courses, rather than the subject knowledge specifically taught in those courses. Institutions planning to use the ETS Proficiency Profile should be aware of its possibilities and its limitations.

These guidelines provide information about the appropriate use of the ETS Proficiency Profile for those who use the scores. They are also intended to protect test takers from unfair decisions that may result from inappropriate uses of the test. Adherence to the guidelines is important.

Test Purpose

Review examples of the test items and the list of skills measured to verify that the skills measured are those that the institution seeks to measure. Examine samples of the score reports and statistical reports, to verify that they include the information that the institution needs. If the institution needs information about individual students, make sure to use the Standard form, not the Abbreviated form.

Selecting Students

If the purpose of testing is to make inferences about the performance of groups of students, it is important to test an adequate number of students from each of those groups, selected in such a way that the students tested from each group are representative of the group as a whole. The best way to accomplish this is to test all of the students. If the institution tests only a sample of the students, it is important that the sample include an adequate number of students from each group about which the institution wants information, selected in a way that will permit the results to be generalized to the group as a whole. It is particularly important not to limit the testing to students who volunteer to be tested, unless the institution wants information that applies only to those students.

Student Motivation

Assessments of student learning outcomes, such as the ETS Proficiency Profile, are widely used in higher education for accreditation, accountability and strategic planning purposes. Although important to institutions, the assessment results typically bear no obvious consequence for individual test takers. This lack of consequence can have a negative impact on student motivation. If the students are not motivated to do well on the test, their test scores will not reflect their actual skill levels.

ETS encourages institutions to implement strategies to boost student motivation to ensure that test results reflect your students’ actual ability. To help with motivation, students taking the standard form of the ETS Proficiency Profile can now earn performance-level certificates of achievement. Additionally, some institutions have had success motivating students by explaining how test results are used and how those results can affect the value of their college degree. To learn more about how you can motivate your students to perform their best, contact an ETS Advisor.

Limitations of Test Scores

A test contains only a sample of the tasks that students are expected to be able to do. On another sample of tasks designed to measure the same skills, the students might perform somewhat differently. Information provided on the score reports and in the ETS Proficiency Profile User's Guide enables the user of the scores to determine how much the scores could be expected to differ if a different set of tasks were used.

The reliability of the individual scores of students taking the Standard form — particularly the skill area scores and the proficiency classifications — should be adequate for counseling purposes and for the identification of students with problems in particular skill areas. These scores cannot be used as the basis for high-stakes decisions about individual students.

The ETS Proficiency Profile measures a specific collection of skills. It does not and cannot measure all the educational outcomes of interest to institutions of higher learning. When the ETS Proficiency Profile is used to evaluate an institution or any of its programs, it should be used in conjunction with other information. It should never be used as the sole means for evaluating the effectiveness of an institution or the educational progress of the students.

Limitations of Comparative Data

The data in the Comparative Data Guide are drawn entirely from institutions that use the ETS Proficiency Profile. Within any category of institutions, those that use the ETS Proficiency Profile are not likely to be representative of all institutions in that category. In addition, the numbers of students tested and the sampling procedures vary from one institution to another, and it is impossible to verify that the students tested at each institution are representative of all the institution’s students at the relevant class level (freshman, sophomore, etc.).

Confidentiality of Score Data

ETS treats all score data for individuals and for institutions as confidential. Individual data are released only to the institution of the students tested. Identifiable institutional data are released only to the institution providing the data, unless the institution gives written permission to release the information to others. Institutions using the ETS Proficiency Profile should adopt a similar policy for the data from their individual students.

Validity Studies

The general appropriateness of using the ETS Proficiency Profile to assist in the assessment of the outcomes of general education programs has been established by research studies carried out by ETS and others. ETS Proficiency Profile scores may be appropriate for some other purposes, but it is important for the user to validate their use for those purposes. Departments and programs using the ETS Proficiency Profile are encouraged to collect validity information by conducting their own studies. The ETS Proficiency Profile program staff is available to provide advice on the design of appropriate validation studies without charge.

Questions

If a department, program or test taker has questions regarding the appropriate use of the ETS Proficiency Profile, contact an ETS Advisor at 1-800-745-0269 or highered@ets.org.

 

Register to attend a Higher Ed Webinar!

Learn how ETS can help you with student learning outcomes assessments.

ETS Proficiency Profile Administrator Portal (Program Workshop)

Order tests, manage test administrations, run reports (for existing customers only)

ETS Proficiency Profile Annual Comparative Data Guide

Compare the performance of your students with those of a large group of students at other institutions.