We Heard What You Said: Drew Gitomer on Individual Category Score Information
Ed. Note: In this issue, we launch a new feature called "We Heard What You Said," in which ETS and Praxis™ program representatives will address issues of importance to customers. Drew Gitomer, Distinguished Presidential Appointee in ETS's Policy Research Division, discusses the company's decision to provide category information to institutions of higher education.
Beginning with September 2006 Praxis test administrations, ETS will provide institutions of higher education (IHEs) with the same level of individual student category information that the company provides to test takers. The information will come with a disclaimer because of ETS's legal and statistical concerns about the data it will be providing.
The Background:
For many years, IHEs have been asking ETS to provide individual category information in the score reports they receive. The IHEs say this information will help them assist candidates in developing study plans and supply accrediting agencies or SPAs with the information they require.
Concerns:
After a thorough multi-year review of the issue, ETS expressed its concern that the individual student category information may not meet the company's strict regulations for releasing data that meet its reliability and validity requirements. This is because of the relatively few items and varying category difficulty across different editions of the same test. These constraints may limit the ability of IHEs to make firm decisions about the instructional direction they should take with individual students.
Short-Term Solution:
Because ETS understands the usefulness of the category information to IHEs, as a short-term solution beginning in September 2006, we will provide IHEs with exactly the same level of category information that students receive on their score reports. The information provided to the IHEs will be accompanied by a disclaimer that will provide direction on how the information should be used and the limits of its use.
Long-Term Solution:
ETS psychometricians are at work on a long-term solution that will provide IHEs and test takers with category-level information that may be interpreted meaningfully and with a high degree of confidence.
Although there is no time frame for the completion of this project, the entire issue demonstrates ETS's ability to work with its clients to meet their needs while safeguarding the standards of quality and fairness that have made us the world leader in educational measurement.