skip to main content skip to footer

How to Tell If a Test Measures the Same Thing in Different Cultures

Frederiksen, Norman O.
Publication Year:
Report Number:
ETS Research Memorandum
Document Type:
Page Count:
Subject/Key Words:
Cross-Cultural Studies, Culture Fair Tests, Measurement Techniques, Statistical Analysis, Test Validity, Testing Problems


A number of different ways of ascertaining whether or not a test measures the same thing in different cultures are examined. Methods range from some that are obvious and simple to those requiring statistical and psychological sophistication. Simpler methods include such things as having candidates "think aloud" and interviewing them about how they solved the problem, and techniques such as using pantomime or moving pictures to give instructions. Another approach is to make the tests different in such a way that they measure the same construct — so that they are functionally equivalent. The variety of approaches that require statistical methods include analysis of covariance, comparing test performances at the level of the test items(e.g., comparing item difficulties), item characteristic curve theory, factor analysis, and a construct validity approach. An understanding of the psychological processes involved in performing the tasks involved in taking a test item, or performing an experimental task in a laboratory, is prerequisite to making judgments as to whether a test is measuring the same thing in two cultures. The methods described provide ways to improve the understanding of such processes.

Read More