Test developers usually accept available criteria as final, and attempt to revise predictor tests to agree with these criteria. However, there are numerous instances where predictor tests that, in terms of content and previous data, clearly should have high validities but have low validities. Conversely, tests that should have low validities may have high ones. Alterations in criteria have often resulted in appropriate reversals in the magnitude of validity coefficients. It is urged that all aptitude test development include evaluation of the criteria, particularly when some of the validity coefficients seem to be inappropriately low or high, or unduly variable.