skip to main content skip to footer

Construct Validity of Free-Response and Machine Scorable Versions of a Test of Scientific Thinking FH

Author(s):
Carlson, Sybil B.; Frederiksen, Norman O.; Ward, William C.
Publication Year:
1978
Report Number:
RB-78-15
Source:
ETS Research Bulletin
Document Type:
Report
Page Count:
70
Subject/Key Words:
Graduate Record Examinations Board, Creativity, Factor Analysis, Formulating Hypotheses, Models, Scoring, Test Construction, Test Validity, Tests of Scientific Thinking

Abstract

The purpose of the study is to investigate the construct validity of free-response and machine-scorable versions of Formulating Hypotheses (FH), one of the Tests of Scientific Thinking. Six free-response and eight machine-scorable FH items were administered (in that order) to 172 college seniors who were majoring in psychology and who claimed an intention to go to graduate school. Tests of other variables that are theoretically related to FH scores provided data for studying construct validity; these included tests to identify eight cognitive and three personality factors. A background questionnaire was also administered, and GRE scores were obtained from files. cannot be considered parallel. The correlations are: Scores were obtained representing the quality, the number, and the number of unusual or "creative" hypotheses obtained with items in each test format. Correlations between corresponding scores for the two formats, when corrected for unreliability, indicate that the two forms cannot be considered parallel. The correlations are particularly low for scores based on numbers of hypotheses. The major method of analysis employed maximum likelihood factor analysis to verify a factor model involving the construct validity variables and to find the extension loadings of the experimental test scores and GRE scores on the factors. 0ne such analysis used cognitive factors and one examined personality factors. The personality factor model identified three factors; social extraversion, maladjustment (anxiety), and acceptance of complexity. GRE scores and quality scores from FH all were related to acceptance of complexity, and GRE scores were all negatively correlated with social extraversion. Number scores from the free-response form of FH were related (positively) to maladjustment. The cognitive factor model involved eight factors: verbal comprehension, three reasoning abilities, cognitive flexibility, two divergent production factors, and knowledge of psychology. GRE scores and quality scores from both forms of FH all showed substantial correlations with reasoning, cognitive flexibility, and knowledge. Only the number scores from the free response form of FH were related to divergent production. An appraisal of all the findings suggested that the machine-scorable form of FH would add little to the information already available from GRE scores. The freerespon form would add information about divergent production not now provided by GRE tests. (70pp.)

Read More