Foundational Psychometric and Statistical Research Center

The Foundational Psychometric and Statistical Research Center maintains ETS's legacy as a leading research institution for psychometrics, statistics and technology for educational assessment and learning.

The foundational research carried out by staff in the center has led to groundbreaking developments in:

• item response theory
• differential item functioning
• linking and equating
• test security
• methods of analysis for large-scale assessment surveys, including:
  • National Assessment of Educational Progress (NAEP)
  • Programme for International Student Assessment (PISA)
  • Program for the Assessment of Adult Competencies (PIAAC)
  • Trends in International Mathematics and Science Study (TIMSS)
• innovative approaches to process data
• automated scoring
• methodologies to assess score equity in the context of automatic item generation

As the fields of education and educational measurement enter the era of "big data," the center will continue to lead the field by developing measurement models and methods that combine the many possible sources of information about learners' knowledge, skills and abilities.

Sources of data can include typical assessment results, log files and process data. This data captures examinee behavior on a test and information collected over the course of learning activities (e.g., coursework, online learning resources, etc.). The center will continue to be a leader in developing psychometric, statistical and measurement methods using these new sources of information, along with the technology infrastructure to support them.

Staff in the center also help to support research, operational testing and ETS business needs by:

• providing statistical advice
• designing and developing:
  • future-state assessments
  • score reports
  • learning environments
  • data collection

Guided by learning science principles, members in the Psychometric and Data Science Modeling group within the Center innovate and apply methods from psychometrics, data science and artificial intelligence to big educational data. They apply these methods to evaluate and predict how learners:

• process and interpret information
• interact with tasks, technology and other learners

The current research of this group focuses on:

• process data mining and modeling, collaborative problem-solving, learner behavior analytics and test security
• response process data mining, analytics and modeling
• assessment of 21st-century skills, such as collaborative problem-solving

Researchers in the center recently developed a new method for aggregating student growth measures (e.g., at the school level) that borrows strength across years to improve accuracy and cross-year stability, particularly for smaller groups. The methodology was recently approved for operational use by the California State Board of Education.

The Research Technology Group within the center recently developed a web application to collect nearly a half million estimated Leaving Certificate Examination marks from teachers across Ireland, which were needed to produce final accredited grades.

The Institute of Educational Sciences^® (IES) recently awarded staff a grant to develop and validate open-source tools that help teachers manage student writing processes. The research team will validate a range of automated measures of student writing and writing processes for classroom management and formative assessment.

Center staff lead the field in developing methods for test security. Examples of this research have:

• been funded by IES
• been published in the Journal of Educational and Behavioral Statistics, Applied Psychological Measurement and the ETS Research Report Series
• served as the foundation of operational test security procedures for ETS testing programs

As leaders in educational measurement, staff have:

• held leadership roles in leading professional organizations, including the National Council on Measurement in Education (NCME), the Psychometric Society and the Northeastern Educational Research Association (NERA)
• held editorships for top journals, including the Journal of Educational and Behavioral Statistics and the Journal of Educational Measurement
• received several research awards