Understanding the Essentials of the SuccessNavigator™ Assessment

People in this video

Narrators/commentators as noted below in text.

Narrator – Ross Markle, Senior Research and Assessment Advisor for the Educational Testing Service

 

Transcript Body

On-screen: [Understanding the Essentials of the SuccessNavigator™ Assessment. ETS® SuccessNavigator™]

Narrator – Ross Markle: In order to assess the psychosocial skills that are critical to success in higher education, we at ETS® have developed the SuccessNavigator™ assessment — a test that is designed to measure noncognitive skills, indicate likely success and provide targeted feedback and action plans to help students, faculty and staff focus on areas of need. We'll now take a few moments to discuss some of the key features of the SuccessNavigator™ assessment, including the Score Reports that are designed to guide institutional actions and improve student success.

The SuccessNavigator™ assessment is an online, nonproctored tool that is designed for incoming college students. Given its focus on improving student success, we consider this assessment a low stakes, diagnostic and developmental tool.

On-Screen: [What is the SuccessNavigator™ Assessment?

In addition to measuring these skills, the SuccessNavigator™ assessment combines students' scores and other data to provide three key pieces of information. First, academic and psychosocial factors are weighted to provide indices around students' likely academic success and persistence. Second, course placement indices are provided to guide course placement decisions, recommending acceleration for students who might be near a cut score for a higher level course. Third, and perhaps most importantly, Score Reports to both students and advisors provide targeted feedback and action plans that can be used in the context of advising, a student success course or even individually by the student.

On-screen: [Table:

General Skill Subskill Definition Example Items
Academic Skills Tools and strategies for academic success Organization Strategies for organizing work and time. I make a schedule for getting my school work done. I take due dates seriously.
Meeting Class Expectations Doing what's expected to meet the requirements of your course including assignments and in-class behaviors. I attend almost all of my classes. I complete the reading that is assigned to me.
Commitment Active pursuit toward an academic goal. Commitment to College Goals Perceived value and determination to succeed in and complete college. One of my life goals is to graduate college. The benefit of a college education outweighs the cost.
Institutional Commitment Attachment to and positive evaluations of the school. This is the right school for me. I'm proud to say I attend this school.
Self-Management Reactions to academic and daily stress Sensitivity to Stress Tendency to feel frustrated discouraged or upset when under pressure or burdened by demands. I get stressed out easily when things don't go my way. I am easily frustrated.
Academic Self- Efficacy Belief in one's ability to perform and achieve in an academic setting. I'm confident that I will succeed in my courses this semester. I can do well in college if I apply myself.
Test Anxiety General reactions to test-taking experiences, including negative thoughts and feelings (e.g., worry, dread). When taking a test, I think about what happens if I don't do well. Before a test, my stomach gets upset.
Social Support Connecting with people and students resources for success Connectedness A general sense of belonging and engagement. I feel connected to my peers. People understand me.
Institutional Support Attitudes about and tendency to seek help from established resources. If I don't understand something in class, I ask the instructor for help. I know how to find out what's expected of me in classes.
Barriers to Success Financial pressures, family responsibilities, conflicting work schedules and limited institutional knowledge. Family pressures make it hard for me to commit to school. People close to me support me going to college.

]

One of the benefits of this online assessment platform is that we can provide immediate scores to both students and advisors, as well as the institution. This allows for effective early interventions.

We talked earlier about the many factors that are considered under the umbrella of noncognitive skills. In the SuccessNavigator™ assessment, we've developed a framework that integrates theory, research and practice. The variables measured here not only indicate success, but are designed to tie directly to the programs and services that already exist on many college campuses.

In this framework you can see the four general skills that outline broad areas, each with more specific subskills that provide further information. First, Academic Skills refer to the organizational and classroom behaviors that help to support academic success. Second, Commitment refers to motivational factors, including a student's drive toward their degree, as well as their institutional commitment. Third, Self-Management deals with students' academically-related stress and self-perception factors, including sensitivity to academic stress, test anxiety and academic self-efficacy. Fourth, Social Support refers to students' social perceptions, including their general connectedness, their perceptions of institutional support and willingness to seek help and barriers to success — which refer to factors outside of academic life that may hinder success.

These factors have been selected not only for their ability to predict success, but their ability to holistically represent the student. Just as there are many reasons why students leave higher education, there are just as many unique paths to success. These skills can help students, faculty and staff first articulate skill-based strengths and weaknesses and develop a plan for future steps toward success.

Again, the SuccessNavigator™ assessment provides three reports that can be used to understand students' psychosocial skills and likelihood for success, both at the individual and aggregate level.

First, the Advisor Report is available, either through email or our online portal, to help guide interactions with students. It provides general and subskill scores, as well as our Success Indices — weighted composites of academic and psychosocial factors that indicate a student's likely academic success and persistence. This Score Report also provides feedback about the student and recommended activities that the advisor can use to focus on areas that might require additional development.

The Student Report is provided immediately to the student for download or printing, and looks very similar to the Advisor Report, with the exception of the Success Indices, which are not provided to the student. Once again, the feedback and action plans are tailored toward a student's unique score profile. Institutions can also provide customized information to connect students with specific programs and services in the unique language and terminology of that school.

Finally, the Institutional Report provides general skill, subskill and success index scores at the aggregate level. This allows colleges and universities to determine differences in their population at large.

Let's take a moment to get a closer look at these various Score Reports.

Here is a snapshot of the front page of the Advisor Score Report. Once again, with the exception of the Success Indices, the Student Score Report looks nearly identical.

At the top of the report, you see the student Success Indices. The academic and retention Success Indices are broken into low, medium and high likelihoods of success, and relate to students' likely first semester GPA and persistence, respectively. These indices are formed by optimally weighting students' high school GPA, test scores and SuccessNavigator™ scores to predict each global outcome.

The Course Acceleration Indices are similar, in that they weight key pieces of information, but they are designed to help guide a more targeted decision. As we will discuss later, these indices are designed to be used in concert with existing placement tests when students score within range of a decision point. If a student scores near the cut score for a given course, and the course acceleration index is green, then one can infer that the student has a profile of noncognitive skills that indicate success in a higher level course, and that student should be accelerated. If the indicator is yellow, then the student can still be accelerated, but might require some additional cocurricular supports.

On-Screen: [Course Acceleration Indices showing the following recommendation: Math: Yes (Green) and English: Caution (Yellow)]

This page also contains the general skill scores for this student. As with the Success Indices, the Score Report presents low, medium or high scores in each area so that students and advisors can gain an immediate understanding of a student's skills.

With each score, feedback statements are provided. These represent key behaviors or attitudes that are characteristic of a student with a similar score, thus the feedback is tailored to the individual's score level.

In addition, score-specific action plans are also provided. The Score Report provides recommendations about programs and services that might be beneficial to the student, given their score. The report also contains hyperlinks to supplemental resources. This Resource Library has tools, tips and strategies targeted for both students and advisors that help focus attention on each skill area.

On-Screen: [Skill Report Example:
Academic Skills – Tools and strategies to succeed in the classroom.
A student with similar skills:

Score: Low
Tools/Tips: The Tutoring Center can provide strategies to help students set goals and organize time. Click here for helpful tips and tools.]

The second page of the Score Report provides more specific feedback at the subskill level. Just like page one, there are feedback statements, action plans and links to the Resource Library where students and advisors can find additional activities.

The third page of the Advisor Report provides important background information on the student that can help the advisor gain an even more complete picture of the student. Academic background, intended area of study and students' expectations are captured here, along with other information.

The Institutional Report can be used to aggregate information across student groups. Reports can be run on an entire institution, or filtered based on key characteristics, such as enrollment in a particular major or program. On the first page of this report, you can see aggregate information about the Success Indices, compared to the distribution of all students who have taken the SuccessNavigator™ assessment. Here, one can determine the relative likelihood of success for the group in question.

On-Screen: [Institutional Report Example:

Academic Success Index      
Success Likelihood Definition Percentage of Students Across All Institutions Percentage of Students at Your Institution
High Projected 1st year GPA >3.0 35% <<42%>>
Medium Projected 1st year GPA between 2.0 and 3.0 35% <<31%>>
Low Projected 1st year GPA < 2.0 30% <<27%>>
Retention Success Index      
Success Likelihood Definition Percentage of Students Across All Institutions Percentage of Students at Your Institution
High Probability of retention >85% 35% <<29%>>
Medium Probability of retention between 65% and 85% 35% <<39%>>
Low Probability of retention less than 60% 30% <<32%>>

]

The next page provides aggregate information on the general skills for the report sample. Admittedly, the SuccessNavigator™ assessment focuses on the individual, but by looking at the distribution of scores across an institution or groups of students, colleges and universities can make better decisions about how to organize and target cocurricular programs and resources.

On-screen: [Your Institution's General Skills Scores with skill areas and interventions listed. Academic Skills Data range from 92 – 105 with a mean of 99. Commitment Data range from 101 – 107 with a mean of 104. Self-Management Data range from 93 – 104 with a mean of 98. Social Support Data range from 101 to 107 with a mean of 104.]

The subsequent pages of the Institutional Report provide score distributions for each of the subscores. While the distributions of the general skill score might inform which programs to develop or emphasize, the subskill reports can aid in determining how to focus on content and structure within these programs and services.

On-Screen: [ETS® SuccessNavigator™. We're here to help. For additional guidance, please call 1-800-745-0269 or visit successnavigator.org.

[END]