Using the SuccessNavigator™ Assessment for Institutional Planning for Success

People in this video

Narrators/commentators as noted below in text.

Narrator – Ross Markle, Senior Research and Assessment Advisor for the Educational Testing Service

 

Transcript Body

On-screen: [Using the SuccessNavigator™ Assessment for Institutional Planning for Success. ETS® SuccessNavigator™]

Narrator – Ross Markle: At its core, the SuccessNavigator™ assessment is a tool that focuses on the individual student. Understanding unique profiles of student characteristics, improving a student's course placement and building a customized plan for student success — these are all indicators that the SuccessNavigator™ assessment was designed and built to help each and every student at the individual level.

However, colleges and universities can also benefit from looking at the scores of larger groups of students. Whether we're looking at the unique populations of a single institution, or gathering data on a cohort of students, either for a cocurricular program or service or for a particular section of a course, these aggregate scores can allow us to understand larger groups and tailor the way that we plan and implement programs, services and activities. I'm now going to present to you some sections of the SuccessNavigator™ Institutional Report and how that can be used in the contexts of institutional planning and student success courses.

Institutions put forth a great deal of effort, time, money and resources to develop cocurricular programs and services. Interestingly, these efforts are often designed to address the very skills measured by the SuccessNavigator™ assessment — time management, study skills, academic goals, self-management and social connections. Although rooted in quote "best practices," colleges and universities rarely have real data to tailor their cocurricular supports to the unique needs of their population.

By aggregating SuccessNavigator™ scores across groups, faculty, staff and administrators can make better decisions about how to work with their population as a whole, or with targeted subgroups, including traditionally disadvantaged populations, specific programs or majors or a section of a class. Moreover, just as with individual students, these reports address the skills and behaviors that are most directly tied to success.

Here is an excerpt from an Institutional Report, focusing on the general skills across one institution. Let's focus on a few key pieces of information.

On-screen: [Your Institution's General Skills Scores with skill areas and interventions listed. The academic skills are behaviors, beliefs and skills that directly facilitate academic success. Data range from 79 – 102 with a mean of 91. The area of communication includes commitment to drive toward and perceived importance of academic success. Data range from 90 – 114 with a mean of 105. Under Self-Management are the ability to anticipate and respond to pressure and stress relate to college life. Data range from 90 – 107 with a mean of 98. Under Social Support is availability for resources to support academic success. Data range from 93 to 108 with a mean of 103.]

First, all SuccessNavigator™ scores have a mean of 100 and a standard deviation of 15. This conveniently allows any user to immediately determine whether a score is high or low, and whether a score difference is large or small.

The white ovals in each column represent the median score, with 50 percent of students scoring above that level, and 50 percent of students scoring below that level. The numbers at the top of each gray box represent the score at the 75th percentile for that group of students, and the numbers at the bottom represent the score at the 25th percentile. We represent scores in this way because, even though we're trying to aggregate information across groups of students, it's always important to remember that there can be a wide array of scores across that group.

Now we can look at actual scores to try and make inferences about ways to best serve our population. For example, in this case, we can see that Self-Management and Social Support both have median scores near the population mean of 100 — thus, our students are about average. For Commitment, our group median is at 105, one-third of a standard deviation above the mean, indicating slightly higher scores. At the same time, Academic Skills scores are somewhat low, with a group median at 91, nearly two-thirds of a standard deviation below the mean.

As an institution, I would likely want to consider ways in which I can use this information to plan cocurricular programs and services. I may want to focus efforts more on academic skills, given lower student scores in that area. I may elect to focus fewer resources or activities on Commitment given higher scores in that area. It is this type of data-based decision making that can be extremely beneficial in tailoring programs and services to an institution's student population.

On-screen: [

Academic Skills Commitment Self-Management Social support
Accessibility Resource Center
College enrichments & Outreach Programs
Advisement
Academic coaching
Academic level student support
Faculty office hours
Career Services
Academic departments
Student advising
Alumni mentoring program
Student employment
Residential life
Student Health and Counseling Center
Targeted resource centers
Religious College Enrichment and Outreach Programs
Academic Coaching
Recreational Services
Targeted Resource Centers
Student activities
College Enrichment and Outreach Programs
Dean of Students
Student Employment
Recreational Services

]

The next step is understanding how existing programs and services align to each of these areas. Here is a sample of work taken from one of our partner institutions. They went through the exercise of taking an inventory of all their cocurricular programs and services, and aligning them to the SuccessNavigator™ general skills. Had they received a Score Report similar to the one we just discussed, they may want to increase the availability of faculty office hours, or promote academic coaching to incoming students.

If you recall, I mentioned earlier how the SuccessNavigator™ assessment's framework of general skills and subskills is designed to align to existing programs and services. Conducting an alignment such as this is a very achievable goal, and allows the institution to better strategize which programs and services it offers and how it can funnel students who need them to the appropriate resources.

Institutional Reports also contain score information at the subskill level. Once an institution decides to focus on one of the general skill areas, determining which resources require attention, the subskill scores can inform how to structure those resources. Whereas the general skills might be valuable for institutional planning, subskill information might be more helpful for program planning.

In building an institution-wide holistic agenda for student success, it's not only important to consider which programs and services should be offered and how they will be structured, but we must understand exactly how students will access resources. Here, you can see a few strategies for engaging students identified by one of our partner institutions. They made a commitment to use the SuccessNavigator™ assessment as part of their early alert efforts, identifying students with low likelihood of success early in their college careers.

On-Screen: [Driving Student Success through Holistic Assessment: Data-Driven Decision Making

Early Alert Coaching and Advising Targeted Student Populations
  • Identify early-risk
  • Develop strategies for intervention
  • Provide programming to increase persistence
  • 1-on-1 support and guidance
  • Resources to navigate through college
  • Connecting students to on-campus resources and programming inside and outside the classroom
  • Develop profiles of noncognitive scores based on race/ethnicity and gender
  • Develop interventions that specifically align to those skills and profiles

]

They also decided to focus coaching and advising efforts across the spectrum of student success, using this as a key lever to direct students to the appropriate resources.

Finally, programs that deal with targeted student populations, such as cultural centers or programs for first-generation college students, were an important way to access groups with historically low rates of success. These are just a few examples, but they demonstrate the importance of developing multiple ways to engage students outside of the classroom.

One valuable way that the SuccessNavigator™ assessment can be used to work with groups of students is through first-year experience or student success courses. Using it in this context is advantageous for both students and institutions in several ways.

First, FYE courses provide a platform through which institutions can access a large number of students and encourage or require them to complete the assessment, addressing issues of response bias or sampling.

Second, the course context provides dedicated time for FYE faculty and students to review their scores, feedback and action plans. Whereas advisors might have difficulty in accessing their students or finding enough time to thoroughly discuss issues, the FYE course provides a dedicated place and time to do so.

Third, institutions with which we've worked have found the SuccessNavigator™ constructs very useful in structuring course content. After all, these psychosocial skills and factors such as time management, help seeking and goal setting are often the intended focus of student success courses.

Lastly, using aggregate Score Reports at the class level can help tailor class time and activities if a group of students all share a particular need. The Resource Library can even be used as a supplement to existing course materials.

In conclusion, the SuccessNavigator™ assessment helps improve student success efforts by providing information about the whole student — not just standardized test scores or background information, but real data on the psychosocial skills and behaviors that research has shown to be critical to student's classroom performance and persistence to a degree.

The test has been designed to facilitate student advising and to improve the quality of course placement decisions. These are two critical areas for improving success that touch nearly every student in higher education.

But this is just part of a larger issue — success does not end with accurate course placement, quality advising, and early intervention. Ensuring student success is a long process that requires thoughtful consideration by each college and university. We hope that the SuccessNavigator™ framework will help facilitate conversations about key factors on your campus. Using data on individual students, as well as larger groups, the SuccessNavigator™ assessment can play a key role in developing an effective student success agenda.

Thank you for joining us today. We hope the information presented here has been helpful to you and ultimately the students you are advising. If you have any additional questions concerning the SuccessNavigator™ assessment, please contact ETS® customer service to learn more.

On-Screen: [ETS® SuccessNavigator™. We're here to help. For additional guidance, please call 1-800-745-0269 or visit successnavigator.org.

[END]