Another Successful Undergraduate Assessment Users' Conference in the Books
By: Victoria Monaghan, Director of Strategic Initiatives – ETS, Higher Education
ETS partnered with Felician University for our second annual Undergraduate Assessment Users' Conference. The event, held June 1, featured presentations from community colleges and 4-year colleges and universities using ETS assessments. The conference allows attendees to network with colleagues as we share research-based practices in assessment and assessment use in the areas of student success and student learning outcomes in higher education.
The conference has been well-received to date. "I found everything interesting. Because we are up for MSCHE next year, I was focused on Accreditation Best Practices and General Education curriculum and revision. I also found the sessions particularly helpful when presenters were demonstrating how they used ETS products to make data-driven decisions," remarked one participant.
The full-day conference on Felician's Rutherford, N.J., campus featured presentations on everything from driving student success using the SuccessNavigator® assessment, to ideas on how to transition from our Proficiency Profile assessment to the HEIghten® Outcomes Assessment Suites. The conference also discussed what to do with SIR II™ data. Poster presentations on assessment use prepared by our users were on display, and there was a "Tech Bar" where participants could ask detailed questions about working with the ETS assessment platform.
Dr. Scott Evenbeck, president of the Stella and Charles Guttman Community College in Manhattan, kicked off the day with a keynote presentation. If you've ever dreamed about starting your own college, with full permission to "throw out the rules" and start from scratch, then you want to hear Dr. Evenbeck speak. Originally called The New Community College, the newest of the City University of New York's seven colleges opened its doors to 300 students in a building across from Bryant Park in the fall of 2012. Drawing upon research and best practices in enhancing student academic achievement and graduation, the college was designed to promote student success. Dr. Evenbeck detailed both the successes (many) and failures (few) to date and lessons learned along the way. "The opening speaker's address was most engaging and insightful," one participant told us. "(I) appreciated his historical focus on the gathering of ideas that ultimately became part of the Guttman curriculum and culture."
We hope to continue this conference as an annual event, allowing colleges who use ETS assessments to present and share their experiences with their peers. The day not only helps us to increase awareness of the ETS Touchpoint™ portfolio of assessments, but it helps us to gather feedback on our assessments and services and helps to inform future strategies. Equally important is the fact that attendees enjoy the conference and find it both useful and informative. One of the participants told us, "I liked the interaction and the opportunities for networking that I received with my colleagues."Back to Top
Cha-Cha-Change Will Do You Good! Gen Ed Curriculum Revision & Assessment
By: Ross Markle, Senior Assessment Strategist – ETS, Higher Education
One of the primary goals of effective assessment processes is to guide change and improvement. At the 2018 ETS Undergraduate Assessment User's Conference on June 1st — hosted by Felician University— representatives from Lebanon Valley College (LVC) presented a powerful example of how assessment results have not only guided improvement, but accompanied a cultural shift at the college. This shift has not only updated and integrated traditional practices such as general education and the first-year experience, but it has also put student learning and success at the forefront of the college.
LVC is a small, private liberal arts college, located east of Hershey, Pa., enrolling roughly 1,600 undergraduate students. As documented in a 2013 story in The Chronicle of Higher Education, LVC had tackled some high-level challenges to general education in the past as it faced a warning from its regional accreditor. In their presentation, however, LVC representatives Marc Harris, Associate Dean of Academic Affairs, Jessica Ickes, Associate Dean of Academic Affairs & Director of Institutional Research, and Laura Eldred, Assistant Dean and Director of Constellation, shared more recent and ground-level updates around the use of ETS assessments within these larger shifts.
The session focused on three main areas. First, the team discussed a structural shift in LVC's general education and first-year experience. Rebranded as Constellation LVC, the team outlined how it is moving away from a traditional cafeteria model — where students select courses simply to fulfill requirements — to one that focuses on a variety of experiences including courses, writing experiences and discipline-specific strategies that integrate diverse perspectives and articulate the value of a student's education while driving key student learning outcomes (e.g., critical thinking).
Second, the LVC team discussed their involvement in ETS's "Big Challenges" research project, which measures both academic and noncognitive skills over a four-year period. Through ETS assessments such as the HEIghten® Critical Thinking Module and the SuccessNavigator® assessment, LVC hopes to learn how certain student skills, behaviors and dispositions, particularly in the context of this new general education initiative, lead to improved student learning.
Third, the team shared some initial insights from the first year of the project. Focusing on results from the SuccessNavigator assessment, they identified not only that the insights were predictive of success, butalso key areas where students both did and did not demonstrate growth. By seeing how students changed in their first year at LVC, as well as how those skills related to student success, the college hopes to make improvements to the first-year experience that can better support students in the future.
Overall, the presentation was well delivered and well received, showing the capability of assessment data to have impactful change when effectively evaluated and supported within an institution. Moreover, it showed the power of collaboration between an organization such as ETS and a willing partner such as LVC.Back to Top
Outcomes Data — From Collection to Use for Improving Student Learning
By: Javarro Russell, Senior Assessment Strategist – ETS, Higher Education
Our most recent Undergraduate Assessment Users' Conference gave us a chance to present on a topic that is at the crux of the work that we do in ETS College Programs. That topic is Data Use. In our work with institutions, we find that they struggle to move from the collection of data for describing student learning to the use of data for improving student learning. This phenomenon often hinders the progress institutions are making toward understanding their impact on student learning.
The pitfalls that beset institutions can occur at many points throughout the assessment process. However, the most obvious place to address and perhaps circumvent data use issues is at the beginning of the assessment process. This is where institutions articulate the learning outcomes for their students. When these outcomes are expressed, a critical element is often missing. That element is the extent to which students should demonstrate the outcome
Articulating the extent to which students should demonstrate the outcome at the outset suggests that you and your colleagues have thought about how the learning outcome is related to the educational experiences students have prior to when they are assessed. It also means that you and your colleagues have thought about the assessment and its relationship to the learning outcome. (Note: Not all assessments of the same skill are created equal, and no assessment will be a perfect measure of the learning outcome.) Most importantly, you have thought about the student population you serve, and the skills and effort they are likely to bring to your assessment process.
Take the following example:
"As a result of our General Education program, 80 percent of completers of general education credits will demonstrate a Proficient level of Critical Thinking or greater as defined here."
Writing the learning outcome in this form helps notify all involved of the expectations for students. Once this expectation is set, the first inference that can be made after your data has been collected and analyzed is whether students are meeting expectations. If all students are meeting expectations, you can decide whether you should or could increase those expectations. This would be the "low-hanging fruit" in terms of data use. However, if students are not meeting expectations, you can begin to delve further into the data to identify where improvement can feasibly be made within your complex learning environment. For the students who are not meeting expectations, you can identify their similarities in educational experiences and target those experiences for potential improvement. You can also identify common areas of the assessment where students who do not meet expectations are having the most difficulty. From there, you and your colleagues can identify curricular or co-curricular opportunities for students to practice and develop the skill. At ETS, we've made it easier for institutions to do this level of data analysis through the many score reporting options we provide.
Moving your institution toward data use should be an important goal for any institutional researcher or assessment professional. Without appropriate data use, the resources invested in assessment are going to waste. Perhaps taking the time to set appropriate expectations for student performance on your assessments will help nudge your institution toward greater use of data. For more tips on data use contact your assessment advisor or send us a note at firstname.lastname@example.org to arrange a meeting where we can share the best practices we've collected over the years.Back to Top
New Version of the ETS® Major Field Test for Biology Now Available
The new version of the ETS® Major Field Test for Biology is now available. This replaces the old version of the test. If you are in the middle of a longitudinal study and require the old version, it will be available in paper-and-pencil format through December 31, 2018, upon special request.
New HEIghten® Outcomes Assessment Suite Modules Now Available
The HEIghten® Civic Competency & Engagement and Intercultural Competency & Diversity assessments are now available for institutions to purchase and administer. Learn more about the new assessments on the HEIghten website or contact an assessment advisor.
Minimum System Requirements Update
To align with industry best practices for security and data integrity, new minimum system requirements for accessing the Program Workshop portal and for online test administration have been implemented. After June 25, 2018, any browser or operating system that does not meet these new requirements will be blocked from accessing all ETS online assessments and logging into Program Workshop.
Please review the system requirements and take action as soon as possible by running the updated system check on all computers that will be used by test takers, proctors or administrators.
Annual Comparative Data Guides for ETS® Major Field Tests and ETS® Proficiency Profile
The Annual Comparative Data Guides for the ETS® Major Field Tests and the ETS® Proficiency Profile have been updated through July 2018. The guides can be accessed through the Reports folder in Program Workshop.
Learn About Our Recent Demonstration Project or Visit Us at an Upcoming Event
Demonstration Project: How Noncognitive Skills Can Inform Student Success Strategies Within Hispanic Serving Institutions (HSIs)
Hispanic Serving Institutions, or HSIs, represent a critical constituent in the modern higher education ecosystem. According to recent data from Excelencia, 492 colleges and universities are currently designated as HSIs, with an additional 333 institutions classified as "emerging" HSIs with increasing Hispanic enrollments. While the growth of HSIs and Hispanic students has increased in recent years, other data from Excelencia show disparate rates of success for Hispanic students.
At the same time, ETS's work in student success has focused on noncognitive factors and the role they play in academic success, retention and persistence. Understanding that things such as study skills, commitment, self-management and social connections — which fall outside of traditional measures of academic preparation — are central to student success, ETS has worked through its SuccessNavigator® assessment to not only measure these factors, but also help institutions build strategies to better support and guide students through mechanisms such as course placement, advising and student success courses.
This work has had particular resonance within HSIs, as well as other Minority Serving Institutions (MSIs). As such, the SuccessNavigator assessment has been administered in 42 HSIs, 11 Historically Black Colleges and Universities (HBCUs) and 4 Tribal Colleges and Universities (TCUs) — each of which plays a critical role in providing support to traditionally underserved populations.
Because of these similar missions, ETS recently engaged with the Hispanic Association of Colleges and Universities (HACU), which represents HSIs nationally, to collaborate on a project to better understand how noncognitive skills can inform student success strategies within these organizations. In the summer of 2017, three institutions — Valencia College (Orlando, Fla.), California State University, Fullerton and Texas State University — were recruited to participate in a demonstration project to determine how assessment results could be used to improve institutional structure, culture and practice.
Over the last year, these three schools have worked closely with ETS to administer the SuccessNavigator assessment and gather data, as well as to train staff such as advisors and faculty to use assessment results. These institutions have also considered how organizational structures, such as advising and student success courses, could better learn from these data to support students.
The emphasis of the project has certainly been more on understanding practice than engaging in research. For example, the goal has been to understand how these three specific institutions can best benefit from a holistic consideration of students' strengths and challenges rather than to generalize about the noncognitive skills of Hispanic students or students at HSIs in general.
Each institution, along with representatives from ETS, will be working this fall to conclude the demonstration project and share the results. Two sessions at HACU's Annual Conference (October 6–8, 2018 in Atlanta, Ga.) will feature presentations on the project's progress and initial results. Moreover, a case study summarizing the work and lessons learned will be published in the fall of 2018.
Ultimately, this project emphasizes a shift in the work ETS does to support institutions of higher education that has been ongoing for the past several years. While quality assessments are central to the mission and operations of ETS, increasingly, the use of those assessments is just as vital. Thanks to willing partners like HACU, important work such as that around the SuccessNavigator assessment can continue to inform the value and use of the data that assessments provide.
Connect with us at one of these upcoming conferences and/or attend a presentation.
Email us if you'd like to schedule some time to meet during the conference.
October 6–8, 2018
HACU Dean's Forum
October 9, 2018
Ontario Learning Outcomes Assessment Conference
October 11–12, 2018
Toronto, Ontario, Canada
Canadian Institutional Research and Planning Association (CIRPA) Conference
October 21–23, 2018
Calgary, Alberto, Canada
Virginia Assessment Group
November 4–6, 2018
MSCHE Annual Conference
November 28–30, 2018
SACS Annual Meeting
December 8–11, 2018
New Orleans, LA
Doing something wonderful with an ETS assessment? Consider submitting a poster detailing how you are using an ETS assessment on your campus. We will be selecting winners for prizes! For more information, email Victoria Monaghan, Director of Strategic Initiatives.Back to Top