How are Universities Predicting Student Outcomes with Better Analytics?

That was the theme of this year’s Advanced Analytics Summit sponsored by the University of Pittsburgh and Othot, Inc. The presenters highlighted the cutting-edge work being conducted at their institutions and raised some important issues. The goal was to provide opportunities for learning and discussions about the issues that confront all institutions as they carry out their analyses.

Keynote speaker, Amelia Parnell, Vice President for Research and Policy at the National Association of Student Personnel Administrators, best summed up the thinking around analytics and data in her slide, “The Promise and Peril of Analytics.” While many institutions are implementing analytics to measure student success and outcomes, she offered some caution about the promise and the peril.

“It’s a tool to use to manage operations. I don’t know if it is 100 percent accurate,” said Parnell. “But it is a tool, and it important not to ignore it.”

Parnell also offered some predictions about advanced analytics at work and navigating the analytics revolution. She commented that analytics could be put to good use to examine a student’s ability to afford college, for training to create new data users, and to leverage data to provide evidence that learning is happening everywhere.

The two-day summit offered case studies, cautions, and counsel, along with some practical advice for institutions using analytics for student outcomes. We’ve pulled together highlights from a selection of the sessions.

Enrollment Strategy

“Why do students select ASU?” asked Blake Slyter, Enrollment Strategy Analyst at Arizona State University.

While a student’s decision to enroll is often nuanced and multifaceted, predictive modeling does not address all possible influencers. Many are ambiguous, like peer influences, a new building or even a personal relationship.

Recruitment models do not drive enrollment; through established correlations, they inform enrollment strategy. It’s important to make that distinction.

An effective recruitment model provides actionable insight at each stage of the funnel. You may be asking different questions of a prospective student versus an admitted student and need a model that can answer questions specific to the different stages of the funnel.

Re-evaluating Your Analytics Toolbox

The University of Texas at Austin (UT Austin) wanted to increase the four-year graduation rate and was not sure if their current model was good enough to meet the goal.

“Could it be better?” said Cassandre Alvarado, Executive Director, Student Success Initiatives at UT Austin.

Defining a ‘better model’ as one that was simpler, adaptable, and more accurate, the team removed some variables, refined the training data, and compared the results between the old model and the new model.

What they found with the ‘better model’ was that there were 259 additional students who could benefit from some interventions to graduate in four years.

Using this 'better’ model, UT Austin increased their four-year graduation rate 33 percent from 2012 to 2018.

Unrestrained Analytics?

Andy Hannah, CEO at Othot; Marc Demarest, CEO and Principal at Noumenal, Inc.

“The machines are coming,” said Marc Demarest, CEO and Principal at Noumenal, Inc.

“Marc often forgets the human nature of what we do in higher education,” replied Andy Hannah, CEO at Othot, Inc.

The two debated about the natural limits (are there any?) of the application of machine learning and AI-based predictive and prescriptive analytics to the student lifecycle, and touched on the ethical implications of those natural limits.

Attendees participated in some quick polls to add some color to the discussion.

According to the polls, 34 percent of attendees are using advanced analytics throughout the student lifecycle and 39 percent are just getting started.

The polls also revealed some of the boundaries or obstacles institutions face, with 41 percent of respondents indicating that the ability to understand the technology and outputs was the top reason.

At the end of the debate, the audience was left with much to consider about what’s happening at their respective universities.

Analytics Can Save Higher Education. Really.

“Analytics alone are the not the solution,” said Christine Keller, Executive Director & CEO at the Association for Institutional Research (AIR). “People make it possible for analytics to result in good decisions.”

Keller’s session brought to light some barriers that institutions face when trying to create a data decision making culture. When thinking about how analytics can be used to impact student success, decision can’t be made with data alone. It requires some data sense making to put it into context for your institution. By translating and interpreting the numbers into a story, you can help people understand what the numbers mean and how to respond.

Beyond the Technical: Establishing a Culture of Inquiry, Engagement, and Transparency

Speakers, Lance Kennedy-Phillips, Vice Provost for Planning, Assessment, Institutional Research, and Karen Vance, Associate Vice Provost for Institutional Research, from Pennsylvania State University (Penn State) and Marsha Lovett, Associate Vice Provost for Teaching Innovation & Learning Analytics, Patience Whitworth, Assistant Vice Provost for Strategic Initiatives and Operations, from Carnegie Mellon University (CMU) share a common belief that people, process and technology are all necessary elements of a data-informed decision culture.

Each shared case studies from their respective institutions about how they are taking steps to create data-informed decision cultures. For CMU, that process was driven by student success initiatives. Lovett and Whitworth shared their framework for the project and offered thoughts on what’s next.

For Penn State, the driver was centralizing the data-informed decision culture. Given the size of the university and multiple branch campuses, several challenges and obstacles were distilled into six key root causes. From there, a mission statement was developed to guide the university on its journey: Trusting Insights Empowering Penn State.

Penn State has established clear protocols and acknowledges that people are the key to a successful data-driven decision culture.

“It’s all about people,” said Vance. “We’re committed to a culture of curiosity and inquiry.”

Final Thoughts

Higher education institutions do not operate in a vacuum. It’s an industry that puts a high value on collaboration and sharing. That was evident at the Advanced Analytics Summit, from the topics to the speakers and the Q&As as to the informal discussions. That culture of collaboration is a contributing factor to the success of the Advanced Analytics Summit, a good reminder of what’s possible, and a glimpse into what’s next.