ML Education: How Pearson Abstracted Away Rookie AI Headaches
In organizations that successfully embrace AI, going on gut feel is going the way of the dodo. So is the phrase: “because that’s the way we’ve always done it.” Where there’s machine learning, automated business processes that amount to paved-over cow paths run into the rigors of predictive models.
Pearson Education Management’s online learning services unit, which supports dozens of colleges and universities, for decades has helped students navigate their educational journeys. Early this year, Pearson embarked on an AI journey to improve their insights into online college students and how colleges can better interact with them. Pearson’s experience offers strategic lessons for companies implementing enterprise AI for the first time.
In part, Pearson’s mission is to help recruit students to its client colleges and guide them through the challenges of the college curriculum. A core goal is minimization of student attrition – i.e., supporting students through to successfully earning their degrees. With the advent of machine learning, Pearson realized it possessed a trove of student behavioral data that could, potentially, be more aggressively leveraged for predictive insights.
“We partner with universities to launch and scale online programs,” said Pearson VP of Analytics and Innovation Daniel Goldsmith. “We help strategize with institutions regarding programs they should bring to market to meet new student demands, or to provide differentiated offerings, or reach learner sub-segments, like adult learners, or students coming back to finish their college degrees.”
In making the move to ML, Goldsmith said he was “primarily interested in how we could use analytics and AI to better understand the nature of student demand for education and help institutions meet learner needs both during the enrollment selection process and during their academic experience. So that meant: How do we use the large data sets we had?”
Pearson IT managers were aware of the agonies many organizations undergo getting ML off the ground and scaling it beyond the prototype phase. So Goldsmith and his team pursued a strategy amounting to a double head start for their data scientists: Pearson partnered with a consulting firm, Atrium, Bozeman, MT, with machine learning experience, and Atrium, in turn, is practiced at using Salesforce’s Einstein AI, a machine learning platform built into the Salesforce Customer 360 CRM system.
According to Atrium CEO Chris Heineken, Einstein abstracts away much of the normally tortuous work of integrating ML insights within an organization’s larger business systems.
“We use Einstein when we can for modeling requirements … around linear logistic regressions,” he said. “…We like Einstein because it’s able to take an insight and plug it into the business community to act on, Salesforce has that nailed for functionality. So we talk about actionable frameworks and how you take an insight and weave it into the business. That’s a huge stumbling block for AI initiatives and one that Salesforce addresses in a really powerful way.”
If another “AI winter” takes place, a commonly anticipated cause is that too many organizations lack the technical skills to make machine learning magic happen. In an article currently running on this site (along with sister publication Datanami), consultant Ben Bloch of Bloch Strategy, Los Angeles cited industry studies showing that “roughly 25 percent of companies have seen half of their AI projects fail. Failure, in heavily technical deployments, like AI projects, is incredibly expensive when data scientist and other team time, technical cost of computation, and resources wasted is accounted for.”
A key to boosting the likelihood of AI success: automation and integration of what often are stumbling blocks critical to project outcomes. Einstein’s integration within the Salesforce CRM, Heineken said, helps get predictive models “out of the laboratory.”
“How do you take the meaningful insights from the model that might have been done in a custom infrastructure somewhere else, integrate that within the data of a CRM system like SF?” he asked. “Figuring out the integration connectivity of that is non-trivial. Figuring out the security around that is non-trivial, and then figuring out how you present that in a format that (users) can consume and act upon, again, is not trivial. So with Salesforce, the way the infrastructure is set up has a way to help people get through those gates in a much more streamlined way.”
Last January, Pearson began working with Atrium as part of the organization’s larger effort to adopt the Salesforce CRM.
“One unique thing about this engagement,” said Goldsmith, “it came in the middle of a migration onto the Salesforce platform. We front-loaded the analytics pieces, Einstein analytics, with Atrium.” He said Atrium and Pearson data scientists worked together to build models using Einstein sandbox version. “We got that in the hands of our academic clients, and in doing that work it helped us better understand how to craft the data environment and help drive the Salesforce migration plan downstream.”
Goldsmith said the application utilizes lead scoring capabilities planted at points throughout the student journey – starting with the initial contact with the client university – to give Pearson better “understanding of which students are most likely to apply to which types of academic programs, which students are most likely to enroll in those programs, and once they start, what are the risk factors associated with that.”
The application also helps Pearson leverage Salesforce and Einstein “to power workflows to help students navigate that decision making process, understanding which types of materials or communications channels were most helpful for that student for engagement during that process,” said Goldsmith.
Once students have begun in their course work, the system identifies at-risk students, “if we see them not engaged in courses, or if they come in with a high degree of risk factors for having a challenging academic experience, we’ll be sure to get them the mentoring and tutoring support,” he said.
Formerly, Pearson used basic progression modeling techniques to analyze students along with “a lot of gut instinct,” Goldsmith said. Instead of replicating that process in the Salesforce operating environment, Goldsmith said Atrium “helped us step back and take a machine learning approach, so while we had some intuition of what those factors (student behaviors leading to success or failure) were, Atrium helped us set up machine learning to look over the hundreds of factors that we had.”
This ranges from students’ lead sources to the pattern of interactions they had during the enrollment process, how often they had a phone call with an enrollment advisor – “each point of that student funnel. It’s more of a dynamic lead scoring approach to help us understand what factors we could influence and what was happening at different points in the student journey.”
Heineken said Atrium’s engagement with Pearson, which lasted roughly four months and involved four Atrium consultants, unfolded in three stages.
The first order of business, he said, is to develop the right question, or set of questions, for the machine learning application to work against. In the case of Pearson, the questions were: How can we improve the enrollment conversion rate? And: How can we lower the student attrition rate?
By contrast, Heineken said, many ML projects fail because the wrong questions are asked. This can be because the question – the analysis pursued – lacks large-scale impacts for the business, thus being of limited value; or there exists insufficient data to support predictive models and are therefore unquantifiable; or the question fails to produce actions that can be scaled for large user audiences.
Some questions, he said are bad because they’re too general, such as: How can AI grow my revenue? “This question is open ended and shapeless to the point that predictive models cannot objectively help,” he said. “The metrics need to be more granular.”
The second order of business: Pearson adopted what Heineken called a “data-centric mindset,” “so they could take a look at their data to figure out if there’s a signal there around student attrition and student enrollment” that can be drawn out and leveraged.
The third stage: generating insights that are actionable. As mentioned, with Einstein integrated into Salesforce, “you can land the insight into the workflow of the user populations to go act on it.”
The end result is system that’s enabled Pearson to shift from a general to an individual understanding of students.
“We’re moving from a one-size-fits-all model to a really customized approach, understand that student as an individual persona and understand the drivers of that individual student.” Goldsmith said. The system has “catapulted us into a future state, so rather than saying how do we port in traditional understanding, or how do we flesh that out a little bit, now that we have a better data environment…, it’s a fundamentally new approach to understanding student behavior at scale.”