3 Common Mistakes That Can Derail Your Team’s Predictive Analytics Efforts

With today’s high demand for data scientists and the high salaries that they command, it’s often not practical for companies to keep them on staff. Instead, many organizations work to ramp up their existing staff’s analytics skills, including predictive analytics. But organizations need to proceed with caution. Predictive analytics is especially easy to get wrong. Here are the first three “don’ts” your team needs to learn, and their corresponding remedies.

1) Don’t Fall for Buzzwords — Clarify Your Objective

You know the Joe Jackson song, “You Can’t Get What You Want (Till You Know What You Want)”? Turn it on and let it be your mantra. As fashionable as it is, “data science” is not a business objective or a learning objective in and of itself. This buzzword means nothing more specific than “some clever use of data.” It doesn’t necessarily refer to any particular technology, method, or value proposition. Rather, it alludes to a culture — one of smart people doing creative things to find value in their data. It’s important for everyone to keep this top of mind when learning to work with data.

Under the wide umbrella of data science sits predictive analytics, which delivers the most actionable win you can get from data. In a nutshell, predictive analytics is technology that learns from experience (data) to predict the future behavior of individuals in order to drive better decisions. Prediction is the Holy Grail for more effectively executing mass scale operations in marketing, financial risk, fraud detection, and beyond. Predictive analytics empowers your organization to optimize these functions by flagging who’s most likely to click, buy, lie, die, commit fraud, quit their job, or cancel their subscription — and, beyond predicting people, by also foretelling the most likely outcomes for individual corporate clients and financial instruments. These predictions directly inform the action to take with each individual, e.g., by marketing to those most likely to buy and auditing those most likely to commit fraud.

In their application to these business functions, predictive analytics and machine learning (ML) are synonyms (in other arenas, machine learning also extends to tasks such as facial recognition that aren’t usually called predictive analytics). Machine learning is key to prediction. The accumulation of patterns or formulas ML derives (learns) from the data — known as a predictive model — serves to consider a unique situation and put odds on the outcome. For example, the model could take as input everything currently known about an individual customer and produce as output the probability that that individual will cancel their subscription.

When you begin to deploy predictive analytics with your team, you’re embarking upon a new kind of value proposition, and so it requires a new kind of leadership process. You’ll need some team members to become “machine learning leaders” or “predictive analytics managers” — which signify much more specific skill sets than the catch-all “data scientist,” a title that’s guilty of vagueries and overhype (but, do allow them that title if they like, as long as you’re on the same page).

2) Don’t Lead with Software Selection — Team Skills Come First

In 2011, Thomas Davenport was kind enough to keynote at the conference I founded, Predictive Analytics World. “It’s not about the math — it’s about the people!” he absolutely bellowed at our captivated audience, more loudly than I’d ever heard since high school, when teachers had to get control of a classroom of teens.

Tom’s startling tone struck just the right note (a high D flat, to be exact). Analytics vendors will tell you their software is The Solution. But the solution to what? The problem at hand is to optimize your large-scale operations. And the solution is a new way of business that integrates machine learning. So, a machine learning tool only serves a small part of what must be a holistic organizational process.

Rather than following a vendor’s lead, prepare your staff to manage machine learning integration as an enterprise endeavor, and then allow your staff to determine a more informed choice of analytics software during a later stage of the project.

3) Don’t Leap to the Number Crunching — Strategically Plan the Deployment

The most common mistake that derails predictive analytics projects is to jump into the machine learning before establishing a path to operational deployment. Predictive analytics isn’t a technology you simply buy and plug in. It’s an organizational paradigm that must bridge the quant/business culture gap by way of a collaborative process guided jointly by strategic, operational, and analytical stakeholders.

Each predictive analytics project follows a relatively standard, established series of steps that begins first with establishing how it will be deployed by your business and then works backwards to see what you need to predict and what data you need to predict it, as follows:

Establish the business objective — how the predictive model will be integrated in order to actively make a positive impact on existing operations, such as by more effectively targeting customer retention marketing campaigns.
Define a specific prediction objective to serve the business objective, for which you must have buy-in from business stakeholders — such as marketing staff, who must be willing to change their targeting accordingly. Here’s an example: “Which current customers with a tenure of at least one year and who have purchased more than $500 to date will cancel within three months and not rejoin for another three months thereafter?” In practice, business tactics and pragmatic constraints will often mean the prediction objective must be even more specifically defined than that.
Prepare the training data that machine learning will operate on. This can be a significant bottleneck, generally expected to require 80% of the project’s hands-on workload. It’s a database-programming task, by which your existing data in its current form is rejiggered for the needs of machine learning software.
Apply machine learning to generate the predictive model. This is the “rocket science” part, but it isn’t the most time-intensive. It’s the stage where the choice of analytics tool counts — but, initially, software options may be tried out and compared with free evaluation licenses before then making a decision about which one to buy (or which free open source tool to use).
Deploy the model. Integrate its predictions into existing operations. For example, target a retention campaign to the top 5% of customers for whom an affirmative answer to the “will the customer cancel” question defined in (ii) is most probable.

There are two things you should know about these steps before selecting training options for your predictive analytics leaders. First, these five steps involve extensive backtracking and iteration. For example, only by executing step (iii) might it become clear there isn’t sufficient data for the prediction objective established in step (ii), in which case it must be revisited and modified.

Second, at least for your first pilot projects, you’ll need to bring in an external machine learning consultant for key parts of the process. Normally, your staff shouldn’t endeavor to immediately become autonomous hands-on practitioners of the core machine learning, i.e., step (iv). While it’s important for project leaders to learn the fundamental principles behind how the technology works — in order to understand both its data requirements and the meaning of the predictive probabilities it outputs — a quantitative expert with prior predictive analytics projects in his or her portfolio should step in for step (iv), and also help guide steps (ii) and (iii). This can be a relatively light engagement that keeps the overall project cost-effective, since you’ll still internally execute the most time-intensive steps.

Good luck, and happy predicting.