Predicting buying behavior using Machine Learning: A case study on Sales Prospecting (Part I)
Artificial Intelligence (AI) is the new buzz word. We all have heard and read that it will change the world. However, most articles fall short on explaining how exactly AI algorithms can be used to solve real-world problems. This series is my attempt at bridging the gap between technical AI and applications of AI.
For this series, I will restrict to Machine Learning (ML) algorithms which is a section of AI where we let machines learn from data.
My focus will be to explore how ML algorithms can be used to model and predict human buying behavior.
Trending AI Articles:
1. Back-Propagation is very simple. Who made it Complicated?
Human behavior and Economic models: Background
During the Christmas break, I started reading a book called ‘Misbehaving: The making of Behavioral economics’. The author, Richard H. Thaler — winner of Noble prize for economics in 2017, is considered one of the pioneers of behavioral economics. In his book, he gives various examples to show how humans do not behave according to economic models.
An example taken from his book.
‘Normal people do not behave like economical theory. They do not optimize’.
Perhaps that is why no economic model has been successfully built to predict human behavior.
So what is our buying behavior
No one wants to be sold but everyones wants to buy.
Most of our buying decisions are not based on well-defined logic. Emotions, trust, communication skills, culture and intuition plays a big role in our buying decisions.
Machine Learning and Human buying behavior
How can Machine Learning help in modeling and predicting human buying behavior?
The most common approach taken by many ‘AI-based’ sales startups is to identify the next buyer by mining internet data. They look at what people are talking about in social media and then identify those who are searching for a given product or service. However, as pointed out in my article ‘Want to grow sales? Stop cold emailing. Start prospecting.’, people who are already actively looking online are not the best potential buyers (or prospects) to sell to.
Lets try to see how top salespeople identify a prospect?
Top salespeople identify a prospect before he or she goes out and announces publicly that he/she is looking for a product or service. They build relations and identify needs of people, often before the prospects may start looking for a solution.
So can ML algorithms identify needs of prospects without meeting prospects?
Although humans do not follow a well-defined logic, we do have some repeated patterns. We often buy the same things, behave in a similar way and follow similar intuitions. So if we can learn the buyer’s pattern, we may be able to identify the next buyer too!
When we look at ML algorithms, Neural networks are one of the most widely used ML algorithms these days. One of the main reason of having widespread use of Neural Networks is because it can create an approximation of any function. The approximation is based on data, which it learns or is trained with. So neural nets are able to learn similar responses for inputs that are similar in nature.
A detailed explanation is beyond scope of this article but if you are interested to know more about Neural networks, you can read here and here.
How can Neural Networks be used for Sales prospecting (i.e. identifying new customers for your product/service)?
I have pointed out what constitutes a good prospect and sales process in two of my previous articles, click here and here to read more details.
The biggest problem that most New Sales Development Representatives face are: a) identifying a good prospect and b) Building a customized process and pipeline suitable for the prospect.
Note: New vs Old buyer
I must note here that the buying behavior (and sales process) for new and old customers are different. In this and next article, I will focus on New customers — namely called New Sales development for B2B customers. In sales term it is called Sales Prospecting. In Part IV of this series, I will write how Neural Networks can be used to understand buying behavior of existing customers. Such, in particular will be of interest to e-commerce sites. One of the simple Neural Networks that is used for understanding such behavior is word2vec.
So can machines be taught to behave like a top Sales person? Let’s give a shot.
Part I: Identifying your prospect and creating a persona
I ask the following four questions to identify who are ideal prospects (taken from the book ‘New Sales Simplified’ by Mike Weinberg)
• Who are your best customer
• Why they became customers
• Why they still buy from you
• Why do prospects choose you over other similar products
The goal is to identify common features among successful and unsuccessful prospects. Normally this is done manually and intuitively.
If we had to solve the same problem via Machine Learning we need to use Neural Network Classifier.
Classification can be defined as the grouping of things by shared features, characteristics and qualities or if you will simply dropping things into corresponding buckets, you could for instance classify the following geometric shapes based on their similarity. [Reference]
Step 1 Feature extraction:
Based on the four questions mentioned above, we try to extract relevant features from answers of the questions. An example of such features can be as following
Who is your best customer: Customer size, Decision maker, Growth last year
Why they became customers: Location, First reference (personal contact, content marketing etc), Product features(Feature 1, Feature 2)
Why they still buy: Customer service, Location, Product features
Why they choose us over others: First reference, Product features(Feature 1, Feature 2), Location
Step 2 Labeling data: Label the data based on which of the leads took least amount of time to covert, medium time to convert, maximum time to convert and did not convert.
Step 3 Training Neural Network: One labeled, we will use supervised learning algorithm to train a standard Neural Network classifier.
Step 4 Testing Neural Network: In this phase, you test how good the model is with rest to the test data.
Step 4 Executing Neural Network on new data: Once trained any new input with the data will be able to classify into good and bad output. Thus we can input either a person or company data and the Neural network will be able to classify.
Figure 1 (below): Neural network classifier
Part II: Creating a customized Sales process and pipeline
Once you know who can be a good/medium/bad prospect you want to create a customized process for that particular prospect. Top salespeople use intuition and experience to create such a process.
There are two potential algorithms that can be used for this. Long Short-term Memory (LSTM) and Reinforcement Learning.
Option 1: Using LSTM
A sales process can be seen as set of actions done over time. Current action is dependent on what has been done before and what has been the response.
LSTM networks are perfect for that. These are part of the broader class of neural networks called Recurrent Neural Network (RNN).
One of the appeals of RNNs is the idea that they might be able to connect previous information to the present task [Understanding LSTM Networks].
As you can see in Figure 2(below), RNN is a series of connected Neural networks. Picture from here.
However RNN suffer from something called Vanishing Gradient problem. Learning is limited within a region of Neural networks and thus RNNs are not able to learn long-term dependencies. LSTM solves that.
We will go into details of LSTM network in part II of this series.
Option 2: Reinforcement learning
Another interesting Machine Learning algorithm is Reinforcement Learning (RL). Reinforcement learning depicts human way of learning. It is a learning based on real-time feedback and not via training data.
The learning algorithm learns best actions based on rewards and punishments it receives after executing an action in real world. Figure 3(below) shows a basic structure on how reinforcement learning works.
We will go into details of Reinforcement Learning in Part III of this series.
Final conclusion
I do not believe machines can replace salespeople. Machines will aid salespeople and can convert an average salesperson into a top salesperson.
An obvious question many of you will ask, do you need to build all of these algorithms ourself? Obviously not. There are libraries like Tensorflow, Keras etc which you can use to train your model. I will go into details of the code in some other post. This article was to give an overview of how ML algorithms can be used.
However is everything so great? Not really. One common problem is that your model is as good as your data. That is why what most data scientist do is basically filter out the good data from the bad data. Thats a challenge!
I find this problem deeply fascinating and would love to connect with similar people who have similar interest. Feel free to share, like or comment on this article.
Rudradeb recently published a book titled ‘Creating Value with Artificial Intelligence.’ It is available on Amazon and already getting great reviews.
