In this section, we will cover the most important part of data analysis.
This is called data pre processing.
As I told you in the introduction to Machine Learning, Section four, practicing data analysts, nearly
80 percent of their time is spent in data processing.
And how well your model will perform eventually depends on how well you have prepared your data.
So let's start with the first thing you should know when you are trying to solve a problem.
You should know about the business context of that problem.
You should know what all factors are important and have impact on the variables of interest.
Only if we know the business, we will be able to identify relevant variables and gather required data
for the analysis.
Remember this.
The quality of your input will decide quality of your output.
Now, there are two primary ways to gather information about business.
One is to gather it yourself by talking to relevant people or stakeholders of that business.
People who are impacted by the problem at hand, or you can also gather information by actually doing
the things yourself.
For example, if you have to increase sales of a particular product.
Tried to go and sell it yourself in the market.
Go and try to buy it, also to understand customers perspective.
Gathering information by yourself is called doing primary research.
In secondary research, you read or listen to the information gathered by others.
Maybe there is some industry related research report or some study done by some consulting firm or someone
else has worked on the same problem before.
We should look at methodology and findings of those earlier works.
Also.
That is an example.
Suppose you're part of a company which is online.
And the problem you are facing is off guard abandonment.
That is a lot of your customers add product to their card, but do not actually putties it.
This is a fairly common problem for online businesses.
Now, if you are given the task to tackle this problem, where do you start?
First, we should go and talk with the relevant teams involved.
Usually there is a marketing team and a product team which cross-functional handle the online sales
process.
We should go and talk to them.
Then we should try choosing from this site ourselves also to understand what is the customer journey
online.
All this will be primarily search.
As part of the cavity search, we should look for previous studies on gut abandonment within and outside
the organization.
Once we have done all this, we would have a fairly good idea of what all variables and what type of
model we are going to use.
Next task is to get Dee dee dum.