Now, when we are building a machine learning model.
There are seven steps to it.
It starts with problem formulation.
Then we do data dating, then data preprocessing, then we split the data between test data and training
data, then we build the model and train it.
Once we get the results, we validate the results and take more accuracy.
And lastly, we use the model for prediction and other purposes.
Let's look at them one by one.
So the first thing which is a problem formulation.
It basically means when you have the business problem with you, you have to convert that business problem
into a statistical problem.
Your business problem can be as simple as increasing your business's revenue.
When you go deeper into it, you may identify that the problem that your business is facing is one of
attrition, that your customers are not renewing their membership or stopping the use of your product.
And second is, you're not able to find new prospects.
You don't have new customers.
So this business problem will then be converted into two smaller problems, one of which is decreasing
attrition, and the second one is getting better customer prospect.
So when we have a tradition and we want to minimize it, we first need to identify the variables which
impacted.
So attrition will be the dependent variable and we want to identify the independent variables which
increase or decrease attrition.
These variables can be anything that as technical problems with the product or the customer service
that we provide to our customers or any other business variable.
Once we identify these variables, we need to collect data for all these variables and attrition.
So this step of identifying dependent and independent variables from the business problem is a problem
formulation.
Once we have identified the problem and have gathered the data for that problem, we need to add it
up.
We need to clean that data so that it is usable for data analysis, for data analysis.
The data should be available in a clear table format with rows and columns.
And in each column, the values should clearly represent different variables.
For example, in this column, if you have M 014 representing a male whose age is between zero and 14,
it should not be kept like this.
This data should be converted into these two columns.
Read one column has the variable of customer six, which contains male or female, and the other which
has each categories such as zero to 14, 15 to 24 and so on.
Once you have clean data, you need to pre-process your data before putting it into the model preprocessing
and include filtering data, which means you are removing some particular type of data from all the
data that you have.
Aggregating values that is assigning aggregate values wherever necessary, missing value treatment,
a lot of times some variables are missing, some values.
We need to treat those missing values as per the choice of analysis that we have decided.
We need to treat the outliers as they can harmfully impact their analysis.
Similarly, women want to transform some variables or reduce the total number of variables.
All this constitutes data processing and we will be covering all this in a separate section, because
this is one of the most important part before starting any analysis.
And when you are in data analysis business, almost 80 percent of your time will always go into data
dating and data preprocessing.
So it is very important to know how to get divided up and in the right format.
The fourth step is to split the data that you have in the past data entry into the training data that
we will use to train our algorithm.
This data will include both input data and output data and this is this data the algorithm will learn
what is the relationship between input and output variables?
And the algorithm will be trying to minimize the error of the expected output from the actual value
that is that we have given in the data.
If you remember the example that I showed you earlier, if you have five images of bananas and apples,
we will use four to train the model and we will keep one aside so that once the model is trained, we
will use that one image to test whether our model is predicting correctly or not.
So that one image that we separated is called testing data.
Testing data will have input data.
You will give this input data to our model and the model will predict an expected output.
We will compare this expected output of the model with the actual output that we have with us.
This will help us assess the accuracy of the model that we have created and whether the predictive function
that we have generated using the model is working correctly or not.
Usually when you have a lot of data, around 80 percent of the available data is used as training data
and 20 percent is used as testing data.
The fifth part is training the model, you have the value of dependent variable, you have the value
of independent variables, that you have output and input variables.
You want to estimate this function effects basis.
The previous data, your model will run and estimate the value of this function in this.
So once your model is trained and you have that function, which is the relationship between your output
variable and input variable, you need to check its performance.
The model that we have created will have two types of added.
These are called into and out of the military.
Instability means we provided some training data to our model to train itself.
After training, it may still not be able to predict all the values in the training data correctly.
So it may be giving Eddowes while predicting the values of training data.
That error is called intent.
So suppose we give four images of apples and bananas to our model to train and out of those four images.
The model is predicting three of them correctly.
Then we have 75 percent accuracy and there is twenty five percent in templated.
Whereas now that new model that you have created, if you apply it to a new sample of four new images
and out of those four new images, just predicting only two values correctly, then those two wrongly
predicted values is Aldo's and Balladur.
The last step is to use this prediction model.
Once you have this prediction model, you should set up a pipeline so that your input data constantly
flows into this model.
You get this output in the form of dashboards and tables.
You should also keep on monitoring the output of your model and improve it by adding more training data
to it or incorporating any new variables that you identify.
Also, if your business requires, you can automate this scoring process.