In this video, we are starting with the first classified technique, which is logistic regression.
Let us consider again these similar credit default data set that we saw in the last lecture.
So in this, we have three predictive variables studying balance and income.
And as a responsible even we have weathered the person defaulted on the credit or not.
If a good deed defaulted very well as Z2 and one that is, if the person defaulted, then one, and
if not defaulted, then ditto.
And then simple linear regression with just one variable, say balance.
This is what I will get.
So using the values of balance, we are predicting the probability of default basis this line.
Now we can choose any boundary value above which we will classify it as one and below it.
We will classify it as zero.
So suppose I select point to as cutoff value.
This point to will be koelie boundary value.
If you want to be less conservative in predicting who is going to default, we can use larger values.
Say point five as boundary value, then any point which lies beyond point five will be considered as
risky for giving credit.
You can see the problem with this regression model for these small values of balance.
We are getting negative values of probability of default.
How do we interpret these negative values?
There is no explanation for these values toward this problem.
We must model this probability with a function that gives output between zero and one for all values
of X.
And logistic regression, we use the logistic function, which is also called sigmoid function.
This function is called sigmoid function or logistic function.
This is where logistic regression is getting its name from.
You can see that this is the linear part.
Initially we had Y is equal to be does it plus B Dominik's.
That linear relationship has been put into this function to get deployability to understand why we did
this.
Let us look at this graph.
Logistic function has this s shape shaped go and it is bonded between VI is equal to zero and Y is equal
to one.
You can see if I put X is equal to infinity here.
Why is tending to one.
And if I put X is equal to minus infinity here why is tending to zero.
These are sensible limits and the output of this function is better able to capture the range of probabilities
than the linear regression model.
So this is how the final function will look like.
We can also see that we are finding out the probability of Y is equal to one.
Using this logistic function, what I mean is if what X is this point fifteen hundred correspondingly
Y values this same point, one will be saying that there is a 10 percent chance that the person will
default.
Now, keep in mind that since way is not linearly littered with X, that is this is the Y value and
Y is not linearly littered with X.
It is some other function of X.
We cannot interpret B.
And B does it all in the same way as we do in linear regression.
That is, if we increase X by one unit this way will not increase by B one unit.
The relationship is a bit more complex than that.
And we will discuss later how to interpret the result of a logistic regression model.
Now, if you remember from the last lecture.
The third issue with the linear regression classification model is that if we have outlaying point,
the line tends to change its slope and then misclassify most of the point.
But that problem is also handled by the sigmoid function.
If we have an outline point also, this code is automatically changing its slope and covering those
outlying points.
So any point here, if we extend this graph, any point here will also lay on this good.
And this slope will not be changed much.
So this is all this.
Using the sigmoid function, we have solved the issue of having probability values between zero and
one.
And the problem of outlaying data point to know that we have the model that can give us the probability
values using this formula.
We need to get the values of this.
B does it.
And we do one such that we have minimum error or maximum correctness and prediction.
In linear regression model, we use a method called or Neddie Lee Square Meter and logistic regression.
The general method used is called maximum likelihood method.
The intuition behind it is simple.
Let us go back to the example to understand how maximum likelihood is applied when a person is defaulting.
The default value is one.
The predicted probability, which is B X should be as close to one in such a case.
And when the person is not defaulting, the predicted probability should be as close to zero in such
a case.
So we want to maximize B X if the person is defaulting and we want to maximize one minus because if
the person is not defaulting.
So this likelihood function is to be maximized, given these two conditions, maximum likely, maximum
likelihood metter is just maximizing this small condition beyond this formula.
I will not show you any more mathematics because mathematics does get a bit complex.
And frankly, as business managers, you will never need more mathematics than this.
Our software packages can easily fit this model.
And so we do not need to concern ourselves with details of this fitting procedure.
However, remember this part, that lake and linear regression we used, or Mary Lee Square in logistic
regression.
We use maximum likelihood, better to estimate decode fishings and maximum likelihood.
Materne is a very popular approach used in many classification techniques.
And then next we do we will learn how to train a logistic model on a dataset using only one predictability.