In this video, we will learn about shrinkage methods.
So we discussed this absurd selection method where we were still using the least court technique.
But on a subset of variables.
In this video, we will learn about techniques where our model will contain all predator variables and
we will try to regularize these estimated coefficient or shrink these estimated coefficient towards
zero.
We will also see how this leads to reduction invariants.
We'll be discussing the two best known techniques for shrinking.
That is delegitimation and the Lassalle.
Let's start with relitigation first.
So in ordinary leased court methode, we minimized.
These are a system which was squared off some of difference between actual value of way and predictively
of late.
Retrogression is similar to ordinarily squared.
Except that the coalition will be estimated by minimizing a slightly different quantity.
This quantity is given by this formula, lamda paint.
Some of.
Squares off all TB does.
So since we will be minimizing this quantity, this whole quantity.
Therefore, we will also be attempting to chinning or reduce the values of these coefficient towards
zero.
This holdem is called shrinkage penalty.
And this lamda is Garlock yawning barometer.
That tuning barometer serves to control legislative impact of these two terms, RSS and Detering Case
Munadi.
Four different values of LAMDA.
Will get different values of B does.
Therefore, selecting a good value of land is critical.
We'll come back to this talk later.
No doubt we are shrinking.
All we does except be does it all because we want to shrink the impact of variables and not the intercept.
Another important thing to know when we are doing reintegration is that since no bit of a loser, part
of the formula which we are trying to minimize the scale of the lives of dependent variables.
That is the X variables.
That ultimate does no.
That is if you have value of one or two predictors and dollars.
And then calculate your beta.
And if it doesn't pound and then you get laid.
We dug in these to be does we're not related directly at the currency exchange rate.
Therefore, retrogression is not scale invariant as well as method is.
But to handle this issue, we will be standardizing the values of predictive variables.
You will not be covering how to standardize the variables mathematically are software packages will
be handling that part for us.
But just remember that before running a digital edition, we need to standardize all the variables.
So how does legislation improve over Lee Squid's?
If you remember our discussion on bias and variance, a less flexible model has more bias, but less
variance.
This is what this additional shrinkage barometer is doing.
It makes the model less flexible.
And as we continue to increase the value of LAMDA, the model continues to become less and less flexible.
So as LAMDA increases our models, bias increases, but at variance decreases.
Now, the decrease of variance is more.
And the increase in bias is less Delta than really of lambda.
At this critical rally of LAMDA, the total letter is minimum.
And it is even less than to the letter of ordinarily squares met her.
If you look at this graph, this green line is showing you the variance.
As you continue to increase LAMDA, this x axis is lamda values.
If you continue to increase lamda value, be part of means guerrera due to variance keeps on decreasing.
And this black line is deep bias.
That continues to increase.
And some of these two values is the total error in the model.
If you look at this pink line.
This point is giving you the minimum added.
And it is at certain value of lambda.
So this shrinkage of coefficients is actually helping you minimize the mean squared error so that it
is even less than the ordinarily squares.
The next technique is called the lasso.
So one major disadvantage of registration is that it will include all predictor variables in the final
model.
The coefficients are sitting towards zero, but they do not become Z2.
This leaves us with a model with B variables which may be less interpretable.
We can overcome this problem by allowing the model.
To shrink devalues ejido.
This is provided by the automated technique called the lasso.
In this technique will be minimizing RSS plus a value.
This value similar to the previous one, but instead of B squared.
Here we are using absolute value of beta.
This small change has the impact of forcing some of the coefficient to become completely zero when the
tuning parameter is sufficiently large.
If coefficients become zero, those variables will not be part of the model.
Therefore, like subcircuit election technique, loss also does variable selection.
Did Izzeldin model is usually more interpretable than the resulting model under reintegration?
If I were to compare LASO and Richard Edition.
In terms of model, indubitably, LASO will be always more interpretable than legitimisation because
it will have less number of labels.
But if I have to compare in terms of prediction, accuracy, there is no universal dominance of one
method or another.
In general, if the response variable is expected to be dependent on a lot of political variables,
then legitimisation is the technique of choice.
But if the response variable is expected to be dependent on less number of critical variables, then
LASO will be the technique of choice.
In practical scenario, because it is easy to run all types of regression models with just a single
line of code.
We run all types of regression models and then we select the one which is giving us the best result
on the test data.