And this video we will discuss about different types of subsects election techniques, as I told you
earlier, in subject election.
We will use a subset of the people to variables instead of using all of them.
But how do we identify the subset?
There are three main ways of doing that.
The first is called best subject selection.
Second is forward step way selection.
And the third is backwards to place selection.
We will discuss each of these one by one.
In best subject to likin method.
We further separately squared regression for each combination of DEPI predictors.
For example, suppose we have tea predictor variables.
First step is to run the model with no predictive variable, which will basically be done the mean value
of the response variable.
Next week on the model with one variable.
Since we have three variables, we have to run this model three times.
Then we will run the model with a combination of two variables.
And lastly, we run with all the three variables.
We will then look at all these darling models to indicate which one is the best.
So since 40 variables will have to rise to what possible combinations and comparing all of these together
at the end may be cumbersome.
The process is usually divided into these three steps.
As I told you first, we will run Dinello model, which will have no predictive variables.
Let us take an example, suppose for that house pricing data.
We take three predictive variables.
One will be room them.
One is air quality.
And third is teacher ratio.
Now, the first step is to run the model.
That is to estimate how Osprey's, without using any predicted this model, will estimate the mean value
of the house every day.
Next step is to don this model with one prettier variable.
So since we had three Briatore variables, we'll have to write three things.
So first model will be house price will be predicted by only against them.
The second model will have house pricing against air quality and third model will have House pricing
against each official.
Once we run all these three models, we will select the best model.
That is the model which is giving the largest R-squared amongst these.
And we will keep that model and say, what does M1, because it had one variable.
So it is M1.
Then we will select two predictive variables to predict the value for space.
So we will use both room NUM and beta ratio to predict the house price room number and air quality stability,
osprey's air quality and beta ratio to the whole space.
So all these three models will be done.
And the best out of these three models will be Sivas as M2 because it has to predict the variables.
Then we'll run it with all three variables which will receive as M3.
Now we have M0, M1, M2 and M3 amongst these four models.
We need to pick the best model that we will select by taking the adjusted R-squared value of all these
models.
The model with highest value of adjusted R-squared will be selected as the best model.
And this third step, we are using a district, R-squared, because if you use Osgood.
You probably remember R-squared monotonically increases as we increase the number of predictors and
entry has the maximum number of predictors.
So if you use R-squared.
We will always end up selecting the empty model.
Two, although this MTV model will always have the lowest rating ever.
But it may not have the lowest.
Tested it.
Therefore, that is why we will be using adjusted R Square, which will be handing us that.
Which of these models has the lowest of.
So when we are working on a software package, all these steps have been in the background.
You'll just get these out of it.
You do not need to do do perform all these steps.
Individually, but it is important because the result also comes in this format.
When we run it for our dataset, which has 16 variables, it will give us all these 16 steps and its
results individually.
So we'll be able to understand that dessert only if you know how we got the dessert.
So all the best subsect election is simple and conceptually appealing.
It involves a large amount of computation and may be limited by a computational limit.
Imagine if we have these equal to 20.
That is, there are two individuals in the model.
Then we need to run the regression model over a million times.
Therefore, we need some computationally efficient alternative to this best subset selection method.
Let us look at those.
So this method is called forward step base election.
It is a computationally efficient alternative to best subsect election.
Instead of going through all possible to do this two Part B models, it considers a much smaller set
of models.
It starts with gains equal to zero.
That is, there is no greater variable than it adds one variable at a time until all predictors are
in the model.
Let us again take the example of three predicted variables in the first step.
We have no predictors.
That is the normal.
Then we consider a case where we have one variable, so we will run the model three times since we had
three variables.
Now, at the end of learning, these three models will be selecting the one which has the highest R-squared.
In the next step.
Instead of learning all the possible combinations of two variables.
We'll keep that one selected variable and only add one variable, which is from the remaining two variables.
Two out of three variables, we selected one variable in the first step, in the second step.
We will select only one variable from the remaining two variables.
This time demotivate, we don't do things because we have two variables remaining.
And again, we will select the best model, which is called M2.
And lastly, we'll have all the three variables, which will be called M3.
Again, we will compare all these MGD and one M2 M3 using the adjusted R-squared.
If I generalize my example of three variables, four be variables.
I've done it one time when I have selected no evil.
When I select one variable, I have to write it beatings.
When there's an activity Abels I put on a B minus one being.
And so on.
Delacey like B minus one variables, and I ran it one time.
So if I add all of this, this is the total number of models that I will get.
It is one plus being two people, one by two.
So you can clearly see that there is computational advantage over, at best, split election.
Well, we were running it to this, to, what, B times?
So if I'm competing for the model, which is to integrity, was this substance election needed to work
or a million times?
But forward step way selection method will just run 200 living teams.
But what does it cost?
That we are paying for the reduction in computation?
The cost is the loss of guarantee that the final model we get from the forward selection will be the
best possible method.
One instance in that example of three predictors, suppose our model M1 has X1 selected.
Now, M2 can only have X1 and X2 or X1 and X3.
If the best possible solution was X2 and X3, it will be missed by this approach.
So by losing the guarantee that we will get the best solution to the problem.
We had considerably reducing the computational efforts of our software package.
The next technique is similar to forwards to base election.
It is called backward step base election.
So only instead of starting with zero predictors, we will start with all predictors.
And then we will remove predictors one by one till we have removed all of them.
So the number of model runs for backward step for this election will also be one plus being two plus
one by two.
And this also, we will lose the guarantee that it will give us the best model.
However, there is one limitation of backward step place selection.
That is, if number of observations is less than number of variables.
So suppose we have hundred variables and number of observations is less than hundred.
This method that is backwards step plays election and even best objets election will not work.
Only forewords election method is a viable election method into destitution.
So these are these absurd election techniques commonly used.
There are few others also like hybrid approach.
But we will not discuss them head.
Just one last thing.
When we were comparing models, having different number of predictors, and I told you that we should
use adjusted R-squared in such a case.
You can also use to test set error instead.
You know, we can split did it in two tested and rounded up.
We can train the model using the training data and then use the test data to finalize the best model
out of M0, M1, M2 and so on.
This can work even better than looking at R-squared.
And that's all for this election.