Now we are going to discuss some very powerful prediction models based on tres.
These methods are also called ensemble methods.
Ensemble basically means taking a group of things instead of individual things.
So these are the three matters that we'll be discussing bagging random forest and boosting.
The concept behind using ensemble methods is simple.
The problem with decision trees is that decision trees have high variance.
This means if I split my training data into two halves and train two models using these two sets of
training data, there are two models that you will get of not very different.
So using my movie does it.
I created two parts of the dating dataset and trained a regression tree on both of those.
And I'm getting two different types of revisionaries from those two does.
You can see for this tree, the first cut is made on budget variable at a value of 37000, whereas for
distri the first cut is made for trailered views at a value of.
447 Dolgen.
Similarly, the next no doubt alter different.
Basically, the whole three is different.
So since the output of busy and remodel, that is highly with the training set that we use.
We say that this country has high variant.
So this is the problem that RBC entry has Iberians.
And a natural way to reduce the variance and hence increase the prediction accuracy is to take many
training sets from the population.
Then build a separate prediction model using each training set.
Finally, we average the resulting prediction to good defying prediction.
So you can see here, instead of using one training set.
I used multiple training sets.
Using each other training set, I create different models, just like the model that we created earlier.
Using those different models, I predict the outcome for each of the region.
So suppose 1-3 is saying that?
The predicted value will be 40000.
And the other three using that predicted value will be 30000.
And so on, the final value of my prediction will be the average of all these predictions.
So instead of using one set, we are using a group of training set.
That is why this method is called ensemble method.
And this particular technique in which we are using all the variables but multiple training sets is
called bagging.
This technique is based on the concept that if I have any observations with variance sigma squared,
the variance of mean of these observations will be Sigma squared by N.
Meaning that when I average a set of observations, it reduces the Gideons.
Of course, practically, we do not have access to multiple training sites.
So instead, we use bootstrapping to create multiple samples from a single draining dataset.
Let us understand how bootstrapping helps us create multiple samples using the same leaders as a.
Suppose I have this smaller data set of five numbers, seven nine, five, four, three.
I want to create three samples or three training set from this.
Single Training said.
The method is I randomly choose one number out of these five numbers and added to my sample one.
Then again, choose randomly from these five numbers and put it in my sample one.
I continue this process.
They like it.
Five numbers in my sample.
You can see that my sample has a reputation.
Since.
I am choosing from the five numbers.
Multiple times I am bound to get some numbers.
More than one Paines.
So in my first sample, I missed out on seven, but I got four.
Two times.
Similarly example, do I get seven two things in sample three?
I get nine two times.
If I continue making samples, there will be some samples which will have some number three times or
even four times.
So basically, in bootstrapping, we are just digging out observations from the same training set.
But repetition of observations, we are able to create multiple training sets out of the same single
training.
Now, when we are growing trees in the bagging technique, we allow the trees to grow woodland.
That is, we do not prune the trees.
Hence, each individual tree has high variance, but low bias.
Averaging these trees reduces the variance.
So overall, our prediction accuracy increases.
Everything works in case we have regression trees.
What should we do when we are building classification trees?
I think you know the answer in case of classification trees for a given test observation, we can find
out declasse predicted by all the different trees created using bagged data.
And then we take a majority would that is the overall prediction is the most commonly agreed class among
the predictions by our different three models in the next video.
We will learn how to implement bagging technique in our software.