Till now, we have been discussing the regression trees.
That is, we were trying to predict a continuous quantitative variable like collections of a movie.
Now we are going to discuss classification trees in which we will try to predict categorical variables,
such as whether the movie will win an award or not.
Although the output will look similar.
There are few minor differences in the model at the back end.
First of all, at the leaf node integration trees, we find the mean of the response, we are able to
get the predicted value.
But for classification trees, we'll be using mode.
That is, we assign that class to the region, which is the most commonly occurring class in that region.
For example, if we are classifying whether us to rent will pass or fail.
This is a number of hours.
That student has to read.
We had a population of 10 student.
In this tends to end, there are five to eight words to read.
More than five hours and five different words to read, less than five words out of these five, two
things.
Four passed and the other group of five students, only two passed.
So in the first region since four out of five past will predict that students and this region will pass
the exam and the second region since only two out of five passed and three students fail, will say
that students belonging to this region will fail.
So instead of using mean to predict.
The outcome in that particular region, we will use more to predict the outcome.
The process of growing declassification is similar to growing integration three.
We use because of binary splitting here also.
However, integration breeze, we chose that split, which gave us minimum RSS.
In classification three, obviously, we cannot use Artosis.
So there are several other possible criterion for making dispute.
One natural law logical criteria is classification error rate.
Let me explain the steps that will be involved in this process.
First, we'll consider all the variables and all possible split values.
After each bullet for each region, we will assign it last to that region, as we saw that we assigned
a class of pass to this region and we assign a class of field to this region.
In this way, we will assign a class to each region.
Then we will simply find out diffraction fraction of training observations in that region that do not
belong to the most common class.
So in this region, one observation in which the student field does not match with the predicted value,
which is pass for this region.
And those students who passed in this region.
Do not match with the predicted outcome of this region.
We just feel so three students out of the total, 10 students are misclassified.
So classification error rate in this situation will be 30 percent.
So for each variable and each split, we'll find out declassification at a rate, whichever split is
giving us the minimum classification error rate.
We will keep that split.
However, it turns out that classification error rate is not sufficiently sensitive for regrowing.
So in practice, we have two other measures which are more prevert.
One is Gini Index, which is depicted by this formula.
Let me explain the intuition behind Deani Index.
Suppose we are classifying into two classes.
Boss entry.
Now, this peaslee probability of bias in a particular region or at a particular node.
If the value of bees is very small, say zero.
So the north is very beautiful.
That is, it has most of the observations belonging to only one class and Gini index values small.
The other scenario is if programs need large.
What extremes and how you say it is one, then all the obligations belong to past category.
Again, there is high purity at this node and the value of Gini index, again, comes out to be zero
because here we'll get one.
Minus one is equal to zero in this term.
You can see that a small value of Gini index is a negative note purity, that is, it indicates that
a particular north contains predominantly observations of a single class only.
And then there are the natives, cross and droopy.
We just got using this formula.
It is numerically similar to Gini index, and it also takes small value when naughties built.
So in practice, when we are building a classification three Deani index or crossing and droopy are
preffered.
As these are more sensitive to North purity.