In this video, we will discuss some of the terms which are used to characterize the performance of
classify it.
So we saw how to create a confusion, matrix and confusion matrix on one side.
We had the true values on the other.
We had predicted values.
And the cross-section of these four classes gave us this confusion matrix and this confusion matrix.
We first assigned the names of Drew, negative and false, positive, false, negative and true positive,
depending on two things.
Negative is always what they predicted.
One.
So and they predicted if it is negative, we'll call it negative.
If it is positive, we'll call it positive.
This true and false is whether we were correct or whether we were wrong.
So it is true negative because predicted was negative and actual was also negative.
So it is true and negative.
This is true positive because predicted was positive and actually was also positive.
So it is true and positive.
The other two are false positive and false negative.
The total true negatives we are denoting here with capital and the total true positives we are dead
denoting ahead with capital P.
The predicted total negatives had denoting with and start and predicted positives.
We had the northing red p stuff out there, several items which can be used to characterize the performance.
This table will help you avoid confusion in all these terms.
The first term is called false positive rate.
It has two other names.
A VERNETTA and one minus specificity.
It is calculated by F.P. by.
And that is false positive.
Divided by.
And so let me give you an example.
To understand all these performance measures.
So suppose you are the manager of this real estate company and you want to use such a classifier to
tell you whether you should choose to pick any particular property to sell or not.
Why are you thinking of using a classifier, say when you pick up property, you will assign an agent
to the property.
You will do marketing for that property and do various other activities to get it sold.
You will incur cost for all these activities.
Now, if that property does not sell, you have sunk cost.
And that will be real money lost.
So from your classifier, you want that whenever it predicts that the house will be sold, it should
be accurate.
Most other times.
So.
And you think which performance measure will that be?
We want whenever my classifier is predicting positive, that is out of these beest us, I want to see
the accuracy.
That is proof positive.
So BP by Pista is one of the measures that I am interested in.
There should be high for my classify it.
That is known as precision BP by P stuff.
In other words, if P by Peter is the amount of sunk cost that you are incurring.
You do your classify it the other way is out of how many times it was not sold.
Your model saved you by saying that it will not be sold.
So that is the end by end, one minus D and by N will be F.P. by N.
That is default postulate being by N.
Is called specificity.
That is how many times your model saved you from the sun cost.
So that measure is specificity.
On the other hand, if you decide not to pick up property under this sort easily, someone else owns
commission that you could have on this is opportunity cost.
That is, you will not lose the deal money, but you lose the opportunity to earn money.
So can you guess which performance measure will give that this was the total opportunity of earning
money?
This.
Although this.
How many times you actually on BP to T.P, BP is giving you the amount of times you actually earned
from the available opportunity, whereas this F in BP is the opportunity cost you incurred to BP by
BP.
Is the performance measured here, which is known as sensitivity.
So all these performance measures have some business meaning inside them and correspondingly, there
are some costs incurred for the inefficiencies of your model.
So depending on which cost is high and wages low, you have to select the corresponding performance
measure and choose the appropriate classifier.
Another common barometer, often stated, while specifying the performance of a classifier is 80 under
the curve of auto CECO.
So this good is known as auto siecle.
The name I don't see is historic and it comes from the communications theory.
It is an acronym for a receiver operating characteristics to create this article on the X axis.
We use this false positive ID that is one minus specificity.
And on the Y axis, we use this positive.
That is sensitivity.
So on the Y axis, we have this sensitivity on x axis.
We have false positive ID or one minus specificity.
And we draw this code.
This code is called Orosco.
This bend of the globe should be as close to this top left corner of the glass fir to be good.
That is the best classifier will have this point very near to this top corner of.
In other words, the area under this code should be as close to the area of this whole square for this
classified to work perfectly.
If this goal is very close to the straight line.
The performance of this classified is not very good.
And it is almost as similar to a random classified.
So whenever we want to compare performance of different models, we can use the auto cycle.
We will specify the area under cover of auto cycle.
That is a U.S. of RC.
Whichever AUC is highest that classify, it will be considered the best it.
So for best scenario, we are discovers exactly following these two axes.
The U.S. value will be one that is the area under the cover will with nearly equal to this whole squared,
which is one into one.
That is one.
For a random classifier, you see, value is nearly half.
So it will have a curve like the straight line.
And the area under the cover will be half.