Having already looked at accuracy as a metric the second metric that we're going to look at is the recall
score.
Recall is also known as the sensitivity the metric is defined as follows.
Recall is the true positives divided by the sum of the true positives and the false negatives.
Now looking at this formula and this definition is very well and good but let's think about a situation
in which the recall score is high and in what situation.
The recall score is low to understand it a little better.
Looking at our fraction here we can see that the number of false negatives is key when this number goes
down say 2 0.
Then the recall score goes up in the extreme.
If the false negatives are equal to zero then the recall score has its maximum value a value of 1.
Now remember from our previous discussion a false negative is when our models said that an email was
not spam when in fact it was spam.
The spammer got into our inbox.
So how do we interpret the recall score.
Well think of recall as answering this question of all the spam emails.
How many of these did we actually label a spam this whole equation as a ratio right.
It's the ratio between the correctly identified spam emails and all the spam emails that were present
in the dataset.
Here's how we can visualize the true positives and the false negatives on our chart.
Remember we had our probability that an email contains certain words given that it is spam on the y
axis and we had the probability that an email has certain words given that the email is non spam on
the x axis.
Our decision boundary separated two regions.
Any data point that falls into the top red triangle here.
Our naive base model will classify as spam and any email that falls into the bottom blue triangle here.
Our model would classify as non spam.
So where are those true positives and those false negatives.
Well plotting all the data points like this the true positives are the ones that are actually spam and
classified correctly.
They would be showing up like so on the other hand the false negatives would be on the other side they
would be spam emails but they would be classified as non spam so what recall is trying to measure is
our models ability to find the data points of interests namely our spam emails.
It's a measure of quantity or completeness.
How many spam e-mails that we get versus how many did we miss.
How let's calculate our recall score in Jupiter notebook.
The first thing I'll do is insert a section heading so called the section heading here.
Recall score and at this point I'm going to throw it over to you.
I'd actually like you to calculate what the recall score is for our youth based model.
So can you store the result in a variable called recall on the school school and then print the value
of the recall score as a percentage rounded to two decimal places.
I'll give you a few seconds to pause the video before I show you the solution
all right.
Here we go.
Recall on the school school is equal to the true positives.
So true underscore P Os dot sum divided by the sum of our true positives.
So to underscore pause dot sum plus falls on a score negative dot sum parentheses and that's it and
we'll print the whole thing.
Recall score is curly braces colon dot 2 percent.
So this is part one of how we format the whole thing as a percentage round to two decimal places and
on the string we'll call the format method and we'll provide recall on a score score as an argument.
Now I've just reopened this notebook actually.
So when I go to sell and run all and there we have it our recall score is ninety three point two percent
brilliant.
But you know the thing is no metric is perfect.
And as we've discussed earlier a weakness of the accuracy metric was when we were dealing with unbalanced
datasets.
But recall actually also has a weakness because there is actually a very very easy way to manipulate
the recall score and maximize it simply by labeling all the emails as spam.
Because if we do that we have no false negatives no mis identified spam emails.
So our recall score would be equal to 1 and we'd have the perfect recall score.
So as you can imagine there exists another metric that we should look at as well.
And this is what we'll look at in the next lesson.
Hey I'll see you there.