In this lesson we're going to talk about another metric to complement our recall and accuracy metrics.
This metric is called precision or it's also known as positive predictive value.
Precision is defined as the true positives divided by the sum of the true positives and the false positives.
In other words we have a ratio of correctly predicted spam messages to the total number of times we
predicted that an email will span.
Now this equation looks very very similar to recall but instead of using the false negatives we are
using the false positives in the denominator and we also see that looking at this definition a high
precision relates to a low false positive rate.
Remember that in our case a false positive is a non spam message that ends up in the spam folder.
This is why many people think of precision as a measure of exactness.
Precision is a measure of quality let's have another look at our chart and see where those false positives
end up showing up the true positives.
As we've discussed before are in this upper triangle and we've got them here showing up in green.
The false positives on the other hand are the non spam messages that are in this upper triangle and
I've colored them here in Orange and they're sitting alongside the correctly identified true spam messages.
Now having looked at this a stylized slide here let's go back to our Jupiter notebook and look at our
actual data and see if we can identify some of these false positives and false negatives on the actual
chart that we generated.
So if I have a look up here I can see some false negatives already I can see that with these red crosses
showing up in our non spam section we've definitely got some spam emails that manage to inbox and down
here I can actually spot a false positive as well.
This blue dot is classified as spam but in actuality is a non spam email.
But one thing that you already notice is that we have far more spam emails that got classified as non
spam than the other way around.
In a previous lesson we've talked about how an imbalance dataset can give us a misleadingly high accuracy
metric.
We had this hypothetical cancer model that simply labeled everyone as not having cancer.
And given the prevalence of cancer in the UK and the population we calculated that the accuracy would
be around 96 percent but if you think about how recall and precision are calculated what would the recall
metric and the precision metric for this hypothetical model look like.
Have a think about this and I'll tell you the answer shortly.
Well the answer is that we would have a value of zero for precision and we would also have a big fat
zero for recall.
Why.
Well let's take a look at these two equations in our thought experiment.
We actually have no true positives.
There's not a single patient with cancer that was correctly identified.
Therefore both recall and precision are equal to zero.
While the accuracy is still very high and this actually leads me onto my next question for you Do you
think that there is a tradeoff between precision and recall.
In other words if precision goes down does recall go up well let me ask you this.
What happens in our cancer thought experiment where we label everyone as not having cancer but then
somehow we're able to correctly identify a single person who has cancer.
In other words we now predict that no one has cancer except one really unlucky guy.
I'm sorry Mr. Jones I've got terrible news for you.
That kind of stuff.
So what we get in this case
well our precision school in this case would be equal to one because we have one true positive and no
false positive.
We've maxed out precision however our recall is still very very very low because we've missed everyone
else and we have a high false negative rate conversely what if we change it around.
What if instead our model always predicted cancer.
Everybody's got cancer cancer cancer cancer cancer.
This by the way seemed to be the inevitable conclusion.
If I was left to my own devices looking up my symptoms on a Web M.D. or at least that's how I remember
it used to be.
What's the what's our recall score in this case.
What's the recall score.
Where the model always predicts cancer well recall score would actually be maxed out.
It's one we've correctly identified everyone with cancer.
However our precision is very very low and low precision was also the Shepherd's boy problem.
He cried wolf so many times but he only had one true positive and this brings us to why we can't maximize
our recall score simply by labeling every email as spam.
If we label every email as spam our model suffers from low precision.
And conversely if we have fewer false positives then we are not classifying legitimate e-mail as spam.
And we have higher precision.
So is there a tradeoff between precision and recall.
Well the answer is that yes very often there is when you're increasing one you're very often decreasing
the other.
And now that we've talked about the theory let's calculate precision in our naive based model Let's
insert a subheading here called precision school and after that I'd like to pose a challenge for you.
Can you calculate the precision of our naive based model store the result in a variable called precision
underscore score and then print out the precision score as a decimal number and format it so that it
is rounded to three decimal places.
I'll give you a few seconds to pause the video and give this a go.
All right.
So here's the solution.
Precision underscore Gore can't spell.
As usual is equal to the true positives summed up divided by the sum of the true positives summed up
plus the false positives summed up.
And that's equal to our precision.
Just the formula from the slides right.
And printing this out with precision.
School is curly braces colon.
Point three for three decimal places.
No percent sign and then the don't format.
Precision on a score score.
There we go.
And here we have a precision score of zero point nine seven nine.
Not bad right.
Well in the next lesson we're going to talk about how to reconcile the precision and the recall scores
into a single metric.
I hope you're as excited as I am.
See you there.