All right.
So in this lesson we're going to start talking about how to evaluate our model.
We've built that naive based classifier from scratch and we've trained it and classified our emails
in our testing dataset but we don't know yet if our model is doing a good job or a bad job.
Speaking of jobs remember that the premise of this whole project is that we're applying for a job at
our favorite company and they've asked all the candidates to complete a machine learning exercise as
part of the application process and to land that job.
We want to be able to show off a good model right.
But here's the catch.
What does good actually mean.
Well if an email was spam then our model should classified a spam right.
And if an email was legitimate then our model should classify this email as non spam.
That's not too much to ask right.
Well I think the first thing to check is how many emails were actually classified correctly.
What is the fraction of the predictions that our model got right in this lesson we'll be looking at
the models accuracy let's add a section heading here that reads Metrics and Evaluation
and as a subheading I'll add accuracy.
This will be the first metric that we're going to look at.
So how many emails that our model classify correctly and how many emails that our model classify incorrectly.
Well the number of correct documents correct.
And it's called docs will be equal to.
Well I'll be equal to where y underscore.
Test is equal to our prediction.
Dot some.
In other words we're taking the true values and we're comparing them with the double equal sign whether
they're equal to our vector of predictions.
This is what we've calculated up here.
And because the result of this comparison will either be true or false summing up all the values which
are true will be the number of correct documents.
So let's print this out.
Print docs classified correctly comma correct docs and yes that's a better variable name.
And let's hit shift into.
So we've got one thousand six hundred and seventy one documents that were classified correctly now suppose
we want to know the number of documents that were classified incorrectly.
I can store this in a variable called say num docs on the score wrong and that would be equal to.
Well it'll be the number of rows the number of entries and X underscore test which we can get with X
and court test dot shape square brackets 0 minus the number of correct documents and that if we printed
out see Doc's classified incorrectly is equal to 52.
Now that seems pretty promising right.
Because the interesting thing about these results is that our model is classifying e-mails that it has
never seen before.
We've trained our model on a completely different data set of e-mails and still out of over a thousand
seven hundred e-mails.
It only misclassified 52 of them.
So now that we've calculated the number of e-mails classified correctly and incorrectly how do we quantify
the model's accuracy.
Well let's look at the definition of the accuracy metric in machine learning.
The mean accuracy on the test data is defined as the number of documents classified correctly over the
total number of documents.
So we've got our number of correct predictions divided by the total number of predictions.
That's what accuracy means in machine learning.
Let's calculate that here for good measure.
So a lot a little common here that reads accuracy and it's gonna be the correct documents divided by
the total number of documents.
In other words the length of X on a school test.
This will give us the same answer as this the number of rows in our testing dataset let's see what it
is I had around 97 percent.
It's actually not all that bad.
Right now no other way to calculate the accuracy is to look at the fraction of e-mails that were classified
incorrectly.
So if I say fraction of a school wrong is equal to num docs wrong divided by alien X on a school test.
That's the fraction of documents that we've got wrong.
And if we print this out say print parentheses fraction classified incorrectly is and I want to provide
this percentage to two decimal places.
I can insert some curly braces here and put a colon in there and then dot and then two and then the
percent sign and then after that single quote for the string I can put a dot then format parentheses
fraction underscore wrong.
So this will print out three point zero two percent and that means if we print out the accuracy of the
model is curly braces colon point two percent dot format one minus fraction.
Wrong.
There we go.
Ninety six point nine eight percent.
Now looking at these numbers is very well and good.
And we're going to be looking at a few more metrics as part of our evaluation process.
But you know what I think we should visualize how our model is doing exactly.
I think we should plotted and create some sort of chart and we're going to do that in the next lesson.
I'll see you there.