Hello and welcome back.
Today we're talking about the confusion matrix and different accuracy ratios.
So let's imagine that we are building a model
that will predict based on images X-ray images of lungs
whether there is cancer or not in the lungs.
So we're going to build a matrix here on the left.
On the top we have the prediction of our model.
And it can be negative no cancer or positive.
Yeah there is cancer.
And then on the left we have the actual state of things is there.
is it negative?
There is no cancer and in reality there is no cancer.
that person doesn't have cancer or positive.
That person does have cancer.
So and once we cross these rows and columns will have four different cells.
The top left one is called the true negative.
and by the way, before we go into this,
the positioning of these cells depends on the source.
Yet you are looking at some,
places draw confusion matrix one way, other places other way.
So and I'll link to a, an article about this at the end of this tutorial.
So back to back to our true negative.
So the cross between negative and negative is a true negative
meaning the model predicted that there is no cancer.
And in reality that person doesn't have cancer.
So in this use case, it's a great result, for the person that, in question.
Now the bottom right is called the true positive.
The model predicted that this person has cancer.
And in reality, they also have cancer.
And while this is not
a great result at all for the person in question, in this model,
at least they know that they have cancer and they can now address it
with their a doctor and potentially
hopefully get better, get treatment.
Now in the top right we have something that's called a false positive.
The model predicts that the person has cancer, but in reality
they don't have cancer.
And this is called a type one error.
And this is a problem because even though for the person
in question, it's a relief that they don't have cancer.
Imagine the emotional stress and suffering that they will go through
when the when they're told by the model that they have cancer,
even though they don't have to go
through the stress and suffering because they don't actually have any cancer.
so it would be much better if the model told them the correct answer.
made the correct prediction that they don't have cancer.
So it would be much better if the model gave them a true negative.
And then in the bottom left we have a false negative, which is a type
two error where the person actually does have cancer in in reality.
But the model says they don't.
And this is a very dangerous type of error because, in this use case,
the doctors won't even treat the person won't even recommend any treatment plan
because they'll think
that the person doesn't have cancer and the cancer can grow and get worse.
So both errors are not great.
And we want to avoid them.
So the less errors our models make, the better.
Now let's populate this matrix with actual figures.
Let's say we have 100 patients out of them.
our model made some predictions and we have 43 true negatives, 41 true
positives.
12 the type one errors or false positives
and four, type two errors or false negatives.
From this confusion matrix, we can calculate the following rates or ratios.
We have the accuracy rate and the error rate.
The accuracy rate is the total number of correct predictions,
meaning the true negatives plus the true positives divided
by the total number of patients in this sample.
So we have 84 divide 100 or 84%.
And the error rate is the total number of incorrect predictions,
meaning the type one errors plus the type two errors.
we have
16 of those divided by the total sample size 100.
So we have 16% error rate here.
So those are two, important rates to be able to calculate.
So that's how the confusion matrix matrix works.
And as additional reading, I highly recommend checking out
this article to, have it handy for future use
where the person investigated, the different
forms of confusion matrix because, different tools, Python
or any other tool that you might be using can, produce
a different confusion matrix is better, is good to read through it.
So you always know which confusion matrix you're dealing with in specific instance
and in the tutorial.
In all practical terms, you'll be dealing with the one that we discussed today
and the one that's pictured here on the bottom right.
So I hope you enjoyed this tutorial.
I'll see you next time. Until then, enjoy machine learning.