Hello and welcome back to the course on Machine Learning.
Today we're talking about Bayes Theorem.
Now our main goal for this section is the Naive Bayes classifiers.
But at the same time we can't proceed to them
without talking about Bayes Theorem.
That's why
we're going to have this lovely tutorial devoted specifically to this topic.
So let's kick it off and get straight into it.
Today we're talking about spanners.
I know it's it's a bit weird.
Why are we talking about spanners?
We're supposed to be talking about Bayes theorem,
but I'm going to be illustrating the Bayes theorem on an example.
And we're going to be using spanners just because it's
one of the things I found on the internet.
One of the first pictures that came to mind.
And also that way you'll always remember
whenever somebody says Bayes theorem, you'll remember spanner
and you'll remember what we talked about today.
So it's a good way to anchor this into your memory.
All right. So let's say we're at a factory.
We're doing some analytics for a factory.
And there's two machines, and one machine produces spanners.
And the other machine produces spanners.
Now the machines work at different rates and they have somewhat
different characteristics, but overall they're producing the same spanners.
The additional information here is that the spanners
are actually marked or tagged.
So we know which machine they came from.
So the top ones came from machine one, the bottom ones came from machine two.
And then at the end of the day we've got like a whole pile of these spanners
and the workers go through them and their goal is to pick out
the defective spanners.
So here we can see that
there's a couple of defective spanners hiding among the pile.
Now, the question that we're going to be asking today is what's the probability.
What's the probability of machine two producing a defective spanner.
So if you take just a random spanner and produced by machine two.
So as it comes out of the from the conveyor belt you pick it up.
What is the probability that that spanner is defective.
And the way we'll get to that probability is through some information
that will already be given to us at the start.
So we'll have a look at that information in a second.
But the rule or the mathematical concept that we'll be using to get to
that probability is called the Bayes theorem.
Here's a mathematical representation of it.
I know it is a bit complicated right now.
It seems like what are all these signs like?
We know that P is probably probability but then b a
and the vertical line what what all the all of them doing here.
What is this relationship telling us.
Well don't worry about it for now.
We will walk through it very slowly step by step,
and you'll get to know the Bayes theorem very intimately.
All right, so let's get started.
What is the information that is provided to us at the very start?
Well, we know that machine one produces 30 wrenches
per hour and machine two 24in per hour, by the way.
Wrenches and spanners.
I found out just now that they're actually the same thing.
A spanner is outside of North America.
A wrench is inside of North America.
So there you go.
You learned an extra additional thing today.
If you didn't know that, I learned an extra additional thing.
So we're just going to call them wrenches from now.
Machine one produces 30 wrenches.
Machine two produces 20in per arm.
Okay, great.
Also, out of all of the produce parts, we can see that 1% are defective.
So after checking the parts, at the end of the day, you know,
thousands and thousands of wrenches, we can see that 1% of them are defective.
Okay, so we know that.
Then we also know that out of all of the defective parts,
we can see that 50% came from machine one and 50% came from machine two.
So here you can see that if you take all the defective parts,
only the defective parts, and you calculate
how many of them came from machine one,
and how many of them came from machine two, you'll see that half and half that
half of them are from machine one and half of for machine two.
All right.
So this is just the defective parts.
And the question is what is the probability
there were a part produced by machine two is defective.
So that's the same question which we just asked already.
What is the probability that if I come up to machine two
and take the part that just popped out of machine two,
what is the probability that that part is going to be defective?
So how do we put all that information together to get the answer
to this question?
That's what Bayes Ethereum answers or helps us do.
So let's have a look at how
we can rewrite this information in more mathematical terms.
So the first line what does it tell us.
It tells us that machine one produces 30in per hour machine to 22in per hour.
So in total,
out of all the wrenches produced, there's 50 wrenches produced per hour.
And also that means that the probability of any given wrench
that you pick up from the pile, just any, whether it's defective or non defective,
if you pick up a range from the final pile, the probability
that it'll come from machine one is 30 divided by 50.
Right.
So if we're producing 30 from machine one and 50 overall per hour,
that means likelihood of any given wrench to be from machine one is about
is exactly 30 divided by 50 or 0 point 6 or 60%.
Similarly, the likelihood of that wrench
being from machine two is 20 over 50 or 40%.
All right. Great.
So we're going to write those down and keep them there for now.
Now let's see what we can get out of the second line.
So we can see that out of all the produced parts we can see that 1% of defective.
Well, it's very simple to rewrite this in mathematical terms.
We can just say probability of a part being defective equals 1%. So.
So even though that was a
simple transition actually these sentences are saying a bit different things.
So the one on the left says we can see that 1% are defective.
So that means that we took
we counted the number of defective parts and we divided by the total amount.
And that means that 1% defective on the right.
We're saying the probability of a single part to be defective was 1%.
So that means if I pick up a random part from the pile,
the likelihood of it being defective is 1%, right?
So that is what we're saying here.
And it's a very simple thing to write in mathematical terms.
There it is p defective or that there's a defect equals 1%.
All right.
And now we're moving on to part three.
Out of all of the defective parts.
We can see that 50% came from machine
one and 50% came from machine two.
So how do we write to that in mathematical terms.
And this is the interesting part.
So let's start with the machine. One part.
50% came from machine one meaning that if we take only the defective part.
So if it's given that we're only looking at the defective parts,
then the likelihood that any part
that we pick out that that part came from machine one is 50%.
And the way trying mathematical terms is this way P of machine
one vertical line defect equals 50%.
So the right part over here, the vertical line in mathematical terms means given.
So here you can see part of a part of machine
one means the likelihood or probability of a part coming from machine one.
Here it's same thing but given some condition.
So this is the likelihood of a part coming from machine one
given the condition that that part is defective.
So you know a priori that the part is defective
because you're picking it out of the defective pile.
So what is the probability of that part coming from machine one?
Well, it's 50%.
And that is the mathematical transcription of this sentence.
We can see that 50% came from machine 150% of defective parts.
Okay.
So now let's do the same thing for the machine.
Machine two it's going to look exactly the same.
But here is going to be machine two.
So probability of a part that the part that we just picked up, the probability
that it originally was created by machine two is 50%.
Given that we're only picking out from the defective pile.
Right.
Because if we're picking out out of all of them, it's actually 40%.
It's 0.4.
But if we're picking it out only from the defective file, it's 50%.
So you can really tell from here, from these two expressions
or four expressions, that the likelihood of a part coming from machine to is 40%,
meaning that it produces less, it produces less ranges,
whereas the likelihood of a defective part coming from machine to is 50%.
So that
if you take any part or any branch, you pick it out from the defective file.
The likelihood that it was originally produced by machine to is 50%.
So you can tell right away that machine two seems to be producing
disproportionately more defective parts than machine one, right?
So it's only producing 40% of the output, but it's accounting for 50%
of the defective parts.
And the question that we want to ask is actually a bit different.
Right.
So the question we want to ask is
what is the probability that a part produced by machine two is defective.
So here it's kind of reverse here where first we're saying
we're taking a part from machine two.
Right.
We're only looking at parts that are produced by machine two.
What is the probability of a random part taken out of machine to to be defective.
How do we write that in mathematical terms.
It's written this way.
So probability of a defective part given that it came from machine two.
If you look at just the probability of a part being defective it's 1%.
But here we're given a condition that it's machine two.
So the way to think of it, you can either think of it
as parts of coming out of machine two and you just pick up a random one.
What's the probability of it being defective? Right.
Or you can think of it in terms of quantities.
The correct term here is frequentist interpretation.
So you can think of it in terms of the frequentist interpretation.
And that means instead of just picking up one part and thinking
about what what is the likelihood of it being defective
if we know that it came from machine?
To think of it as there's a pile of wrenches
that only came from machine two, so you have a pile of, you know,
a thousand wrenches that came from machine two.
What is the number of wrenches that are going to be defective?
Right.
So what is a portion of wrenches that are going to be defective?
So those are two different ways of thinking about probabilities
or the Bayes theorem.
But they're pretty much the same thing.
It exactly the same thing.
It just different ways of thinking about it.
So it's either what's the probability of this one part
that just came out of a machine to being defective?
Or what is the portion of all of these parts that came out of a machine
to what is a portion of them that's going to be defective?
So that's the question that we want to answer given all this information.
All right. So let's start solving this problem.
And will illustrate how the Bayes theorem
will help us convert this information to this information.
All right.
So first of all we won't need the probability data for machine one.
And the probability for machine one given defective.
So we're going to
get rid of that out of our information just to clean things up okay.
Now let's just move everything slide everything over to make some space.
And now let's write down the Bayes theorem, which we had a brief
look at before.
And Bayes theorem for this particular problem will look as follows.
So the probability of a defective part, given that it came from machine two.
So probability of a part being defective, given that that part
that that wrench came from machine two equals.
And here here we go. This is the fun part.
So you will take the probability to start from right to left.
We'll take the probability of a part being defective.
Overall we will need to multiply that by the probability
that a part came from machine two given that it was defective.
And we'll need to divide that by the probability
of the part coming from machine two.
All right.
So for now this might seem a bit better already than we saw previously.
At least now we we can read these terms right.
And we, we know how we got them here.
So we actually discussed each one of these terms.
And they're pretty straightforward even though even though
the mathematical representation might look a bit scary at the same time,
they're pretty straightforward terms we know which what we're talking about.
At the same time, we don't really understand where this formula came from.
And, more importantly, on intuitive level,
doesn't really make sense at this stage.
Let's okay,
let's for now, plug in the numbers and then we'll proceed to the
intuitive part of this formula.
So if we plug in the numbers what we'll get is 0.5.
So we'll just take numbers from the top.
Or if we start from right to left again the probability of a part being defective
overall is 1% multiplied
by probability of a part coming from machine two.
Given that we're only looking at defective parts,
or given that that part is known to be defective, it's 50%.
So there it is.
And we're dividing that
by the probability of the part coming from machine to overall.
And that is 40%.
So there we go.
So if we plug that in we get a 0.0125.
So one and a quarter percentage is a 1.25%
is the probability that machine two will spit out a defective part.
Right.
So if you come up to machine two and you pick up a part,
then the probability of that part being defective is 1.25%.
Or the frequentist interpretation is
if machine two produced 1000 parts,
then according to Bayes theorem, 12.5 of them will be defective.
So you can't really say 12.5 wrenches will be defective.
So let's say it produced 10,000 parts.
Then the Bayes theorem tells us that 125 of those wrenches
are going to be defective.
So as you can see, we converted all this information that we knew
about the process and about the results of the process into exactly what we want.
Now let's move on to the intuition, the fun stuff.
Right.
So with this 11. 25%.
So let's move this to the top.
That's our Bayes theorem.
And let's look at an example.
And you will see that it's actually very very intuitive what we just did.
It all makes total sense.
So let's look at an example.
Let's say we produced 1000in all total.
So in total we have a thousand wrenches after the two machines
were working for some time.
So we know that 400 of them came from machine two.
Right.
So we know that machine two produces how many wrenches per hour does it
produce produces when you reach this per hour and machine one produces
30 wrenches per hour as per our information from the very start.
And that means that out of the thousand wrenches, 40% will be
would have been produced by machine two.
That means 400 came from machine two.
Okay, so that makes total sense.
Then we also know that 1% have a defect.
We actually see this the workers who look at the wrenches.
At the end of the day this was also given to us.
They see that 1% have a defect.
And that means out of 1000, 1% is ten.
So ten wrenches have a defect. Okay, great.
And so what's the next step?
Next step is that we know that 50% of those ten came from machine two.
So we know that exactly five of those defective wrenches came from machine two.
That is also given to us
because those defective wrenches actually have labels on them.
And we can tell by the labels that five of them came from machine two.
All right.
So the question is what is it, percent defective parts from machine two.
Well, it's very easy now right.
Because we know that how many defective actually came from machine two five.
And we know how many wrenches came from machine to 400.
Right.
So all we have to do is divide five by 400 and we'll get 1.25.
Right. So we get the same answer.
Not only do we get the same answer, we actually did exactly the same process.
If you think through what we did here, we performed exactly the same process.
We had 1000 wrenches. Right.
And then we turned that into 400.
So we multiplied a thousand wrenches by 40%, which is the probability of a part
come from machine to. So this is our denominator right.
So instead of probability machine two what we have what we just did
this 400 is p of machine two times a thousand right.
So we have p of machine two times a thousand in the denominator
instead of just p of machine to in what we're doing here
right then 1% have a defect that means ten.
So where did this ten come from.
That's 1% times a thousand.
So instead of p defect at the top
we actually have p defect times a thousand right now.
So just visualize that for a second.
So instead of P machine two we have permission two times
a thousand p defect times a thousand.
And then this 50% the probability of them of them 50% came from machine two.
Right. Equals five.
That's this line is actually us performing this operation here.
We're saying the probability of card
coming from machine two, given that it's defective is 50%.
So basically to get to this, what we did is we took this part.
So we took 50%.
We multiply it by the ten which is actually.
So we took 50% multiplied by 1% times a thousand.
So here we've got x times 1000.
And here we've got permission to what we did is
we actually put in p machine two times a thousand.
So we I just added an extra part over here.
Times a thousand and part over here times 1000.
And then we still divided
that final result, that five that we got by the 400 that we got.
So the logical steps that we took are exactly the same as the Bayes theorem.
The only difference is that we looked at a specific example of a thousand wrenches.
So as you can see the Bayes theorem is a very intuitive theorem.
So we're not even going to go into the mathematics of how it's derived.
This is about us understanding it intuitively.
So as long as you remember the mathematical representation of the Bayes
theorem, it actually makes total sense what it is helping us calculate it.
And in the steps that it is implying for us to take to calculate the final result.
So there we go.
Hopefully that all makes sense.
And we only have one final question left.
We only have one obvious question here.
So the question is why do we have to go through all of this complexity?
Why do we have to go through all this complexity?
Why can't we just count the number of defective wrenches from machine two,
and then count the number of total wrenches that came from machine
two and divide one by the other and get that same result.
Why can't we just do that?
Why don't we have that
input of the total number of wrenches that came from machine to right away?
So that's the question.
If the items are labeled, why couldn't we just count the number
of defective wrenches that came from machine two
and divide by the total number that came from machine two?
So if you think about it, that's exactly what we are doing.
So we're dividing the total number of.
So this is the total number of wrenches that came from machine two at the top.
If you multiply by the thousand at the top we've got the number of machines to a.
So defective the probability of a part
coming from machine to if it's given that is defective times
really would be defective.
So at the top as you remember we got that five.
So the top is actually the number of defective parts that came from machine
two at the end of the day we're actually doing that exact thing.
The question is why can we do it right away?
Why couldn't the workers just go and count the number of defective parts
that came from machine two and divided by the number of parts?
That's the overall number of parts that came from Washington.
Why not save some time?
Well, the answer to that question can can be multifold.
First of all, it might be very time consuming like in this example.
Example might be time consuming
to calculate the number of wrenches that just came for a machine to,
for instance, we might know it's might be like a standard metric
that the factory measures how many wrenches produced overall.
Right?
So we know that number 1000 might be 100,000.
But to count them for machine two, it might take them some time
to sit and count through those wrenches. And therefore it's just faster.
Choose Bayes theorem.
The other thing here is that sometimes you
might not have access to that information.
Sometimes the problem might be such that you just cannot perform that.
And you have some inputs like we had in this case, and that's it.
And it's not as simple as in this example.
And therefore you just don't have access to that information.
So there are numerous reasons
why you could be in a situation where you can't just go through
the straightforward, easy, obvious way and you need to best your Bayes theorem.
So therefore it's a useful thing to have in your data science arsenal.
And moreover, now that we know Bayes theorem,
we can proceed to the Naive Bayes classifiers.
And to finish off today's drill, I've got a quick exercise for you.
Perform the same calculation
or similar calculations for this value that we want to calculate.
What is the probability of a part being defective
given that it came from machine one?
So there you go.
It's a handy quick
exercises to solidify this knowledge, and I look forward to seeing an external.
Until then, enjoy machine learning.