All right.
So in this lesson it's where all the pieces are gonna come together.
Conditional Probability joint probability the independence assumption and Bayes theorem.
Now I highly encourage you to watch the probability explainer video in the previous module if you haven't
done so already.
But as a quick recap to calculate the joint probability of two events happening say A and B we simply
multiply the probability of eight times the probability of B and the reason this formula is so simplistic
is because we're making a strong assumption here.
We're assuming the events are independent in the case of our spam classifier.
We're calculating the probabilities of these tokens right.
We're calculating the probability that an email is spam given that it contains a particular token.
So if the email contains two tokens say the word Viagra and the word free.
Then we calculate the joint probability simply by multiplying the two conditional probabilities together
the probability that the whole email spam is simply the product of the probability that the email is
spam.
Given that it contains the word for Agora times the probability that the email is spam given that it
contains the word free.
So if you had a sentence like this.
Hello friend.
Want free via agora.
Then the probability that this email is spam is the probability of multiplying all these conditional
probabilities together now.
So far what we've been doing is we've been working out a lot of these numbers that are on the right
hand side of this equation here.
We've been counting the occurrences of our tokens and we've been working out all these colored terms
and we've saved all these calculations in the text files that we've imported into this notebook.
The thing that we haven't done though is multiply all these values together for all the tokens and that's
what we're gonna do now.
So if we just had one term say the word Niagara then the formula for Bayes theorem is pretty straightforward.
It's what we've discussed previously the probability that an e-mail is spam given that it contains the
word Niagara is equal to the probability of Viagra occurring given that the email is spam divided by
the probability of Niagara occurring in any email spam or not spam times the probability of receiving
a spam email.
Now what have we had a whole array of tokens instead of a single token for the word via Gra.
What if we're dealing with our ex on a school test X on a school test.
If you recall looks something like this hits and no higher rate with over a thousand rows over a thousand
columns and it's got integer values for all the tokens in our emails.
How do we adapt Bayes theorem to deal with all of these tokens at the same time.
Thinking back to a distant math class on linear algebra we can adapt this formula here for all our tokens
like so.
So the probability then e-mail is spam.
Given that it contains a set of tokens will be equal to the same terms that we have above for one token
but multiplied by our array of tokens we have to multiply our probabilities by our X underscored test.
Pi array and the technique that we're going to use is called the dot product the dot product.
Simply one way of multiplying two matrices and it's the dot product that will help us multiply our X
and this code test number higher rates with our probability number higher rates.
Let me quickly show you how the dot product works in practice.
Before we implemented on our project so let me add a quick markdown so here that reads calculating the
joint problem ability.
I am also going to add a little subheading here that reads The Doctor product.
So I'll start with a toy example to show you how this works.
See we have to dump higher rates the first arrays called a s and p dot array parentheses square brackets.
Let's give three values C one two three.
Yeah and b it's also going to be an NPR array.
And it's also gonna have three values c 0 5 and 4.
So both A and B are one dimensional.
In other words they have a rank of 1 and you can think of A and B inspectors.
Just a single column of values.
So if you print these out print parentheses single quotes A is equal to any single quote comma a and
print single quotes B equal to and single quote comma b the dot product between A and B would be a dot
and then the word dot d o t parentheses B and the result of that is equal to twenty two.
Now the reason I can do this is because none pi arrays have this dot product method built in so you
can simply put the name of the array which in our case is a put a dot after it and call this method.
And the argument we supply is our second all right.
Now why do we get twenty two.
Why did we get this number.
Let me show you how we arrived at this value.
We simply multiplied the first entry and a.
So that's 1 times the first entry in array B which was 0 and then we added the product of the second
entry in array times.
The second entry in array B.
So two times five.
And then we added you guessed it the product of the third entry in array.
So three times for the third entry in array B.
And the result of this calculation is twenty two.
So that's pretty straightforward right.
But what if we had something a little bit more complex.
What if we had a two dimensional array that we're dealing with.
I'm going to be very imaginative with my variable naming here and call it c is equal to and P array
parentheses square brackets and then another set of square brackets and I'll put in c 0 and 6 and then
I'll add a comma add another pair of square brackets and c put 3 and 0 and then another pair square
brackets after the comma and put 5 and 1.
So here we've got an array of two dimensions and this array has three rows and two columns.
Let me print out the uh the shape of C is and then comma C dot shape.
And for good measure I'll also print out C itself.
This is what it looks like an empire.
Here is three rows two columns and you can see it printed below.
Now what would you guess is gonna be the dot product of a and c and also what s gonna be the shape of
this dot product.
What's gonna be the shape of the result.
Well let's find out shall we print out a dot that duty parentheses see and I also print out the shape
of the dot product is in single quotes comma a dot dot parentheses see Dot shape check it out.
OK so in this calculation we get two values twenty one and nine.
That's pretty interesting right.
Let me show you how to work this out manually.
So our first vampire rate has three values.
It's one dimensional just one two three.
And our second now Perry has two columns with these values.
Now to record this lesson often zooming in a little bit here.
And every time I've got unsafe changes and save those changes I've got this thing bouncing around here.
So I want to go to view and toggle header to hide this and get a bit more screen real estate and less
of this bouncing around.
So the way we can manually work out this 21 a nine is by multiplying the first value in a which was
1 times the first value of the first column in C which was 0 and then added to that is the product of
the second value in a which was two times the second value of that first column in C which was three
and added to that you guessed it it'll be the third value in a which was three times the third value
in the first column in C which was five three times five.
And it's gonna be a similar pattern for that second column L B one times six plus two times zero plus
three times one.
And the result is 21 and nine.
Now as a challenge can you figure out the dimensions of the dot product between X underscore test and
prob and it's got token on a score spam.
I'll give you a few seconds to pause the video and give this a go ready.
Here's the solution.
Now the shape of X and the square test was 1723 three rows and two thousand five hundred columns two
thousand five hundred was the size of our vocabulary and one thousand seven hundred and twenty three
was the number of emails in our testing dataset.
Now prob underscore token unskilled spam dot shape had of course two thousand and five hundred entries
one entry for each token for each word in our vocabulary.
So if I print out the shape of the dot product
this is gonna be X on a school test.
Dot dot dot prob I just got token and it's got a spam dot shape and the result is one thousand seven
hundred and twenty three.