Hello and welcome to this art tutorial.
So in the previous tutorial we implemented Thomson something from scratch.
And now time for the moment we've all been waiting for.
Let's see if Thomson sampling can beat UCB.
So in fact we are ready to execute
this code section here and find out about the final result.
So let's remember the random selection gave us a total reward of 1200 on average.
The UCB algorithm gave us a total reward of 2178.
And now let's see if Thompson sampling can beat that.
Now let's select everything from here to the top.
Because, you know we haven't imported the data set.
So we will execute everything all at once to immediately
get this final result that we were so excited to find out about.
So ready I'm going to press Command plus enter to execute.
And let's see who is the big winner.
Here we go.
And it turns out to be Thompson Sampling
because we got a total reward of 2602.
So we have some random factor.
So let's not scream victory yet.
We are going to execute that again to see the new total reward.
Will we get 2600. Almost.
We can do that again.
And basically it's averaging around 2006 hundred.
So yes definitely it's beating the upper confidence bound algorithm.
And by the way remember that with the UCB algorithm
we almost doubled the total reward of the random selection algorithm.
But now with Thompson sampling we're not only beating the UCB algorithm,
but also we are doing better than doubling the random selection total reward
because we get this 2600 total reward on average,
which is more than the double of 1200.
That was the total reward of the random selection algorithm on average.
So great.
Definitely. Thompson sampling is the big winner.
And now we have last thing to check.
You know remember we need to check that Thompson sampling
also gives us the best ad that has the highest conversion rate.
You know, on which the users of the social network would click the most.
And so we need to make sure that it's also the ad version
number five, which was the adverse and found by the UCB algorithm.
And to check that out very efficiently, we can select this code section here
and execute to look at the histogram.
And here we go.
We also get that the ad version that was most selected is ad version number five.
And by the way in UCB we had some higher bias here if I remember correctly.
But here with Thompson sampling we can clearly see that
this was this ad version number five here that was most selected.
You know, this bar here corresponding to the ad version number
five is clearly dominating the other boys.
And that's because
Thompson sampling quickly figured out which ad is the best to select.
That is it.
Quickly figured out which ad has the best click through rate.
And so now we can congratulate ourselves because we clearly solved
very efficiently this click through rate optimization problem.
And the best algorithm that we found for this is Thompson sampling.
All right.
So congratulations for having implemented these two
algorithms UCB and Thompson sampling.
That's the end of this section.
And that's also the end of this part reinforcement learning.
So I look forward to seeing you in the next part.
Natural language processing.
Until then, enjoy machine learning.