Hello my
friends, and welcome to this new practical activity
which will be, you know, an exciting one because after implementing the UCB
algorithm, we're about implement a new one and we will see if this new one,
which is Thompson Sampling, will beat the UCB.
So I just want to say here that it's important that you check out
first the UCB section, you know, and the practical activity
before doing this one because we will work on the same data set.
And I will mention several times the UCB so that we can compare
the two performance.
All right.
So make sure to check out first the section on upper Confidence bound.
And if that's the case you ready to go okay.
And then let's make sure everyone here is on the same page
I give you the link to this folder.
Machine learning
is that code and data sets right before this tutorial in the article.
So make sure to connect to it.
And now all good we can start. So let's do this.
I can't wait to not only implement that new model Thompson sampling,
but also to see if we're going to beat the UCB mini,
if we're going to manage to catch that best ad in 500 rounds or less.
Okay.
So once again, I give you the slide of the whole Thompson sampling algorithm.
So I really recommend that you open it.
You have the three steps which we will implement together
in the following tutorials.
And once again you will implement the steps on your own
before we do it together.
Because that's the best way to practice and progress in machine learning.
So that's the slide.
And now we're going to go into the Python folder
in which you're going to find two files.
Well the exact same data set after 10,000 rounds.
Or you know the 10,000 users in the rows and then ten ads in the columns.
And so I remind that successively we're going to show ads to each of these users.
And because this data set is a simulation, well,
we know for each of them on which ad they're going to click.
So for example, this user is going to click only on this ad at number eight.
All right.
So that's the same data set.
And now here is the implementation Thompson sampling in the Ipynb format.
So if you're ready let's open it in either Google Colaboratory or Jupyter Notebook.
And I will show you what we're going to do with it.
We won't have actually to re-implement it from scratch
because it is actually very similar to the UCB implementation
in the sense that we have the same data preprocessing,
the same beginning of implementation when implementing the model
in the cell, and the same code for the histogram in the end.
So we will actually only reimplement a part of this cell.
You know, where we have indeed
to implement those three steps of Thompson sampling.
Okay.
So but first of all, as usual this notebook is in read only mode.
That's because you all have access to it.
We can't modify it of course.
Therefore we're going to create a copy here by clicking file.
And then save a copy in drive.
And this will create a copy.
And that's where we will re-implement part
of this implementation where we have something different than before.
With UCB.
All right.
So while speaking of this let's see where we have that difference okay.
So first of all you see that we have the exact same structure as before.
You know we first import the libraries.
Then we import the data set.
Then we implement the Thompson sampling algorithm.
And finally we visualize the results in the histogram.
I'm not going to click on it because we will keep the surprise for the end.
But there you go.
That's the exact same structure.
Now not only this is the exact same structure, but here this is the same code
here as well.
This is the same code.
You know, we imported do it the exact same way.
And here well here we import a different library which is the random library.
And that's because, you know, we will have to work with beta distributions.
You know, when we take a random draw from the beta distributions.
Well, we do this with this random library instead of the math library
in the UCB implementations, which by the way, I've kept here because,
you know, at the end we will compare the two results of Thompson sampling and UCB.
So there you go. We import this library.
Then we have the exact same parameters here.
That's the total number of rounds.
Or you know, the total number of users to whom we show successively the ads.
Then this is the number of ads.
You know, we have ten ads among which we want to find the best one,
you know, the one with the highest conversion rate.
And about that, I insist to remind that there is this important assumption
in the data set that each ad has a fixed conversion rate.
And the goal of our implementation, you know, the goal of what we're doing
here, you know, reinforcement learning with online learning.
Well, the goal is to find that ad having the highest conversion rate.
All right.
So so far exactly the same.
Just that library changes.
You know we're using a different library.
Then here is the same.
We make this variable to prepare.
You know the list of all the ads that will be selected over the rounds.
And we initialized that list as an empty list.
So exactly the same.
Now this is where things change.
So actually we will delete everything from here.
You know we will delete all this cell.
You know all the part of the cell from here.
And we will re-implement all this because all the rest is the same
and same for this histogram.
You know, this is the exact same code as in the UCB.
Basically everything here is the exact same code as in the UCB,
except for this line where we import different library and only what we're
about to implement right now is specific to Thompson sampling.
So that's why I told you to really study first the practical activity of UCB.
If you did not do it already,
because indeed we're going to start from here
instead of re-implementing everything we already did.
So there you go, my friends. Now we're ready to start.
So join me in the next tutorial to implement Thompson Sampling.
And then in another tutorial after that, we will of course visualize
the final result and compare them with UCB.
I can't wait to start!
See you in the next tutorial and until then, enjoy machine learning!