Hello, my friends.
All right, let's begin this upper confidence bound implementation.
All right.
So we're going to start efficiently thanks to our data preprocessing template.
Because indeed as you see
the first steps are just to import the libraries and import the data sets.
So let's do this.
Let's go to our data preprocessing template.
Let's get our libraries which we will indeed use
you know in this USB implementation.
And same for Thompson sampling.
And here let's create a new code cell.
And let's paste that right here. Indeed
we will use matplotlib because you know at the end we'll plot the histogram.
And we will use of course pandas to import the data set.
And speaking of importing the data set, well that's our next step here.
So let's actually only take that line of code
because of course for reinforcement learning
we don't have to create a matrix of features or a dependent variable.
So let's create a new code cell and let's paste it right here.
And that of course replace the name of the data set by the real one
which is remember add CTR optimization okay.
So let's do this add underscore city R
underscore optimization okay.
Because we are optimizing the click through rate of the ads okay.
We're trying to maximize the clicks of users to a specific add.
And we will identify what is that specific ad
having the highest click through rate okay good.
So that's it. Now we're going to run these two cells.
But first we need of course to upload the data set.
So I clicked on this folder here.
Right now the notebook is connecting to runtime to enable file browsing.
And also to run the cells here.
And in the second there we go.
We should see the upload button.
So let's click it and then please fine.
The machine learning is that codes and data datasets folder.
Wherever you put it on your machine I put it onto my desktop.
So let's go inside then let's go to part six.
We're past half halfway now. Congratulations.
Part six reinforcement learning.
Then upper confidence bounds.
And there we go. Python.
And we select this data sets.
Make sure also to have this slide open on your machine especially for
the next tutorials where we will,
you know, implement each of the implementation steps.
First you and then us together okay so let's select this and there we go.
Let's click open.
Let's press okay.
And we will have the data set okay. Good.
Let's double click it to make sure we have it correctly.
All right. So we have the ten ads.
And we have you know
well you know you would have a lot of page because you have actually 10,000 users.
So remember each of the rows here corresponds to different users
connecting successively to the web page or wherever we show the ad.
And then for each user we have a one if the user clicks on the ad, or a zero
if the user doesn't click on the ad and I remind that this is a simulation.
We're not supposed to know all these, but the only way we can simulate
the execution of the UCB model and the Thompson sampling
model is to have indeed this data set with the real truth.
Okay.
And I remind, the important thing is that each ad has a different click
through rate.
And the goal of our UCB or Thompson algorithm will be to identify
as fast as possible the ad that has the highest click through rate.
Okay, good. So let's close this.
And now well let's just run the sales starting with this one
important the libraries and then this one importing the data set.
And now my friends we are ready to implement the UCB algorithm.
And of course we'll start fresh in the next tutorial.
Until then
please have a look at the slides you know, to get familiar with the steps.
And make sure you understand them.
And whenever you're ready, let's implement this together.
Until then, enjoy machine learning.