Hello my friends, and
welcome to this new part on association rule learning.
In this part, we're going to cover a new way of learning relationships.
You know, correlations.
And this time what we will learn is some association rules.
You know, like that famous statement people who but this also
but that you know,
that's what Amazon, for example, does in its recommendation system.
It predicts what customers will buy based on what they bought before.
And that's why you see
all these suggestions of new products when you buy a certain product.
Well, and this new part, I'm going to teach you how to make a model
that can do these kind of things. All right.
And that is called associate rule learning.
So it's very different actually than what we had before.
You know, before we were either
predicting a dependent variable and we knew what to predict.
We also did clustering where we learned some patterns in the data
so as to create a new dependent variable a posteriori.
And now what we're going to learn
is some association rules inside and in symbol of transactions.
All right.
So very useful especially for retail businesses or any kind of e-commerce.
So if you are a data scientist working for an e-commerce company
you will definitely use it.
And if you are yourself, you know, a business
owner of a retail company or an e-commerce company,
then you will definitely benefit from it as well.
All right.
So let's do this.
Let's start to learn this new technique.
And before we start, let's just make sure everyone here is on the same page.
This is a folder containing all the codes and data sets of this course.
And I give you the link to this folder right before this tutorial.
So make sure not to miss it.
Click it.
And now we should all be on the same page.
All right so let's do this.
Let's get into part five association rule learning.
And we are going to cover two models.
The first one a priori
which is actually the best one of the two issues in my opinion.
And and of course we're going to start with a priori
and we're going to start here with Python.
All right.
So inside this Python folder you will find as usual the implementation
a Priory dot ipynb
which you can open with either Google Collaboratory or Jupyter Notebook.
And of course you will find the data set which is called market basket optimization
because actually association rule learning is used to do
market basket analysis or optimization.
That's why I called this data set this way. All right.
So speaking of which let's describe what this data set is about.
All right.
So for starters let's imagine the beautiful region of south of France.
You know with all these cute villages with happy people walking down the streets
and going to the grocery store every now and then or to the coffee places.
Well, you know, imagine a very lively place.
Well, people hang out a lot and love
going to the different shops, the different commerces
not only to buy their favorite product, but also to, you know,
chill out in the beautiful, lovely town and imagine that you are the business
owner of one of these stores, you know, selling food and delicious stuff.
And so, you know, you are the business owner of this shop and you would like to,
as any business owners, optimize and boost the sales, and you have an idea
it is to offer some new great deals to your customers.
And the deal is that you have in mind is to identify,
you know, the best association rules among the different products,
but by your customers in order to offer, you know, this very famous deal,
buy this and get that for free.
You know, if you buy this product, you're going to get that product for free.
All right.
And so now you see the idea we're going to use association
rule learning to find the strongest rules.
Saying if customers buy this product
then they will have a high chance to buy that other product.
And we will measure that chance so that in fact, if they get this product,
they will very likely want this other product,
and therefore they will very likely
get that deal, get this product and get that one for free.
All right. So that's what this owner would like to do.
And this owner just hired the best data scientist of a thousand friends.
Which means you of course, to do the job, you know, to find these association rules
and therefore what this owner did, because he knows a little bit
about data science.
He knows that he has to collect some data to provide to the data scientists
in order to learn these rules.
And so each week, this owner collect all the different transactions
of the customers.
And at the end of the week, you're given this list of transactions
which you have in front of you so that you can learn the association
rules and return the best deals that this owner should offer to the client.
All right. To that's your mission.
You have to identify the best deals so as to maximize the chance
that you know the customers will get the deals.
You know, we'll buy this product and get another one for free.
And of course, the price of this,
buy this, get one for free will be well calculated by the owner.
But that's a different question.
So that it can indeed not only optimize the sales, but also the profit.
All right.
So that's the mission.
And now let's make sure everyone understands the data set.
You know each row of the data set corresponds
to different transactions you know from different customers actually.
And for each of these transactions you have all the different products
that the customer who did the transaction purchased.
All right.
So for example, this first transaction corresponds
to a certain customer who bought in one basket some shrimps,
some almonds, avocado, vegetable mixed green grapes, etcetera.
Up to, you know, olive oil. All right.
Then this customer or you know, this transaction corresponds to a customer
who bought in one same basket, some burgers, meatballs and eggs.
All right.
This customer just bought chutney.
This first customer bought turkey, avocado and etc..
Right.
So all these transactions correspond to different customers
and all these different transactions.
There are actually many of them there actually 7500.
You know, if we scroll down up to the end, yes, 7500.
And these 7500 transactions were collected during the whole week.
You know, each week the owner of the shop does this.
He records all the transactions and he gives them to you,
data scientist, so that you can learn the association rules.
All right.
And your mission is to return to this owner as fast as possible.
Well, the best association rules, you know, of two elements
so that this owner can find the best deals for his clients.
All right. Good.
So now that we clearly understand the data set, well,
I suggest that we move on directly to the implementation.
So we're going to go back to our folder.
And we're going to open this a priori that ipynb
either with Google Colaboratory or Jupyter Notebook as you want.
Choose your favorite. Now it is loading.
It is laying out the notebook and in a second will have it opened.
All right. Perfect.
So as usual, this notebook is in read only mode.
And therefore what we're all going to do now is go to file here to create a copy
by clicking here on save a copy in Drive.
This will create a copy inside which we will be able to re-implement
that file from scratch. All right.
Because I remind that this course
is an action based course I want all of you to learn by doing,
because this is how you will maximize your progress in machine learning.
Okay, so now again, as usual, let's remove all the code cells
you know, only the code cell so that we can keep the well
highlighted structure of this implementation.
So don't remove the text cells here.
All right.
So there are a few code cells but we should be done soon.
This should be the last one.
Oh no. One last one. And there we go.
All right. So good.
We see the whole structure in one page.
So let's have a look.
So first we import the libraries as usual.
You notice
that I kept my data preprocessing template because we will actually use it a bit.
At least you know the first cells.
Then we have the data preprocessing phase.
There you go. That's inevitable.
Then we're going to train the primary model on the data set.
And then we're going to visualize the results.
And by visualizing the results here I mean, you know, visualize
all the different rules sorted by relevance as you can see.
Well, for his display the results meaning the rules non sorted
and then the results meaning the rules sorted by descending lifts.
You know, is that metric of measuring the relevance
of an association rule which you saw in the intuition lectures with Cairo.
So you see we're going to
display the rules by descending lift so that we can see the most relevant
and therefore the ones having the highest chance to convert the customers.
All right. So that's the structure.
And now whenever you're ready let's meet
in the next tutorial to start this implementation.
It will be a very exciting new journey.
I can't wait to implement this model with you.
And until then, enjoy machine learning.