Hello and welcome to
this art tutorial and welcome to part nine Dimensionality Reduction.
So we are starting this part with our first technique
of dimensionality reduction which is PCA principal component analysis.
And so you know in dimensionality reduction
there are two techniques feature selection and feature extraction.
We did feature selection in part two when we implemented the backward
elimination model to select the most relevant features
of our matrix of features.
That is the features that explained the most the dependent variable.
And now we are starting this new technique of dimensionality
reduction, which is feature extraction and PCA.
Principal component analysis is one feature extraction technique.
So as a reminder let's say your matrix of features has m independent variables.
Well what PCA will do is that it
will extract a smaller number of your independent variables.
But there are going to be new independent variables like new dimensions.
And these new independent variables extracted are going to be
some new independent variables that explained the most
the variance of your data set.
And that is regardless of your dependent variable.
And that makes PCA an unsupervised model in the sense
that we don't consider the dependent variable in the model.
So that's PCA.
And remember in part
two and part three we worked with 1 or 2 independent variables.
Well that was for two specific purposes.
The first purpose is that we needed a graphic visualization of our results.
And since each independent variable corresponded to one dimension in the plot,
well, we could visualize our results with at most two independent variables.
And the second reason is that thanks to this PCA dimensionality
reduction technique.
Well, even if we have a lot of independent variables at the beginning,
well, we can end up with much less independent variables.
But that are going to be relevant independent variables, because
these independent variables will explain the most the variance of your data set.
And therefore, since we can reduce this number of independent variables,
well, we can end up with 2 or 3 independent variables
and therefore visualize the results as we did in part three.
And this is exactly what we're going to do in this tutorial in the following
tutorials of this section and the following sections.
When we cover other dimensionality
reduction techniques like LDA and also kernel PCA.
Well, we will have many features at the beginning
and therefore it will be impossible to visualize the results.
But then when we apply PCA or LDA, we will reduce the number of features
down to two and therefore will be able to visualize the results.
So let's start right now and let's start by setting the right folder
as working directory. So as usual we go to our machine learning.
It is that folder.
Then part nine dimensionality reduction.
And here we are in the first section of this part nine
Principal Component analysis.
That's our first technique. Let's click on it.
And that's the folder we want to set as working directory.
Make sure that you have the one dot CSV file.
And if that's the case you're ready to click on this more button here
to set this folder as Working directory.
Perfect.
And now we're going to open another file that we made in part
three classification which is our logistic regression file.
Because what we're going to do is take this logistic regression code.
Then we are going to change the name of the data set,
because we will be working on a new data set, which will be the wine dot CSV file.
And we will apply PCA on this data set.
And of course I will explain quickly the business problem behind it.
So I'm going to take everything from here down to the bottom.
Here we go. Copy.
And I'm going to paste that in my PCA file this way.
So let's go up and let's change the name of the data set.
This is not social network add CSV.
This is now one dot CSV.
That's perfect.
So now what we're going to do is first import this data set
and then apply data preprocessing.
Maybe we'll need to change
some things like the index is here but this will be very quick.
So first let's import the data set.
So I'm going to select this line and execute.
Here we go. The data set is well imported.
Here it is.
And now let's expand the business problem behind it okay.
So first of all this is a very famous data set
well known in the machine learning literature.
And that you can find on the UCI machine Learning repository, as you can see here.
And you can find this page at this link.
So basically
first what are the independent variables and what is the dependent variable.
Well the independent variables are all the variables from this one.
Alcohol up to this one proline.
And this last variable customer segment is the dependent variable.
So in the original data set this dependent variable is not called customer segment.
This is actually the origin of the wine.
But let's imagine that we as data scientist are working
for one business owner.
And this one business owner gathered all these informations in this data set.
And so first what this business owner did is that it gathered all the informations
of these independent variables here
that are chemical informations of several wines.
And this business owner applied some clustering technique
to find some segments of customers
that like a specific wine, depending on the informations of the wine.
And by applying these clustering techniques,
this business owner identified three segments of customers.
That's the first one here.
Then we have the second one and eventually the third one.
So based on these
informations and thanks to its clustering techniques,
well this one business owner managed to find some segments of customers.
Each segment having a specific preference for a specific wine.
So basically this business owner found three types of wines,
each type of one corresponding
to one segment of customers and therefore three segments of customers.
And why does it create added value for his business?
Well, that's because now
what this business owner can do is take all these informations of the wines,
as well as the information about the customer segments,
and make a classification model like logistic regression,
in which the independent variables are all these variables
and the dependent variable is the customer segment.
And therefore for each new wine, it can predict to which customer segment
it should recommend. This wine.
So that adds a lot of value for this business owner.
But then if this business owner wants to have a clear visual
look at the prediction regions and the prediction boundary
of the classification model that we're going to build, to be able to
see if the predictions are in the right spot of the customer segments.
Well, it cannot be done with all these independent variables because of course,
we cannot represent these many independent variables in one plot.
That's impossible.
So what we need to do is apply some dimensionality reduction techniques
to extract
two independent variables that explain the most the variance,
and then we'll be able to see the prediction regions
and the prediction boundary.
And therefore we will clearly be able to see where the customer segments are
and where these predictions
of the customer segments are according to the extracted features
of all the informations of our independent variables.
And remember, these extracted features are called the principal components.
All right.
So now that we understand the challenge and the business problem, let's apply PCA
to see how we can reduce the dimensionality of this data set.
Because indeed it contains 13 dimensions because it contains 13
independent variables.
And we'll see how we can use PCA to reduce this number
of independent variables down to two independent variables.
But be careful.
It's important to understand that the new two independent variables
that will have in the end are going to be new ones, as opposed to
feature selection, where, you know, we end up with two independent variables
that are among these original 13 independent variables.
Here with PCA, we'll get new extracted one.
And that's the important distinction
to make between feature selection and feature extraction.
All right.
So before we apply PCA as usual we need to preprocess the data.
And this is actually going to be very quick because our template is ready.
We will just need to change just a few things.
So first data set equals data set three five.
That's just to select the independent variables that matter for our problem.
But here everything matters.
We just want to reduce the dimensionality of this data set.
So we will keep all our independent variables here.
And therefore we don't need this line here.
So I will just remove it okay.
So first section and bring the data set ready.
Well executed.
Now let's move on to the next section.
So the next section is about splitting the data sets into the training set.
And the test set.
And here be careful.
We just need to change this name of the dependent variable.
Because in logistic regression we're dealing with the social network add
CSV file.
And the dependent variable was purchased.
But now for a new business problem the dependent variable
is not called purchased.
It is called customer segment.
So we just need to replace purchased here by customer segment.
Here we go.
All right.
Do we keep a split ratio of 75%.
Well let's rather take 80%.
But that's as you want.
It's just that 80% is a good split ratio to take.
So we will go with that.
And then here for training set and test set we don't need to change anything.
So we are ready to split our data set into the training set and the test set.
So let's do it I'm going to select all this section here
and press Command Plus Enter to execute.
Here we go.
The training set is now created as well as the test set.
Great.
So ready to move on to the next section.
The next section is about feature scaling.
And for PCA it is way better to apply feature scaling.
You can actually apply it by playing with the parameters
of the PCA function that we're going to use afterwards.
But let's take this feature scaling part of our code
template to put our features on the same scale.
So here we just need to change the indexes.
We actually need to specify the indexes of the features we want to scale.
So basically the features we want to scale are all the features
from alcohol to proline.
And so what we can do is specify that we want to scale all the variables
except the last one customer segment that has index 14.
So therefore here instead of putting the indexes of the features
we can replace it by -14.
We can remove that.
Let's copy this because we will do the same for the others.
So let's replace that here by -14.
And -14 here as well.
And eventually -14.
All right.
So now the feature scaling part is ready.
So we are ready to select the section
and press Command or Control plus enter to execute.
And now all our variables are scaled.
As you can see we can clearly see that all our features are on the same scale.
And of course the customer segments kept its labels one, two and three.
And same for the test set. Let's make sure that.
All right perfect.
So feature scaling done.
And actually the pre-processing phase is completed.
So we did that quite efficiently.
But that's good because now we're getting to the exciting part.
Applying PCA to our data.
So actually we will do that right here.
You apply PCA right after the data preprocessing phase.
And just before you fit your logistic regression model to the training set,
because of course,
you want to train your model on your new data set with the new extracted features,
that is, with the two new extracted features
that will explain the most variance.
And after you trained your classifier, you're ready to predict the test results.
Make the confusion matrix.
Then you can also visualize the training set results.
Remember, this section is applied on a data set that contains two features,
and so we will see what we get by extracting these two new features.
All right so to finish this tutorial I'm just going to introduce
this new section here that I'm going to call applying
PCA. All right.
And in the next tutorial we are going to apply PCA.
And then eventually we will build our model on our new reduce data set.
So I look forward to doing that in the next tutorial.
And until then enjoy machine learning.