All right.
So, as usual, you know, in order to be as much efficient
as we can, we're going to do this with scikit learn, right.
This data science library that has all the tools
and the tool that we're about to use is a class called standard scaler,
in which will exactly perform standardization on
both your matrix of features of the training set
and the matrix of features of the test set.
So let's do this.
Let's start by importing this class, which we have to take first from.
Well scikit learn of course as k learn.
And then from which we're going to get access to the pre processing
module perfect which is a module that contains that standard scalar class.
All right.
So then we're ready to import what we want which is the standard scalar class.
Perfect. So now we have the class.
And then the natural next step here is of course to create
an object of the class I'm about to reveal soon.
That answer to one of the most frequent questions in the data science community.
So let's create this object.
We're going to call it SC for standard scalar.
And then well this object will be created as an instance of the standard
scalar class.
So I'm taking it here pasting that right here adding some parenthesis.
And good news here.
We don't have any arguments to input because what we simply want to do
is get that mean, get that standard deviation,
and then apply this formula to all the values in the feature.
And for this we don't need actually any parameters.
This will automatically do the job.
All right. So then next step.
And now well now is the time for me to reveal
the answer to that question, which is one of the most frequently asked
questions in the data science community.
And that question is do we have to apply
feature scaling, you know, standardization
to the dummy variables in the matrix of features?
This is one of the most frequently asked questions.
You will find it everywhere online as well.
And once again, actually
the answer is pretty obvious, but only after you get the explanation.
So let me tell you the answer.
The answer is no.
The answer is no because simply
well remember the goal of standardization or feature scaling in general,
it is to have all the values of the features in the same range.
And since I told you that standardization actually transforms your features
so that they take values between more or less minus three and plus three.
Well, since here are dummy variables already, take values
between minus three and plus three because they're equal to either 1 or 0.
Well, there is nothing extra to be done here with standardization
and actually standardization will only make it worse because indeed
it will still transform these values between minus three and plus three.
But then you will totally lose the interpretation of these variables.
In other words,
you will lose the information of which country corresponds to the observation.
Now we perfectly know that you know, remember one,
zero and zero corresponds to France because that's how it was encoded.
And then zero, zero and one corresponds to Spain.
But you know, after
we apply feature scaling, if we apply it on the dummy variables,
we will get nonsense numerical values and we will be absolutely incapable
to say which tuple of three values here correspond to which country.
So we will totally lose interpretation.
And besides, this won't improve at all to training performance
because indeed our dummy variables are anyway already
between the same scale range as your other variables
you will see online that applying standardization to your dummy
variables might still increase slightly the performance.
You know the final accuracy of your model.
But I've experimented many times and I've never seen,
you know, a considerable difference that would justify here to apply feature
scaling on the dummy variables.
So really don't do this.
Only apply feature scaling to your numerical values.
Right here we have clearly some variables taking values in a very different range.
Right.
The age goes between 0 and 100 and the salary goes between 0 and 100,000.
So clearly here if it's better for the machine
learning model we have to apply feature scaling.
But let's leave these dummy variables alone
so that we can keep the interpretability of the model.
All right.
So that was the other very important questions.
Now I've covered everything.
There should not be any confusion left in data preprocessing.
I'm really glad that you know this.
And therefore I encourage you now to press pause on this video.
And guess what will be the next step to apply feature
scaling to R matrices a feature extreme and excess.