Hello and welcome to this art tutorial.
So in the previous tutorial we learned a very efficient way
to evaluate our model performance.
So that was about evaluating our model performance.
And in today's tutorial we are going to learn a technique
that is going to be about improving model's performance.
So we already built very powerful models but we can still improve them.
And how can we do that?
Well it's by finding the optimal values of the hyperparameters.
Because indeed any machine learning model is composed of two types of parameters.
The first type of parameters are the parameters that are learned
through the machine learning algorithm, and the second type of parameters
are the parameters that we choose.
Like for example, the kernel in the SVM model, or the penalty parameter,
or even some regularization parameter in ridge regression or lasso,
and basically in any machine learning model,
we have a lot of parameters that are not learned.
And so far, what we've done in this course was just to choose one single value
for these parameters.
We never actually experienced with these hyperparameters.
We never tried several values of them.
And so that gives us still a lot of room for improvement, because maybe there were
better choices for the values of these hyperparameters,
you know, a better choice than the value we chose.
So this technique called grid Search, will answer to one of your questions
that we're asked many times in this course, which is
how do I know which parameter to select when I make a machine learning model?
What is the optimal value of these hyperparameters?
And grid search will give an answer to that
because it will find the optimal values for these parameters.
So let's see how it will do that.
And let's implement grid search.
So we're going to work on the same problem as in the previous tutorial.
That is you know, this classification problem
where we need to classify the users in the social network and predict
if they're going to click yes or no on the ad to buy the SUV.
And therefore we are working with this data set, social network ads.
And besides, that's a good example because the kernel SVM model
has many parameters.
You know, it has this penalty parameter, this gamma parameter.
And some of you asked me which values we should choose for these parameters.
And so grid search will tell us exactly that.
So let's do it.
We are working in the same folder as in the previous tutorial.
That is the model selection folder.
And make sure that you have the social network ad csv file.
And if that's the case you ready to go.
So we are going to apply grid search anywhere
after the data preprocessing phase, because we are not actually going to use
this kernel SVM model
and then doing some parameter tuning on it to find the optimal values.
We are actually going to use a new package
that you will see is a very practical package to build
the same kernel SVM model as this one here,
but by applying grid search on it at the same time.
And this package, you already know it.
It's the carrot package.
And really, when we talk about machine learning in R,
the carrot package is one of the most practical packages
because basically with this package you can build any machine learning model.
And not only you can build any machine learning model,
but when you build a machine learning model with the carrot package,
well, it will give you the model you want with all the optimal parameters.
So that's pretty powerful.
And therefore that's one of the most powerful packages in R.
So as I told you, we can build
this new tuned model anywhere after the data preprocessing phase.
So let's actually make it right after the k fold cross validation
section to, you know, keep the other interesting code sections here
so that for example, you can compare the two models, you know, the one we made here
and the new model we were about to make with the carrot package.
Feel free to compare their performance results.
So we're going to build this new kernel SVM model right here.
So applying
grid search to find the best parameters I prepared this code section here.
But actually it's not only applying grid search it's also building a new model.
So first the first thing that we have to do is to import the carrot package.
We already installed it in some previous tutorials, but let's
make sure you install it if you didn't follow those previous tutorials.
So I'm just putting back this command and inside of course carrot in quotes.
All right. And then I'll put that in command.
Here we go.
And now let's import the package with the library command.
Here we go.
Library and carrot inside. Perfect.
So now let's build the same kernel SVM model but with the carrot package.
And you're going to see that it's going to be something you.
Okay.
So since we're building this new classifier the kernel SVM classifier.
Well let's go our model with the usual name that is classifier.
All right.
So that's the variable.
And then equals.
And then that's where we use the carrot package.
And what is the function we're going to use. Now.
It's one of the most used function of the carrot package.
It is the train function. So parentheses.
And now what's really interesting to do is to take a browser.
Here it is.
And let's type carrot
enter and let's take the GitHub link.
So let's click on it.
And inside of it you need to click on this link right here.
And then let's click on six available models.
Because in this section
you'll get all the models you can build with the carrot package.
And there are a lot of models there
actually all the models we built in this course.
So right now some of you might be thinking then why didn't we build all these model
with the carrot package, since indeed the carrot package
will give us these models with the best optimal parameters?
Well, that's because the packages we used to build all the models in the course
have great options, some of which you cannot use with the carrot package.
So it's good to know how to use both the packages we used before in the course
and the carrot package, but definitely for parameter tuning
you should use the carrot package.
All right.
So this list you see
here is the list of all the models you can build with the carrot package.
And since right now we are building the kernel
SVM model, well you will see that it is available.
I mean it is part of this list and
we can find it right here at the bottom of this list.
Because indeed we can see that we have many support vector machines model.
And the one we are interested in is the support
vector machines with radial basis function kernel.
Remember the radial basis function kernel is the Gaussian kernel,
the most commonly used kernel to build the kernel SVM model.
And so this is the model we are going to build right now with the carrot package.
And the information that we need to take right now that is going to be one
input of the trained function we were about to use.
Is this information here?
That's the input of one of the parameters of the train function.
The method parameter.
And it is with this parameter that the train function will know
which model to build and which model to tune.
So right now what we have to do is to take this name,
copy it, and we will paste it as input of the method parameter
inside the train function.
So let's go back to RStudio.
Here we go. And now let's build the model.
So we can actually press
F1 here to get the informations of the arguments.
So let's see what we have to input.
The first compulsory argument we need to input is this form argument.
And of course that's the formula.
So let's do it.
Let's input this first argument form equals.
And then we need to put the formula
exactly as we used to do when building the previous models.
So that's the dependent variable which is as a reminder purchased.
You know it's the social network and business problem.
So here we need to input purchased.
Here we go.
Then tilde and then all the independent variables.
And remember we don't have to put the names of all the independent variables.
We can use this shortcut here.
That is this dot.
All right then comma and then next argument.
So the next argument is data.
And that's of course your training set.
You're building your classifier on a training set.
And so you need to input here data
equals training set.
Here we go.
So now is the form the formula and the data the training set.
You have all the information you need to train your model.
But then of course we need to specify which model we have to build.
That is we need to specify that we want to make the kernel SVM model.
And that's what happens with the third argument,
which is not one of these arguments here.
These are not compulsory but it's the method argument.
Actually you have the link here, the link we just browsed on Google.
This link will give you the list of all the models available and correct.
And so that's for this method parameter that we need to input.
Once we copy it in this link, that is SVM radio.
So let's input that right now coma.
And then method equals then quotes.
And inside of these quotes we need to paste SVM radio.
And that's actually all with these three arguments
the formula the data and the method you will build a kernel SVM model.
And then for the parameter tuning you're going to see what happens.
So first what we're going to do is execute the data preprocessing phase.
Because we need to import the data set and applying the data preprocessing phase.
So let's do it right now we're not going to execute these sections
because this is to build the kernel SVM model the other way.
So we just need to take everything from here
up to the top.
Here we go. Let's do it.
All right. Data set. Well import it.
We have the data set the training set and the test set and the data
preprocessing phase was applied all correctly.
All right.
So now let's build the kernel SVM model with the carrot package.
So let's make sure we import the carrot package.
Here we go.
And now let's select this line and execute it.
All right.
It's taking a little time.
Well around one second that's fine.
And here we get our classifier.
So that's the kernel SVM model classifier.
It is not yet tuned but you're going to see what's going to happen.
Because now what we're going to do is pressing enter here.
And we're simply going to type classifier and selecting this.
And pressing enter.
And there you get a lot of very interesting informations
about your kernel SVM classifier you just built with the carrier package.
Because indeed what you see here is the conclusion of some parameter tuning
that was done on this kernel SVM classifier you just built.
And what is this conclusion?
It is that accuracy was used to select the optimal model using the largest value.
So that means that the accuracy
was the performance metric to evaluate your model performance.
And just below you see that the final values used for
the model were Sigma equals 1.12,
1.13, and c equals 0.5.
And that's the results of your parameter tuning.
That's the optimal values of the hyperparameters of the kernel
SVM model, the ones that will make your kernel SVM model
even more performance than the one you built with the previous method,
because of course, maybe the default value for the C parameter was 0.5.
This looks like the default value, but definitely the default value
for the sigma parameter, which is another hyper parameter of kernel.
SVM was definitely not 1.13 and was this 1.13 value.
For the sigma parameter, you get the optimal accuracy.
An accuracy that is higher than the accuracy
you obtained with the kernel SVM model built previously,
and you can even see what this accuracy is.
It is shown right here.
Accuracy is 92%.
And besides this is a relevant accuracy.
This is not the accuracy measured on one traintestsplit.
This was measured on several traintestsplit
which you can exactly see here with this information.
Resampling bootstrap was 25 repetitions,
which is exactly the same principle as k fold cross-validation.
That means that you take different samples of training sets and test sets,
and you build your model on the training set
and test its performance on the test set for each of these samples, and eventually
you take the mean of the accuracies and you get a 92% accuracy.
So that's definitely a good accuracy.
And besides, that's the best you can obtain with
these optimal values of the hyperparameters.
Sigma equals 1.13 and c equals 0.5.
And now I'll give you an even faster way to get these optimal values
of the hyperparameters.
You just need to well let's keep this classifier here.
But let's copy that.
You just need to take your classifier and then add a dollar sign.
And then you can add here best tune.
Here you go.
You select it, you execute it.
And you get directly the optimal values of the hyperparameters
sigma equals 1.13 and c equals 0.5.
And now congratulations you got your best kernel SVM model
with the best training coefficients and the best hyperparameters.
And now the choice is yours.
You can either take this kernel SVM classifier you built here
with the carrier package,
or you can take the classifier you built here with the usual way.
But inside of this SVM function you can input
the optimal values you found here for the hyperparameters.
That is, sigma equals 1.13 and c equals 0.5.
I will let you that for practice
and judge yourself which method you want to choose.
And I'll see you in the next and last section of this course
in which we are going to implement
one of the most powerful models in machine learning, XGBoost.
So this is going to be a very exciting section.
I can't wait to see you there.
And until then, enjoy machine learning.