Hi. In this video we will learn more about cross validation.
Cross validation is a method of evaluating a model on multiple folds of data.
This also a resampling procedure used.
If we have limited data available. Uh, it's best illustrated on a diagram.
We have already split our data, uh, to train and, uh, test it,
and we trained our model on the whole training set.
What we do with cross validation is to, uh, further,
uh, split the training data to multiple faults.
In this case, there are five faults and we train our model, uh,
five times, uh, with the same parameters each time on four faults,
and ed on the fifth fault. This way, uh,
we will have a better view, uh, how our model generalizes on unseen data.
The test data is used, uh, same as before,
as last and final evaluation. When you get, uh,
satisfiable results using cross validation,
and you have found a good hyper parameters of the model.
We can use a cross validate method for this, uh, where we input our model,
uh, training data. Uh, we want just, uh, three folds. Uh,
we want on also cut.
We can use cross validate method for this,
where we input our model training data. We want just,
uh, three fold.
We also want to see a performance of the model on training data,
and we also specify the scoring metrics, which we would like to see
when we run it. Uh, we see the results on these three folds,
and we see that our model is a bit over fitting.
That means it predicts excellent on training data, but worse on data.
E D L is approximately the same performance on both. Uh,
we see it has, uh,
almost a perfect performance on training data in all of the metrics,
uh, but worse, uh, on the test data,
what we could do is to change few parameters of the model. For example,
number of trees or depth of the trees, uh, to lower values.
Uh, but uh, in our case it is not so bad. Uh,
but if it would be more than 10%, we would have to do that. In
our next video, we will start to make pipelines.