Hi. In this video we will learn how to make machine learning pipelines in sticky
learn. First, we need to answer a question.
What is a machine learning pipeline?
Machine learning pipeline is a series of tips which needs to be
performed with the data, starting from data reading from various sources,
uh, to making final predictions. Uh,
what is in between highly depends on your data and your choices of algorithms.
And a goal, what you are trying to achieve.
A typical machine learning pipeline may contain these steps, uh,
but be aware that, uh, not all are necessary. Uh,
first you load the data from some source,
like a database I p I or c s V files and so on.
Uh, data pre-processing, uh, this step, uh, includes all necessary steps,
uh, to process the data to its right form and shape correcting errors, uh,
data cleaning, handling outliers, inputting missing values,
or encoding categorical variables.
Then your pipeline may include steps where you are creating new, uh,
variables from existing ones.
This is the feature engineering step feature selection.
Uh, after this, uh, if your dataset has many variables,
you may consider a feature selection where you drop unnecessary variables,
for example, uh, which have very low variance or a lot of missing values.
Sometimes you also need to normalize your dataset. This is the feature scaling
dimensional reduction. Uh,
you may also use some algorithm for dimensional reduction like, uh,
principle component analysis. For example,
feature dimens reduction methods are trying to preserve most of the
variants in the dataset, uh,
but they are trying to transform the data set that way. Uh,
so its output will have much less dimensions like the inputs.
And then you perform hyper parameter optimization where you search for the best
hyper parameters of your model or even the right parameters in other steps of
the pipeline. For example,
you could optimize depths of the trees in XG boost algorithm,
but also you may want to search for the optimal threshold for qual range used in
robust scaler.
The model evaluation step is present in a previous step, uh, but uh,
you may want to explore the model, uh, into more detail. And finally,
you make the predictions after you created your pipeline,
uh, and your best model, uh, you have to deploy it and set actions.
What should be taken based on these predictions.
And this is sometimes even more difficult than the model training, uh,
to set it properly. Today,
most of the machine learning is basically about data engineering and
about, uh, driving as many reasonable variables to the pipeline.
And to have enough resources like, uh, CPUs and ra, uh,
to perform hyper parameter tuning to have, uh, satisfiable results.
It's common that even for simple business problems like propensity models,
you throw thousands or even tens of thousands of variables into the pipeline.
So everything should be automated.
You can usually train your models on a good sample of your data in terms
of millions of observations, or even a few hundred thousand,
but sometimes we may need also a lot of resources to make predictions for all of
the data, which can be in billions of observations.
So it may be exactly the opposite,
that making predictions will require much more resources than the training.
In short,
machine learning is mostly about available resources and data engineering,
which takes a significant amount of time. Now,
we can go into our notebook.
LEA provides us a very effective framework how to perform most of the
steps I mentioned. Uh, it has a pipeline class,
which can be used to create a machine learning pipelines.
In a list of two pole, uh,
we specify name of the pipeline step
and the estimator or transformer. Uh, we want to use. Uh,
the difference between, uh, ticket learners, uh,
estimators and transformers is that transformers only
transform the data and they must have the transform method. For example,
the minimax scaler is a transformer.
Estimators are basically the models used, and they make the predictions.
So they must have the predict method. For example, the E G B classifier,
you can even create very easily your own transformers or estimators.
Okay, now let's create our first pipeline.
The first step is, uh, feature selection, where I used, uh,
Univa eight feature selection method,
which in combination with the select K best returns,
only the best, uh, 20 variables according their ANOVA F value.
As a second step, I choose the robust scaler with default parameters.
And the last step is our XG boost classifier
pipeline will have the same methods as the last transformer or
estimator in the pipeline. In our case,
the last step is the X GB classifier, which has the feed method,
predict method,
and also predict probus method to predict the probabilities and score method.
So we can call these methods on the pipeline as well.
When we call the feed method behind the scenes,
the pipeline will move the input data into the first step.
Call the fit transform method on the first transformer,
move the output data of that step into the next one.
Call the fit transform method on that step,
and move the output data to last step,
where in our case it'll call the fit method. And after fitting our pipeline,
we call the score method,
like here all steps in the pipeline,
except the last step must be transformers. In case of the last step,
it may be transformer or an estimator.
When we score, uh, the pipeline,
we see that it has the accuracy on the test set, uh,
only 93%. Now,
let's go ahead and cut.
Now let's go ahead and create our second pipeline.
Here is an example of normalization with the standard scaler
in combination with, uh, P C A.
This is the principle component analysis, which is, uh,
dimensionality reduction technique. Uh, the parameter and,
and components tells us how much variance I would like to
preserve. I put here 90%, but, uh, this means that,
uh, the P C A could return any number of columns. It can be 10, 20,
or 30, or any other number.
You can specify here on integer visual ensure that the P C A
will return only specific number of columns in P C A,
there are called components.
Regardless of how much variance is preserved in the transformer data.
I highly encourage you, uh, to learn more about this technique.
When we feed the pipeline and score it, uh,
we see the accuracy of 9 3, 3, uh, percent as well.
Now a little bit, uh, on how to access and the steps of the pipeline.
We can access the transformers or estimators in the pipeline very easily as
well. Uh, for example, uh, here, uh, in this cell,
this one, uh,
I access the p c a step where I printed out the explained, uh,
variance here of all the resulting components.
We also see, uh, that, uh,
with retaining of 90% of variance in the data, it,
uh, reduced the number of dimensions from 57 to 43.
Here in the next cell, uh,
we see that we can call the transform method on this already fitted, uh,
P C A, and we see, uh, what is the output of it, uh,
which is then, uh, uh, fed into the XG boost algorithm.
And of course, uh, we can make the cross validation of the pipeline,
same as with the single model,
and we see is the results here.
In our next video,
we will learn more about hyper parameter tuning focused on grid search.