Hi.
In this video I would like to explain a little bit the notebook I created for
this module. So in addition to Jupiter lab,
you'll learn a little bit more about national learning, uh,
specifically about the binary classification.
I use the same data like in the fourth module,
but here we will create a model to predict if the email is spam or not.
This is called a binary classification problem because our target variable can
have only two classes, zero or one as inputs.
To this model, we will use all other remaining variables.
Models that we will try is the logistic regression and, uh,
random forest classifier.
I'll not go into details about the theory or inner workings and assumptions of
these algorithms.
Important for us is that these models can be used for classification
problems like ours. Now let's go into our notebook.
You'll see that typical machine learning workflow will consist of these steps.
You load the data from some data source,
then you make our introductory and exploratory analysis.
After that, you will pre the data, so that is suitable for modeling.
You may also do some pre-processing steps during or before the initial analysis.
Then you fit the data, you evaluate the model with suitable metrics.
You score the data.
That means you assign the predicted values or probabilities to each of the
entry,
and then you save this back to some storage where other users may access it.
In the first sections, we prepare the data and product view charts,
uh, just to check if the protein is working fine. And, uh,
considered this as a very, very basic, uh, explanatory analysis.
Then we splitted the data to train and test it.
After that, we have fitted the logistic regression model and evaluated with, uh,
accuracy metric on train and test set.
Then we protested the confusion metrics and cumulative gain charts.
We did the same with the random forest classifier.
These evaluation methods and metrics depend on what problem you are dealing with
if the data set is, uh, balanced and on the model used as well.
Uh, one word to accuracy metric. Uh,
this is basically a percentage of correctly classified emails.
Doesn't matter if it is zero or one class.
For example, accuracy of 90% might look excellent,
but consider in our case,
if we would predict for all the emails zero,
then we would be correct in, uh, 60% of the cases.
Now let's compare our two models.
We see that the accuracy of the logistic regression is, uh,
92% on train and test set. And, uh,
for the random forest classifier,
there is 74 and 70% accuracy.
We see that the random forest classifier is definitely a worse model,
but, uh, let's go a little bit deeper and explore the confusion metrics.
The confusion metrics tells us how many emails were
correctly or incorrectly classified depending on the class.
We see that emails, uh, which are spam. This is the class one.
We correctly classified them in 570 cases and we
wrongly classified it, uh, in, uh, 63 cases.
Uh, the same is for emails which are not spam. This is the Class Zero.
We currently classified, uh, these emails in 832 cases,
and we were broke in 54 cases.
When we compare it to the random forest classifier, uh,
the logistic regression is a little bit, uh,
worse in classifying the zero class,
but it's basically comparable, almost the same, uh,
but the random first classifier is much worse in classifying the positive class.
Uh,
basically it is wrong in predicting the positive class in around 60% of
the cases.
With cumulative gain charts,
we can go even deeper to have another view on how our models are performing.
They are used to evaluate effectiveness of a binary classifier like ours.
Uh, it can be used also for comparing two or more binary classifiers.
Intuition behind it is very simple.
You order the entries by the predicted probability that's sending,
and you check the percentage of correctly classified emails. We have,
depending on selection of top and percent of records,
the dotted line, uh, this is a random classification.
We see that, for example, logistic regression.
When we look at the top 30% based on our
predicted probabilities,
we have targeted around 70% of,
uh, spam emails.
When we compare it to the random first classifier,
we see that in the top three details. Uh, they perform, uh, the same basically,
but, uh, the random classifier is, uh, much worse in, uh,
ongoing deciles.
Okay, that was it for model evaluation,
and our last step is to save our scores to disc.
In our next video, we will explore how to manage Jupiter Labs extensions,
and we will try to use some of them.