Hi.
In this video we will create and evaluate a single XG boost binary model.
For this model, we will use, uh, familiar dataset, if you remember,
uh, from one of our previous modules, uh, the dataset,
which was consisting of, uh, multiple statistics of emails,
uh, like a number of currencies of some word, uh,
like make address all and so on.
And, uh, target variable,
the noting if the email was a spam or not. Uh,
this data set is, uh, whole numeric without any missing values,
uh, but it contains a lot of, uh, zeros.
We see it has, um,
58 columns and 4,600 observations.
We also see that the target class is nicely balanced, uh,
with 60 to 40 ratio, and basically it doesn't need any further,
uh, pre-processing. Let's go to this section.
Training BU model.
We have to split the data to inputs and a target. Uh,
then we also have to rename, uh, this, uh, variable, uh,
because XG Boost doesn't like, uh, bracket in the column names.
Then we have to split the data into train and test set with, uh,
30% to be the test set.
And here in the cell, uh,
we instantiated XG boost model with the default parameters. Uh,
we fit the model on train data and we evaluated the model
on the data with the score method. Uh,
the score method uses, uh, behind the stains accuracy as a scoring metric.
In our case, it is suitable, uh, because we are dealing, uh,
with binary classification where the class, uh,
zero and one are more or less balanced,
and we see a very nice performance of 95%.
Uh, but let's evaluate it a little bit closer.
A very useful tool, uh, is the classification report. Uh,
let's check what all these numbers mean. Uh,
precision for class zero, uh, the number of zero point, uh,
95 means that from all predicted values of
zero, we were correct in 95% of the cases.
In case of recall,
the number of 0.97 means that from all true values
of zero, we correctly identified 97% of them.
The same applies for class one.
The F one score combines precision and recall,
and it is the harmonic mean of these two.
Uh, support is the number of actual currencies.
Uh,
accuracy is basically just a percent of correctly predicted observations
regardless of the class.
Macro average is an average of given metric across all classes,
uh, treating all classes equally,
and the weighted average takes class imbalance into account. Uh,
metrics for each class are calculated with a weight that reflects the true
distribution of the classes in the dataset.
We have discussed the cumulative gains curve already in one of our modules.
Uh, but let's, uh, make a refresh. Uh, let's make an example on class. Uh,
one, uh, that means spam emails, which is the green line.
For example,
if we would target the top 30% emails based
on their predicted probability to be a spam email,
we would target then around, uh,
70% of all spam emails.
This approach might be very handy if your actions are associated with some
costs.
We have covered also the confusion metrics, uh, and we see here that, uh,
from all spam emails, we had, uh,
wrong prediction in 40 cases,
but we were correct in 519 cases.
Another important thing,
what needs to be checked are the features importances of the model
to examined features.
Importants of the model is very important because we can check, uh,
what variables are driving the performance of the model.
Uh, this is one of many things which make, uh, uh,
the IG boost and decision three algorithms, uh,
very popular compared to neural networks, for example.
Because neural networks are basically black box algorithms,
we can check, uh, it via the feature NCES attribute.
We see, uh, that, um,
the occurrences of word remove contribute the most to the performance of the
model, followed by, uh, frequency of the, uh,
dollar sign and exclamation mark in the text of the email.
In our next video, we will learn more about cross validation.