Halaal, before going deep dive into this session, let's have a quick recap of what we have done below
in this project.
So from a starting, if I will talk about, we have the lots of analysis on data, some spatial analysis
and some amazing analysis that we have, find some kind of trend for our data, as well as from machine
learning aspect we have found here.
What are exactly my most important feature, and we have basically set this using our coordination concept.
After what we have done, we have basically used this design features concept call this year, month
day plays a very important role at the time of my model creation.
So that's what we have done.
So in the previous session, what we have done basically we have put forward feature encoding on our
data to intersession what we have to do.
This is exactly the assignment that we have to cover so that any force problem statement is we have
to handle outliers.
What exactly are the outliers?
Let's say there are some data points that are too much far away from the normal one.
So those are exactly my outliers.
So let me explain it in very layman's terms.
Let's say you have a data of, let's say, a hundred persons whose age is between, let's say one,
two hundred, but let's say one hundred person has an age of, let's say, seven hundred years.
So that hundred person will be considered as my outlier.
So whenever I'm going to build a model, that model will get impacted badly by this outlier.
So we have to handle this outlier condition in our data as well.
So let me show you a trick, how you can find outliers in your data.
So whenever you have to find whether there is some outlier or not, there is basically a baby approach
you can just follow.
It is exactly your distribution.
So let me found what exactly is our distribution over here.
So let me give you very first a quick overview of what exactly is my data.
I'm just going to call ahead over there.
So if I'm going to executed, all the stuff gets executed over here for you over here.
Here you have multiple numerical features.
Let's say I'm going to talk about this lead on a school time feature.
So let's say I'm just going to call my distribution of this feature.
So I'm going to say yes and then start this plot.
And very first, I have to access my data.
So I'm going to say data frame of this feature.
Just execute this cell and you will get this amazing distribution pattern here.
And you will see over here the distribution of this feature is a little bit right skewed.
And you can clearly observe over here, this is exactly that range.
Those are exactly my outliers.
You will see most of the data points that is back to this feature always line between almost 200 to
300.
But there are some low number of data points that are almost my six hundred eight hundred.
So these are exactly my outliers situation.
So we have to handle this situation.
So how to handle the situation?
So what we are going to do, basically, we are basically going to calculate log of this lead in a school
time or you can see we are basically going to replace this original feature with its log, because once
we take a log of this feature, this is Skewness up to some greater extent will get solved.
That's our approach.
So what I'm going to do over here, I'm just going to define a function, let's say, handle on this
core outlier.
So this function will be final over here whenever we have to handle our outlier in my data.
So what this function will receive the very first is exactly whatever column name I'm going to pass
it to function.
That's it.
And after it, what I have to do, I have to basically say I'm just going to call my log on whatever
feature I'm going to pass over here so that if I have to import my name by model because it will function
of log, it's exactly available in this number.
So I'm going to say in NUM by as ENPI, that's it.
Now what we have to do, we have to perform this log one B, so if I'm going to shift a stat then you
will get all the documentation of this function, you will see all your custom parameters.
So here what we have to pass that is first I have to access the frame.
So I'm going to say this is exactly my data frame and whatever column I'm going to pass over here,
this will exactly convert that feature into its log.
That's what I am going to do.
Now, what we have to do, we have to update this feature as well.
So I'm going to say this, nothing but just this one.
Now, what we have to do, we have to just execute this.
Now that simple, we have to simply call.
It functions, I would say, handle an outlier and simple, I have to just follow my lead and escort
time feature and let me show you one more thing.
Let's say once I will execute this function and again, if I'm going to execute the cell over here and
if again, I'm going to call it a distribution plot of this data frame, let me access my data frame
very close.
So I'm going to say data frame of, let's say, need an escort to execute the cell.
And you will visualize this time you don't have that much skewness in your data.
It means you don't have that much outliers as what you have earlier.
You will observe you will clearly observe here in a similar way.
We can also handle this outlier in my price feature, which is exactly my ADR over here.
So for this what I'm going to, let's say just check what exactly the distribution of this ADR, I'm
going to say data frame of ADR, execute the sale.
Now you will see it has certain kind of outliers.
You will see the maximum data point in this area is exactly somewhere close to five thousand.
And but I think that you have to notice most of the data points are going to lie between zero to almost
200 to 300.
And these are exactly my data points that are outliers, which are approx, close to five thousand.
It means, again, we have to handle this outlier situation.
So for this, I'm just going to call this function and I have to just pass this feature.
That's it.
Outlier gets handed over here.
And again, if I'm going to copy all this stuff, let me just paste and let me again execute it.
But before executing it, you have to also take care of whether any missing value is available in this
area or not.
So if there is any missing value in such scenarios, what you can do, you can just call drop and that's
it.
That simple.
Just execute the sale.
And this time you will see it is somehow very close to your normal distribution.
So now this is exactly that data that you really want for your machine learning purpose.
So that's all about the session of the session.
Very much how exactly I have handled this outlier condition in my data for hope.
You love the session very much.
Thank you.
Have a nice day.
Keep learning.
Keep going.
Keep practicing.