In our data, we need to do our prior treatment.
Also.
What is an outlier?
First of all.
Outlay values are the values that appear finally or diverge from overall pattern of a variable.
Outlays are usually because some error occurred in the measurement or data entry or sampling process.
For example, the error was during data entry and you wanted to enter one point three, but instead
entered 13 because you missed the decimal point.
Or it was measurement error.
And while measuring one of these sensors malfunctioned and given at an Israeli.
Reasons may be any, but it must be handled carefully before feeding the data to our model, because
these outlying values will otherwise lead to high error variants and lesser prediction accuracy.
To find outliers, we should use EDT and visualization tools such as both blogs, scatterplot and histograms.
We will see this when we vote on this software tools.
Once we have identified the outliers, we place these values with some other values.
There are several options of what value we should use to impute outliers.
First, let me show you how one single outlet value impacted these data.
And how can we identify these outliers?
These two datasets are seeme, except the one on the right has one outland value.
Three, 300.
This lead dimino, right, want to be shifted towards this outlier?
But median is not doing much.
So, Wendy, Data has outliers.
There is a larger difference between mean and median.
Also, the standard deviation in such a case is higher.
So once we have identified the outliers, we have the following options.
We can do capping and floating.
What this means is we find the upper limit and lower limit beyond which we will change the values.
So suppose we selected ninety nine percent day as a parliament and all valued larger than this value
will be assigned a particular value.
I hope you remember what the percentile value is.
If you order all the values that are available with you in ascending order, then 99 percentile value
is the value, which is larger than 99 percent of other values.
To only one percent of values which are larger than this value will be considered as outliers.
And they will be assigned a different value.
Usually we assign a value such as three times the 99 percentile value.
This murder, Gladder three is a personal choice.
Similarly, for lower limit, we can use first percentile value and multiply it up.
Point three.
Second method is exponential.
Moody.
In this method, we extrapolate the decode of 19 fifth percentile to ninety ninth percentile beyond
the 99 percentile point.
And the values which are going beyond 99 percentile point to fall on this good.
This may seem difficult, but software packages do it straight away.
However, I always preferred Method one.
Third is the Sigma approach, which is popular in the manufacturing industry in this approach.
All values beyond some sigma.
For example, all values which are farther from mean by more than three times standard deviation.
Are replaced with the value mean plus three times to end the division.
This gives results similar to the first one.
So one and three are actually equivalent.
With this, we now know how to identify our lives and how to treat them.