Now, we have imported the data and the raw data is ready.
We will first take a look at the sample of this data.
To take the Simpson will write our variable name, which is D.F..
And little red dot head.
Fit on there.
We are getting five top rolls from raw data.
There's the sample of forward data.
We have 19 columns.
And the sample you can get, the type of all the variables, what values they are taking.
Next to look at the count of each variable and to look at their data, Day-To-Day will write D.F. Dot
info.
Here you can see we have over variable names on the left.
Then we have a column for number of values.
If you notice, four and Hoz bats, the number of loses 98.
That mean eight values are missing from this variable.
We need to handle this.
Missing were loose before running logistic regression or any other kind of machine learning algorithm
on this data.
And on the right more side, you can see the type of each variable name is sense for numerical data.
We also have object, which are distinct types of data that eschatological data.
To get the number of rose and columns in your data, you can also use Daudt shape.
So if I had a beef dot shape,
I wouldn't get the number of rows and columns.
So as you can see, we have five hundred and six rows and total.
There are 19 columns.
In which it bin columns are of over independent variables.
And the last column is for word dependent variable.
Now let's rent a duty on this data and buy a 10:00.
You will get a duty only for your numerical values.
So you will not get unity for airport waterboardings and bus terminal variable.
Two grand.
Really?
We just have to write down describe.
So we'll write D.F..
This is our very own name.
And then describe.
If we run this, you can see all our numerical variables are listed on the top.
And we have value such as count mean standard deviation minimum maximum.
25Th percentile.
58 percentile.
Seventy fifth percentile.
For each of these different variables.
Count, it stands for total number of values in that variable mean send for the mean or average of that
variable.
Standard deviation is the standard deviation of that variable.
Min and Max are for minimum and maximum value.
That variable is speaking in our dataset.
Then we 25th, 58 and 75 percentile value percentile.
If you remember, just mean that if you arrange all the values in ascending order, your 25th percentile
value will be the value that is occurring at the 25th percentile of that range data.
Similarly, 50 percentile is the value that is occurring at the 15th position of that data.
This is same as the median of data.
And similarly, 75 percent, 10 cents for value.
That is occurring at 75.
Position of the Arain straight up.
Now we will look at all of these variables one by one.
Not what we want from this.
It really is the number of missing values.
And the variables which have outliers, outliers, means values that are not following the pattern of
that variable.
So, for example, if the values of my some variable is between one and 10.
And then there is just one value, which is in range of thousand or 10000, then we call that value
as outlier.
So first we want to identify missing values.
Second, we want to identify the outliers.
Third, we will look at their distribution of categorical variables.
We have already identified missing values that in four or you can also look at the count, but all of
this UDD.
To identify the outliers, there are two methods.
First, you have to look at the difference between the mean and the median.
Median is that 50 percent had value.
So if there is any outlier, there is a huge difference between mean and median on outliers on the affect
mean, not the median value.
That's.
If there is any outlier in one of our variable, we'll see a huge difference between mean and median
value.
We can also look at the distribution of this minimum under 25 percent annually, 50 percent.
Lou, 75 percent value and maximum to notice any outlier.
If you see any major difference between any two consecutive values, that means there is an outlier,
for example.
And then what first column, which is for estimated price.
You can see minimum wage, Lewis fight and 25th percentile value is seventeen.
Then again, 50 percentile value is 21.
If you notice, that is not a great difference in any two of this category.
The difference between these categories are falling in between four to 25.
So you can see there is not a huge difference.
Or we can say that there is no outlier in our cristeta.
Now, similarly, you can look at all of this individual variables.
I will radically jump to the variable in which there are defects.
So if you look at the variable and hard Groomes.
The minimum value is ten.
The 25th percentile value is eleven.
Again, 58.
And so in the fifth percentile value are well.
And 14.
And the maximum value is one zero one.
You can see there is a huge difference between so in the fifth percentile value and the maximum value.
So we can say that there is something wrong with this data either.
There is a skewed distribution or there is an outlier in this data.
Similarly, if you look at the rainfall data.
The minimum value is just three.
When the 25 person dead will lose 28 and then 50 percent, then value is 39 and so on.
You can see that there is a outlier on the lower end of this data to confirm our assumptions.
There are several methods will first draw bulk splotched for Rover and Hot Grooms and then will use
a scatterplot blurring to file players for overtrained for.
One more thing.
There is no hard and fast rule for identifying outliers or identifying by turning your data.
These are creative processes and you have to come back.
Few are finding any difficulty in handling this data.
Later on in the process.
Now, before drying our box plot for Anha Grooms, I will first explain you what box plot is by using
an example for and Herzberg's so I will write as soon as that box plot
and then y equal to and hoz husbands.
And then data equal to B.F..
So this is our box looks in the middle, you can see a rectangle with a line in between, the upper
line of this box is that top quartile value, which is same as the 75 percentile value.
The bottom line of this box plot is for the first quartile value, which is the 25 percentile value
and model.
The line is for the 50 percent annually.
Also known as the median.
Now, the difference between the upper line of this box and the lower line on this box, which is the
75 percent down only under 25 percentile value is known as and third quartile range or also known as
AQR.
And this this skirts, which are present at the top and at the bottom of this lines are calculated using
this I.Q..
So by default, this is one point five times a cure from the upper and the lower end of this box.
We are not going into detail about this, but whatever points lies outside this with skirts are known
as outliers.
If you want to know more about box plot, you can write this statement.
You can use caution, Mark, before any function to open that help for that function.
So if we execute it, you will get the before you, Hal, about this function.
So you can use questionmark an airplane to open the help section.
So now let's grow a box, look for and heart rooms will write as soon as dot box blood.
And then what via variable is and hot Groomes.
And the data is D.F..
Fee executer, you can see the box and the skirts are flying at values below 20.
There are two points present outside this box spot which added values around 80 in hungry.
Clearly, these two points are outlier for this variable.
There are no values between 20 to 80.
These two outliers are the only value which just lying about 80.
So from this box plot, we have identified two outliers in our anha crumbs.
Now let's discuss the second method of identifying outliers.
Which is scatterplot draw droid scatterplot between our X and Y variable here, our X variable will
be there in full and our Y variable will be our dependent variable, which is solid to plot scatterplot.
We'll write as soon as Stojan plot,
then our X is rainfall.
Y is sold.
And data is D.F..
You can see on the top we have the Instagram of a word dream for data on the right hand side.
We have this program of over sored, which is dependent variable.
Our dependent variable is on there taking well, lose zero and one.
That's why we are only getting two bars in our Instagram.
And in between on the plot area, we have the scatter plots.
And as you can see, all the points are line between 20 to 60 except one point, which is near to zero.
We can clearly see this as a kind of outlier for this rainfall data.
And we will create this outlet in the later part of forecourts.
Now we have two observations.
From over a really off numerical data.
First, there are missing values and horse bets.
Second, there are outliers on the higher end and end hard rooms, and that is an outlier on the lower
end and rainfall variable.
Now, let's look at our categorical variables for our categorical variables.
We will plot borrowed graphs.
So our first categorical variable is airport to grow.
Alert for that.
We will write as soon as dot com plot.
X equal to airport and they take to D.F..
You can see our airport variable is sticking to values.
Yes and no.
And the height of this, but represent the number of occurrence of these values in our data.
As you can see, there is nothing wrong with this data.
Now, let's move on to our next Freamon, which is what everybody will copy this call and will write
Waterboardings and set off airport.
So over waterboarding, variable is taking forward values.
River Lake.
None.
And river in lake again.
There is nothing wrong with this data.
Next, get a political video of Elizabeth Steadman, then we'll just ride bus terminal and sort of what
body and what this.
Here you can see bus terminal is on there taking one value, which is yes.
So in a way, this is not a variable.
This is a constant.
It will not provide any additional information to what model.
It will just act as a constraint.
Therefore, we can delete this variable or grob this variable from our data as it will not provide any
additional information to that model.
This will just act as a concern for our model.
Nobilis known, our observations are the first observation was missing when he was in and Herzberg's.
I would second observation was bus terminal has on the US well use and it will not provide any additional
information for our model and we can drop it.
I know our third observation was.
About old players and and what, Groomes and brimful.
So always perform this EBD and univariate analysis before craning your model.
This will help you clean your data and make your data usable for your machine learning algorithms.