Now that we have looked at univariate analysis and by vinyard analysis, it is time to sit back and
again think about these videos that we have.
Remember that we are trying to keep only relevant variables in the analysis.
So variables like bus terminal, which has only value.
Yes.
To our needed I it.
It is pretty indicative that this variable is not going to add any information to our model.
It may be relevant.
Maybe if a city does not have a bus terminal, the prices will be lower there.
But we cannot see that by looking at the sample because dissemble has no observation.
Devalued bus terminal is no.
So since this variable is not useful, we will remove it from our day, does it?
Think about all these points again, these four points to decide if the variables we have are relevant
or not.
Fosters rebuild with single unique value such as bus terminal.
Those should be removed.
Even for non categorical variables, if they have only one value to old.
That is basically not a variable.
That is basically behaving as a constant.
We don't need such variables and we will be deleting them.
Variables with low filtrate are the second point.
We saw that.
And Horsburgh Variable had eight values missing.
So we decided that we will replace these values by the mean of the other values.
But suppose we had only 50 values and the rest 450 values were empty.
Does it make sense to impute values in that case?
In such a case will not be able to capture the actual effect of that variable on the airport.
Even if you keep the variable and impute mean value in the missing places and on the analysis nearly
always, there will not be any significant relationship between such a variable and the output.
So we have the option of deleting Saitoti, able to.
Thirdly.
Remember that businesses are working within a regulatory framework.
I'm regulatory framework may not allow usage of certain variables.
For example, if you decide to build a model.
To identify credit worthiness of a customer.
This is the profile of a customer.
And your model sees that a person's gender.
All religion is a significant variable.
Now, if you base your decisions on this model.
You will be treating people of different genders or different religions differently.
This situation can be considered as one of discrimination.
And if you cannot base your decision on a particular variable, that is, if you will not accept the
result of your model basis, a particular variable.
There is no point keeping that variable.
So keep this in mind when selecting sensitive variables for analysis.
Last point is, again, trading the importance of business knowledge.
Don't just take the variable because you have data available for it.
I mean, yes, we can do exploratory analysis where without business knowledge, we try to identify
a pattern.
But when we are establishing a cause and effect relationship, try to keep only variables that make
logical sense.
To complement your understanding whether the relationship is related or not, you should use the by
various analysis.
If you think house prices will increase with better air quality, that should be reflected in these
scatterplot.
We saw scatterplot of crime did.
It was non-linear, but still it had a recognizable pattern indicating some type of relationship.
But when we plotted rainfall, it was completely, uniformly distributed regardless of race.
This is telling us that rainfall may not be having a significant impact on the outward variable.
So we have an option to delete these rainfall variable.
Also.
As you have seen, identifying, adding and removing variables is an iterative process, and we'll be
doing it even post regression analysis.
We will remove the variables we have identified using these software tools from our dataset.