In this video, we will learn about by Verrett analysis.
When we looked at EGD and backlots, we looked at individual variables and that was univariate analysis.
Now we will pick two variables and this will be called by vinyard analysis.
So when we have two variables, we look at the relationship they have amongst themselves or whether
they seem to even have a relationship or not.
There are two published ways of looking at two variable relationship.
One is a graphical way, which is called scatterplot, and the second one is tabular way, which is
called correlation matrix.
How we should use scatterplot as we should first plodder scatterplot of each independent variable against
the dependent variables.
Then we should ask, is there a visible relationship between these variables?
If there is none.
We should go back and check.
Business knowledge.
If there is a visible relationship, then we should see if it is a linear relationship or not.
If it is a linear relationship, we will straight away use that variable for linear regression analysis.
If it is some other sort of relationship, we will transform the variables so that the transformed variable
is no linearly related.
We will look at variable transformations in a couple of minutes.
So Basic Scatterplot, we will decide whether to keep, discard or transform or variable.
Next is correlation matrix coalition matrix guilty linear correlation coefficients between all pairs
of variables of data set into Purdum's.
If X2 increases, when X1 increases and x2 decreases when X1 decreases, the variables tend to be highly,
positively correlated.
Similarly, if x2 degree increases, when X1 decreases, the coalition is set to be negative.
And if the change is random, that is excellent.
X2 increases sometimes and decreases sometimes with increase in X1.
Then the coefficient of coalition will be near zero.
We will discuss this in depth in incoming videos.
Then we will bloddy coalition metrics.
We will look for the following two things.
One, if the value is low, that is near zero.
Between the dependent and the independent variables.
This will basically represent that probably there is no better correlation between the independent and
the independent variable.
And we can consider discarding that independent variable.
Second, if there is very high correlation amongst the independent variables.
So if we take two independent variables and correlation between these two us coming too high, it may
suggest that the independent variables that we have selected may not be truly independent.
And we may have to let go.
One of them, because having both in the analysis leads to a type of ethical medical insanity.
All these types of errors will also be covered in the later part of this course.
For now, we can consider that will be two moving.
Anybody will of the beard of independent variables, which show very high correlation coefficient,
say, more than pointed here in this light.
I have some scatterplot examples.
Here we take one variable on the x axis and other on the Y axis.
And each data point is plotted on the graph.
The idea of doing this is to see if there is any visible relationship between the variables.
Most of the time when we applauding scatterplot, the y axis will be taken by the dependent variable,
that is the variable we are trying to predict.
Now, if you look at the first scatterplot, it seems to be a linear relationship.
It does not mean that a single lane will pass through all points.
But overall, if excess higher values y tends to be on the higher side.
And when X increases, Y increases proportionately.
So this can be used straightaway in a linear regression problem.
But what does get applauds, the relationship seems to be not linear.
Why is increasing as X is increasing, but either faster or slower on in this right plot?
Initially when I was increasing X, Y was increasing at a larger pace, but eventually Y starts increasing
at a slower pace.
This type of distribution is offer logarithmic function.
If we move to the next graph with vias starts increasing slowly, but as we move ahead and have higher
values of X, the increase in the value of Y is higher.
This type of distribution, it is an exponential distribution.
And lastly, if you look at this distribution, all the points seem uniformly distributed on this graph.
There is no particular functional distribution which seems to satisfy the relationship between these
two variables.
We can't really say with confidence that increasing X will increase Y or decrease way.
So when there is no recognizable pattern we can do to discard it.
This is our business knowledge or we can even keep it still and let the model discarded later.
What the graph to entry when the relationship between the two variables is some other functional form
will need to transform the variable.
We'll see how to transform a variable in the next couple of late.
So now let us focus on the Diebel transformation, one type of erasable transformation is what I told
you in the previous lead.
That is, if the dependent and independent variables are not related linearly, but in some other functional
form, we can modify the independent variable so that the modified version has a more linear relationship
with the dependent variable.
Keep in mind, transforming a variable is not a mandatory thing.
We are only transforming variables with the hope that it will eventually fit the model better once we
know how to run the model.
I would suggest that you run the model without doing the transformations and then we're doing the transformations
so that you can see which of these two models fit your data better.
Now we can do other types of variable transformations also so that our variables give us more information
or they become more relevant.
One type of transformation is where we have two or more independent variables depicting similar type
of data and having similar type of relation with the dependent variable in such a case.
We can decide to take the average of the values of these variables and put it into a new variable and
use this variable instead of the other variables.
Let me tell you an example of how this is done.
Let us go back to our dataset of host pricing.
If you look carefully, we have these four variables which tell us the distance of hose from four different
employment locations.
Basically, all four of these are trying to represent the availability of job opportunities in the nearby
locations.
Probably because we think house prices are more where jobs are easy to get.
Now, since all four of these are trying to represent the same thing, having forward variables is actually
over representing this feature of employment.
Also, house prices may not relate to each one of these individually, but there is higher chance that
it will relate to a variable which represents overall ease of getting a job.
And one way of creating that could be to get the mean distance to employment locations so we can create
a new variable, get the mean of these four values for each of the relations stored in it, and then
remove these four variables from our dataset.
This is one way in which we can transform multiple variables representing the same thing into one new
variable which will later have better relationship with these dependent variable.
Next method of transformation is creating racial variables which are relevant to business.
What this means is we may have phone.
Suppose that the price of house depends on the quality of education maintained in the locality or the
city.
So we may think that probably number of schools in the region or a number of teachers in the region
can reflect this feature.
But if you think about it, quality will not depend just on the number of teachers, but their identity.
That is how many teachers, but all the students.
Also, bigger cities tend to have more population and more number of schools.
So a number of schools or teachers will not be independent in the true sense in such a case.
Therefore, in a particular business context, racial variables often make more sense.
So transform such variables to racial variables before the analysis.
Third point is when we discussed in the previous late in the next two slides, I will be showing you
some common type of relationships between two variables and how to transform one of them so that the
relationship becomes closer to linear.
So this first graph depicts the shape of.
Way as natural log of X.
If the shape of your scatterplot looks like this, ignore this scale and ignore where it is, cutting
the axis does mastership.
If this shape is similar, you need to take the log of this variable, too, instead of X, use log
of X.
If your ex has values between zero to one, it is advisable that you add a constant to X, that is,
add one to X and then take log because log of within a range of zero to one behaves in a very erratic
minute.
Once you do this transformation, this new created variable, which contains the value of Log X, will
have a morally linear relationship with the way.
Similarly, in this graph, in the second graph, Y has an exponential relationship with X in such a
case.
You need to take E to the power X instead of X.
Remember that your graph may not look exactly the same, its shape should be similar.
It may not intersect the axis at the points that we have shown.
It would just look similar to this.
But look, graph.
And the third one is of.
A higher order point.
I mean, so if it is exquisite, it looks like this.
Excuse me, has a higher slope.
If you look at it deep, the power X and it square.
They also look similar.
Does that the exponential function grows much faster than the normal function, too.
If your scatterplot is showing something like this, probably we need to take a higher order value of
the X.
So private XCOR, private excuse, whichever you think is the type of relationship with which X has
with Y.
And this slide I'm showing you the effect of putting a negative sign on a plot of y it makes if you
have a plot of X Q it looks something like this.
But if you take a plot of minus excuse it look something like this.
This plot is actually just the mirror image of this plot on the Y axis.
So whenever you negate the X and then put a function on it, the new function is just the mirror image
of the previous function where we are telling you this is because sometimes you will see a relationship
like this and you will think that it looks like a excuse.
But how do a model it does take the negative off X and take its cue.
This is how it will look.
As I told you, you just need to remember the shape of different types of plot.
And you can use that information to transform your variable.
This is a politics to Dibala minus X will be an mirrored image of this.
So it will look something like this.
The way I'm moving my Moscoso.
But if I do do the politics and then negated, it becomes a mirror image on the x axis.
So this will come down.
So this is and this is a mirror image on the x axis.
So if you see something like this, it is Eataly.
Politics does a negative in front of it.
You need not handle this.
If you just take an exponential, your model will also find that it has a negative relationship.
You need not worry about negating the function.
But if you see something like this and you have to negate the variable itself, must negate the variable
and then apply the function.
The second thing I want to emphasize is the effect of adding a constant if you add a constant to X.
It will shift this entire graph to the left hand side.
So if you are looking at a graph which has a shape like this and you want to move this graph to a little
left, add a constant.
By how much more you want to shift this go.
So if you look at excuse, it is passing through zero.
But if I add into it.
It is passing through minus 10.
So it is just shifting this whole graph to the left.
Instead.
If you add a constant to the function.
If you add 10 to do the politics, it will just push it.
Then units are able to whatever constant you will add here.
It will push the whole God does one unit up on the Y axis.
So basically, you need to remember the shapes of different types of functions and that you can use
to transform variables.
So on out, they dusted off house blazing.
We'll be looking at the relationship of crime rate versus house price, and then we'll be transforming
the crime rate very well so that it has a more linear relationship with house place.