In this session on bigoted analysis, we are particularly interested in finding the relationship between
two independent variables in the previous sessions I talked about.
Dependent variable being dependent on many independent variables.
Please note what I just said, I'm interested in finding the relationship between two independent variables
itself.
If there is a relationship between two independent variables, I can predict one independent variable
from another independent.
Very.
Ideally, I don't want this kind of relationship among my variables.
As far as developing machine learning model systems.
This is what is known as multipolarity, that is one independent variable can be predicted from another
independent variable itself's.
OK, so let's see all the techniques that are available in my video analysis.
So by immediate analysis, helps find the relationship between two variables, right?
I know I said we are more interested in finding the relation between two independent variables, but
why we need analysis.
Broadly speaking, can be used to find out the relationship between any two variables.
Right.
So we are looking at both association and this association, meaning there is a relationship, there
is no relationship that can be a positive relationship, that can be a negative relationship.
All the possibilities are endless.
OK, and by various analysis, can be performed for any kind of combination of variables.
That is, one variable could be continuous, another variable could be categorical, like both the variables
could be categorical.
Both variables could be continuous.
There are different types of techniques for different types of combinations.
We are going to see all of them in this issue.
OK, we will first start with the relationship between two continuous variables.
That is two members, so to speak.
Remember, I gave the example, right?
Number of marks of the number of hours I study.
That is an example right here.
The primary technique that we'll be looking at is what is known as a scatterplot, OK?
It graphically shows whether there is a relationship between two variables like you can you can be sure,
Lucy, as I mentioned, it can be a positive.
It can be a negative relationship.
That can be a relationship, that may not be a relationship and the relationship can be a linear which
a straight line can be fitted.
OK, what is called is the line of best fit, OK?
Or it can be a non-linear relationship, like a polynomial kind of relationship, OK, or a logistic
relationship.
So Scatterplot shows the relationship between two variables, but it doesn't tell you the strength of
the relationship.
OK, so for this, we use what is known as quotient of correlation.
It derives from the scatterplot because it's got to be food, what is known as the line of best, OK,
using the line of best food, you can find out what is the extent of correlation between two continuous
experience for this?
As I said, the use coefficient of correlation.
So correlation there is between minus one and plus one.
OK, one indicates.
Minus one indicates perfect negative correlation, plus one indicates positive linear correlation,
Zettl indicates no correlation.
This is minus one.
OK.
The point is many times, you know, you won't get this minus one, plus one, or you will get values
that are closer to one.
So which means in order for this point nine six, if it is closer to minus one.
So it means it's a nearly a perfect linear correlation.
You can even get a value of eight point six.
That means positive correlation to some extent.
So those are the judgments, you know, you need to make based on the quotient of correlation.
Right.
So this is the scatterplot, right?
In this example, I have drawn this kind of prop for a relationship between applicant's income and loan
amount.
Remember, these are two different variables.
We can find that there is more relation between the two, right in the other scenario, the applicant
income KOPLIK and then that is also being compared.
But see, the scatterplot divide is scattered all along and quite literally, it is scattered all around.
That means there is no relationship like you can plodders, whenever you have two variables and you
want to find the relationship immediately.
Plot scatterplot right.
A very, very simple tool to ascertain the relationship between any two variables.
And you can be sure you see if there is any relationship or not.
You can also see, is it a case of linear relation or is it a case of a polynomial or other types of
relationship?
All those things can be seen here, you know, using this scatterplot.
Right.
And as I discussed the earlier, I use quotient of correlation.
That is the extent of correlation.
That is the numerical measure of correlation is the correlation coefficient.
But in this case, you are taking applicant's income and loan amount by we are getting a correlation
coalition of point Feiss, which means that is a moderate relationship, a more positive relationship
between applicant income and corporate income.
It is a negative correlation, but it is a weak correlation.
Please understand this is closer to zero, right, than to minus one hour plus one.
If you look at the relationship between applicant income and long term, it is in fact closer to zero
point minus point zero.
For that is a little weak relationship between two variables.
Are you getting what I'm what I'm trying to say?
Right.
Scatterplot helps him to visually see the relationship.
Right.
Whether that is a relationship or not, who have Schindall correlation, gives us a numerical measure
to find out the extent of correlation.
That is that between two variables, I can use this concept to determine whether that is in any relationship
and decide my course of action.
OK, what is the course of action?
Ideally, I don't want correlation between two independent variables.
I will remove such variables.
Right, so that is the primary action.
Now, the other typists, categorical and categorical, if the two variables are most of them are categorical
in nature, we use what is known as a chi square test, right.
It is used to derive the statistical significance of the relationship between two categorical variables,
right.
It is based on the difference between expected and observed frequencies in one or more categories.
Right.
So categorically, it can have multiple levels of categories.
Right.
We create what is known as up to a table expected and observed frequencies.
Right.
It basically returns the probability for the chi squared distribution.
Basically returns the probability if the probability is zero.
It means that both the categorical variables are dependent probability of one to choose.
Both are independent, right.
So that's closer to zero.
Closer to one.
But we are looking at a 95 percent confidence level, which means that you will take a P value of point
zero probability value of one zero four if it is less than one zero five.
It means that there is a relationship between variables.
My.
Please note, this is what we will be using for.
Finding out the relationship between two categorical variables, it chi square test is what you're going
to use to get to understand this, OK, because we'll be using the concept of hypothesis testing.
I frequently talked about dependent variable and an independent variable rate, the high school mathematics
concept.
This is also something that we learned in our high school and probably colleges.
Right.
So hypothesis testing is the concept that I'm trying to derive an inference about a population based
on the information I have about sample.
Remember, the population was a sample concept that we discussed in the earlier session.
Like I am doing an opinion poll by taking feedback from some of the groups.
Right.
From some of the people who are eligible to vote.
Right.
I'm not going and taking feedback from all of the people who are eligible to vote.
I'm only taking feedback from some of the people and hence I'm interested in that particular particular
party is expected to win the elections.
Right.
So how can I conclude that the responses of a few people are.
Good enough to determine the outcome of an election that more people are employed.
So that is where the concept of hypothesis testing, cos I am trying to draw inferences about a sample
based on the performance I'm trying to draw on in terms of the population, based on the results of
a sample, based on the sample.
I thought about the entire population for this.
I'm making use of the concept of hypothesis testing.
Right.
So if you see the photo that is there, I apply fertiliser on the plant goes.
Right.
So what is the hypothesis?
The hypothesis is application of fertiliser increases plant.
Right.
So that is stated as an alternate hypothesis.
Alternate hypothesis is what we want to prove.
Right.
What we don't want to prove is data.
Neither my life, what the application of fertiliser does not increase plant growth.
Right.
There is no impact.
Right.
So an alternate hypothesis is what we are trying to prove.
I'm not like what?
This is the opposite of that, right?
We actually don't try to prove hypothesis.
Rather, we try to either accept or reject the argument hypothesis.
So that is the way the concept of hypothesis testing works.
Right.
So in our case, the alternate hypothesis would be that is the relationship between two variables.
It's not there is no relationship between two.
It is like if the P value is less than one zero five.
We accept that.
We conclude rather than is the relationship between two variables.
Is this clear?
Is this clear to everyone?
Let's see some examples here.
I'm taking two variables, right, the education level and the whether the individual is self-employed
and there's no kind of scenario.
Right.
So please note that in all, if there are more than two levels, also, you can still use Bisquick,
not just when I have to live.
It's like education, graduate or not graduate, self-employed person.
But if I have a variable in which more than two options are it, then also quietist can be used.
Although the example that I showed you that only two possible outcomes for each of these variables might.
Two possible levels here.
The alternate hypothesis is education and self-employed are not independent.
Look at the way I am structuring this.
I'm saying they are dependent.
So I'm essentially saying they are more dependent.
Like the null hypothesis is they are independent of one another.
There is no relationship.
And the chi square value that I'm getting here, OK, is point zero to three six.
But I am more interested in the P value.
Right, that is what I'm interested in.
So the p value is point eight, seven, seven, eight, right?
That means.
The value is greater than one zero five.
That means we are accepting the null hypothesis.
That means there is no relationship.
In the case of let's see, one more example, education and property.
Right, so here also the P values turning out to be greater than one zero five, that means there is
no relationship between these two buildings.
So whenever we have categorical variables, we use the concept of high school year.
The primary metric that we are looking at is the value it is.
If it is greater than one zero five, we can conclude that there is no relationship between the two
buildings.
So for this, we are using the concept of hypothesis testing to explain a little earlier in the session.
Yes, now we are going to look at the relationship between categorical and countries, right?
So for this, we are going to use the test detector on.
Again, these are coming from statistical concepts, right on all counts whenever we are looking at
more than two groups that are more than two groups.
Right.
Whereas if we are only looking at comparing two groups, we use the test already.
This both are in fact, similar.
OK, but it is used whenever the number of observations is less than 30.
Right.
Obviously, in most of the cases, the number of observations, the number of data, the number of historical
data that we are taking up for building the machine learning model will be more than the people invariably
will be going for a test.
Right.
So what what do I mean by two groups, which is what the categorical it is?
Right.
I have two variables if I want to compare more than two.
We are looking at Kanawa.
Like the example that you're going to see are only to fight, but if you have more than do, you can
definitely use an all.
Right.
So here also we are going to primarily look at deep value.
It is a P-value that is something that is of importance to us in this case, we are looking at if there
is any relationship between applicant income and loans to.
Right.
This is a look at this.
We are computing V Z, right.
And they are using the Z test.
Right.
And we are computing the probability that is in one year.
Also, we are stating we now like lotuses and alternate hypotheses, alternate hypothesis.
That is the relationship model, that is more relationship.
The P value comes to be greater than point zero five.
In fact, it is closer to one.
Right.
We can very well not reject the null hypothesis.
Remember the way I am stating this is the way the world of hypothesis testing works.
I either accept or reject the null hypothesis.
Right, which is there is no relationship here.
I do not reject.
OK, which means I accept whatever my hypothesis tastes like, but if you want to understand it, simply
don't bother about the Amala argument by Waters's right.
If this is confusing to you, just look at the people.
Just see for the P value is less than or greater than point zero if it is greater than one zero five.
That is that there is no relation.
OK, nothing to worry if there's less than one zero five that is.
And you need to worry.
And the course of action is you remove the variable.
Right.
That is involved in the.
Relationship like I do when there is a relationship between two independent variables, you want ideally
remove one of them like so that's a you take, right.
So a concept that is related to this is what is known as multipolarity.
Right.
And I predict an independent variable from another independent body.
So it is like, for example, can I predict the outcome based on the applicant's in?
Like, if such a relationship existed, it is definitely a big cause for concern.
And the previous we saw there is a relationship, but here I'm going one step further, right?
I'm checking if an independent variable can be predicted from another independent body.
This is known as multicasting inheritance for those fathers would be using the concept of various variants,
inflation, factor of value.
And if you use this.
Preloaded library that comes with Python, right, you can find out valiance, inflation factor in a
jiffy.
Right.
So see the readings inflation factor for each of these independent variables.
There are some there are some variables where it is higher, like six, 10.
Right.
Many variables.
Many of the variables in it is not so high.
So obviously, our interest level is higher than the average office.
Higher.
Right.
Anything beyond fine is a cause for concern.
But if it is greater than 10, you must do the more the MultiKulti and D right.
Remove some of the highly correlated independent variables, which is the loan amount to multi-client
feeling that right.
You can also combine independent variables such as you can add them together.
You can add, let's say, for example, applicant's income and corporate and income.
Right.
That is one way of looking at it.
Or you can perform an analysis that is meant for highly correlated variables such as principal component
analysis and partially square regression.
Those are techniques you can adopt.
But the one that I have found the best uses in already more than.
Take a call and remove it all, you can add them together, right?
These two have worked very well.
So what is the primary action that you will be taking, if that is multipolarity, that is ViiV is greater
than 10 remolded.
So these are key, and I also don't want to highly correlated variables.
I mean, we saw two guys square regression, right, which is the scatterplot and Zetterstrom.
That also indicates that is the relationship between the two.
I don't want to I don't want any relationship, especially a positive relationship between a positive
relationship, between two independent variables.
Right.
So I don't want such variables in my data set because they will definitely pull down my forecast accuracy.
So that's what I wanted to cover in this session, will be seen how to handle missing values, what
what actions you need to be taking in the next session.
OK.