Hello, all so in all of our previous session, we have learned some basics of this season, which is
nothing work, which is just like a season kind of thing, which just like a rule-based kind of thing.
And according to the conditions, you have to proceed.
And we have discovered the basics, how to how to build up.
This isn't using this and properly and information in the very first factum.
The second one is exactly the genie.
And back then you can decide that you need decks or you guilty.
Then we have also learned when to use pre and post.
Then we also have learned what is overfitting and and how how to get rid of this issue.
So in this session, we have this use case and this is exactly why I use this word.
I have this X feature where I have this Y feature and I have the Z feature.
And basically, depending upon these independent features, this is mind dependent features that exactly.
We have protected the system and that training.
So this is my training data.
So let's see what if what if you have some new entry and then according to this new entry, you have
to find something.
So it means you have to basically construct such a decision that can predict with respect to that condition.
So let's try to understand what exactly is this entropy and information?
Yeah.
What is this entropy?
What is this entropy information, Kilmichael?
So what exactly is the very force to understand?
What is this entropy?
So entropy is all about.
It's all about.
It's all about your randomness.
And you do the how random your data is, how random your data is, what I can see, or what is a probability.
What is a probability?
Probability of occurrence.
Probability of occurrence of a data in a certain space.
That's like.
That's all about this entropies, similarly, or I can see it basically gives my ability, it basically
gives some impurity inside data.
So now you will think, what what is this?
What is this?
What is what does it mean?
I didn't understand.
What is the meaning of this impurity?
Let's hear.
Let's say you have some bowlegs you have some power.
Let's say over here you have some Aleksi here you have some Baskett.
Flexi here you have three apples, let's say one eight, two eight three.
Similarly, over here, let's say you have three apples.
And what you have to do, basically, you have to pick you have to pick one item from this board and
one item from this from this basket.
Let's say let's you have to pick this.
So what exactly?
It probably will be one, because probability of picking apples on this and this would be one because
we only have apples over here.
But what if let's say you have let's say you have another scenario.
Let's say you have some analysts in Arizona that say, let's say here you have a ball.
Let's say here you have some basket hexa.
Here you have some, let's say bananas.
I am going denoted as B1 and Axia.
Here you have some grapes and here you have lettuce of apples.
Similarly over here, let's say BE1 Gevalt and Iowa.
So here, if I will ask you, in previous case, I have this one and impurity is zero.
Similarly, over here, if I will ask you, what is a probability, what is the probability that you
have to fetch apples from this and apples on this?
So you will see.
You will see there is there is there are certain jobs that the probability can be one.
It means your impurity.
It means your impurity can't be zero over here.
That's what is the exact meaning of this impurity.
So if I will talk about this entropy, it will nothing.
But it it just gives in purity inside our data.
It just gives impurity inside this data.
So let me write down our basic formula, how to how to compute your entropy for any feature which is
nothing but which denotes by E which is nothing but minus of B, I log two of B.I or if I have to expand
it it is nothing but minus B of log to of be.
Plus, couple of luck to us, you were this dude is nothing but basically fun minus B, that that's
that's what you have learned in school.
So what we will do, basically, we will compute this entropy, this entropy for acts like denote as
X and for Y and entropy for that X, E, what is it.
And after computing this, we are going to compute what is the information gain of X. Let's say idea
of X X, the idea of Y legacy idea of Z and whosoever whosever future has a highest information, then
that teacher will get selected as my Rupel.
It let me assume that it just has highest information that Z will be considered as my partner.
Then you have some conditions that said that what that's what this entropy and information will do.
So now how do computers entropy using this basic formula?
So if you will see our use case here, you have this X feature where you have three ones and zero zero.
And this is exactly my class.
So let me right over here.
Let's say I have X over here and then X I have initially two was one zero and here I have this.
And with this back to this X, I have some class, I have some class label the cancelable.
So basically I have labourer's first class, let's say.
I think this is also first class.
Yeah.
So let me let me visit first class, first class, second class, second class.
So now you have to compute entropy now.
So basically you have to compute entropy for each and everything.
So here you have to compute entropy for zero and you have to compute entropy for one aspect.
So basically what exactly it's meaning.
So it means in this let's say in this X feature, in this X feature, you have basically one and zeros
and inside one you have you have either you can consider you have class one and you have plus two.
Similarly, in the zero you have some class one and you have some class two.
So basically you have to compute, you have to compute this entropy for this one and entropy for this
one.
So what exactly is the meaning of this?
Such meaning is basically what exactly what exactly is the probability of occurrence, of probability,
of occurrence of Class one, because you will see here you have class one.
When the value of X is one, that's it.
It is the simplest way of explaining these things.
It is a simple clustering, literally.
It is a simple clustering.
So what is the probability of occurrence of Class one then?
Well, you of access one, so you have to compute polarity in such case.
So you will observe here you have two one value classes one whereas you have Dotel one is three, simple
is 280.
Similarly its meaning is similar, meaning it.
What is a probability.
All occurrence of.
Glass to read your ex is one, so what is the probability of glass to when your ex is one simple one
by three?
So you will think how I have come, why you will see here you had just just one count here.
You had just one count where this is glass two.
It means this one and I have total count this three.
So let me.
Yeah.
So here it is.
Nothing but just one by three.
Similarly over here, what is the probability that a glass is one when X is zero.
So it is nothing but my zero by one because you will notice over here because you don't have any value
where your glass is one and you have to deal with it means here you have this one and count of zero
is basically one.
So I can say it is nothing but just zero by one.
Similarly, well here it is nothing but just one by one, because here you had just count.
We are this and your glass is exactly two, which is exactly this is.
So if I have to write what does I thought the entropy of X so I can denote it as let's say E of X.
So it is nothing.
But in this case I have something called something called it is minus 280.
Love of two by three plus.
Minus one by three, logoff one bite.
So this is this is the probability.
Where this is a probability, when X equals two, one for both classes, for both classes, either either
for either for one and for two.
So it is a probability.
It is basically a probability when it goes to one for both classes, similarly for X equals to zero.
What is it?
What is your entropy, not what is entropy.
So the entropy is nothing but basically zero.
How are you?
Because it is nothing but just 051 into a Log of 051.
Plus, here you have one.
Minus, I can say minus one by one, log off one by one, so long one is equal to zero, so it will
become zero zero zero.
It is nothing but zero for four X equals to zero.
You how in Prophetess Siedel, whereas for X it goes to one for both of the classes you have Entropa
as Dismas, so this is that value that plays important role when you are going to compute your information
gain with respect to X in a similar way, you can compute this entropy for Y as well.
You can compute entropy for Y as well.
Similarly for that as well.
So let me compute, let me compute for Y so he will see a data set.
Let me, let me open it.
B So this is my wife.
Let me, let me read it to my Y is nothing but just I have double one and zero for this back to I have
some class so my class is nothing but class one.
I have class two.
So again, again I have something popular here.
So here I have some, let's say entropy of Y or I can say tropicals y y one where my this Y is one.
So what is the entropy order here.
So it is nothing but.
So let me let me write down as so your your wife has two categories, basically one and zero, you will
see one and zero.
And in this one, either you have this one or you can have this one.
This these two cases can only possible over here because I have it just two classes.
The first, the second similarly in zero.
I have this similarly hit.
I have this.
So with respect to this, this when class is first and you revised one, your probability will be nothing
but two to.
And love of Dubai.
It's I guess it's entertainment, because you will see you have two cases.
You have just two cases where we had two classes, one, and the rise of it did seem to be worth it.
In here you have zero by two or you can write it.
And that in this way to love the way to this one.
This similarly over there, you can write as zero by two because you will see you don't.
It means it is.
You do, because these two here, you can see zero by two log zero like two similarly were here.
You have to wait two because countersurveillance two and allow us to wait to see what you have to do.
You have to just just just place this value.
So entropy is like a student entropy or why is you so basically you have to entropy already.
So very first one is nothing but minus.
Two by two and logoff two by two.
Plus.
Minus zero by two, and it will be nothing, but it will become basically zero.
It is a thing, but it's just zero.
Or you can also see its entire value zero because this is also zero because it is not a lock one and
lock one is also.
You said, well, in that way it will be zero.
Similarly for this one also because this value is zero and this value is also zero.
So it means the entire entropy y here is zero in a similar way.
You can also compute you can also compute for that as well.
That we compute for that is that we compute so far that you have your data set where you have one disk
on this one and all these things can be done.
All the things with respect to that, you have one zero one zero and you have a class.
You have a class level and a class label at one one this second and the second is so you have all these
class labels.
So basically.
In this, I can see and said, you have something, one, you have some things, you basically you have
two classes and inside this one you have something.
Class one, you have something.
Class two, similarly inside zero, you have something.
Class one, you have something.
Class two.
So that you have to compute.
You have to compute.
So with respect to this, what is the probability of occurrence of class one venue that is one sixth,
the probability is nothing but one, two by two local one, because you will see it has just as one
and two total call the stool.
Similarly, here it is nothing but it is just one my two logoff one by similarly over here it is nothing
but one by two local one by two.
Because with respect to zero you have just one count where your classes to a total count this this much
and this much similarly count this.
Similarly over here you have one by two and log of.
So what you have to do, you have to just compute it or you can say you have to just place over here.
Similarly you can see.
Just drop off and drop your said when you when you said the value is, let's say let's say with respect
to one, you have to compute it.
And here, with respect to zero, what you've said that it is zero.
Here, I have to say, it is nothing but just minus one by two blocks of one by two.
And here you have to also say it is nothing but this plus minus one by two, log off one by two.
So does nothing.
But it will just give me a value as one.
Similarly over here.
It is nothing but similar.
Similar to just similar.
This one, and I can do this one, so it is also gives me a value.
Hesla So finally we have computed and Trappy for X, Y and Z.
So the last session we are trying to learn how to compute this information gain from this entropy that
we have computed over here.
So that's all about the session.
Hopefully you will learn and understand all these things that I have explained over here in a simple
test to it in a easiest way.
Well, thank you, guys.
How nice to keep learning.
Keep growing, keep practicing.