Hello and welcome back to another class of our course about the complete introduction to the science,
basically in today's class, we are still talking about some statistical concepts and one of the most
important statistical concepts that is very, very, very used in data science is called decision trees.
So once again, we can make a full 40 hour course about decision trees.
But my goal here is not to create a full course about decision trees and talking about all the mathematical
concepts that are around decision trees.
But just to give you guys a brief introduction so you will understand that it works well, that it's
really, really present into their science machine learning as well as all the things that are around
those concepts.
All right.
So basically, what exactly is decision tree?
So if we talk about ourselves, we have a brain and let's say we, for example, want to make a decision.
So our brain is a huge computer.
Let's say, for example, our brain wants to make a decision so our brain will automatically create
a decision tree.
Once again, it's not going to be like it looks like this is going to be more different.
And, well, the brain will say, OK, if I make this, I will have this outcome or if I make this,
I will have this outcome.
For example, let's say you have the choice between eating healthy and eating a pizza, for example.
So your brain will say, OK, if I eat healthy, what will happen?
So the outcome, for example, if I eat healthy, I will be well, I don't know.
I will build muscles if I build muscles.
Well, this will happen if this happens.
Is that good or bad?
OK, it's good.
OK, if, for example, I eat pizza, what's going to happen?
OK, I will be satisfied right now because I just ate pizza and once again I will be happy.
But what will happen in the long run.
OK, if I eat pizza I will be happy.
But after that I won't be happy anymore because I won't be in shape, et cetera, et cetera, et cetera.
So basically how we can see this reentries, those are simply diagrams or charts that people use to
show statistical probabilities or to determine the possibility for an action to happen.
So basically, you can see it just right here.
So once again, I tried to find a very simple decision tree.
So right here we have an example.
So let's say, for example, A is higher is lower than B, so if yes, then B is lower than C and etc.
And at the end, this would be our question and we will be able to find out an answer that looks something
like this just with.
Yes, in those.
So at the end with a is is smaller than B, if it's smaller than we can say at the end that A is smaller
than C, it's smaller or equal to B or C is smaller or equal to eight and smaller than B.
And same thing right here, for example, B is greater than eight.
This could be our outcome.
So the main goal right here is really to go from having a question that we ask ourselves to an outcome
that we will have at the end.
So basically, the best outcome would be one of those.
And the best outcome usually comes with.
Well, there are mathematical formulas that we will talk about a bit later in this course will not talk
really in depth about them, but just to give you guys, well, an introduction to this.
So as they said, you will have your well, you will have this that needs to be resolved.
And it's resolved by using mathematical formulas and by building up decision trees.
So like here, for example, we have a decision tree and the decision at the end is done with the mathematical
formula.
For example, we have entropy that will be calculated and we'll have some other things that will be
calculated as well.
So if we stay in decision trees, once again, we can have some examples right here that are found.
Once again, those decision trees examples are pretty simple.
Just for you guys.
Understand what a decision tree is.
So right here, for example, is a person fit?
So if the person is lower than 30 years old, if yes, does it a lot of pizza?
Yes.
Then the person is unfit.
If no, then the person is.
That same thing right here is the pea is the person higher than 30 years old?
Well, no.
Is the person exercising in the morning?
Yes.
And the person is.
If not, then the person is unfit.
Once again, this is really, really simple.
Even the decision tree right here is pretty simple.
Usually you have way more variables and it's way more calculated.
It's way more hard to calculate everything and find out the best combination.
But once again, this is just a small example of what you can find if you work with decision trees.
So basically your brain does this automatically so you don't even think about it.
Your brain does this automatically when you have to take a certain decision.
It could be basic decision.
For example, I'm walking or I'm driving.
I will turn right or left, or it could be more complex decisions.
For example, when you are sitting at a certain exam and you need to write down an answer.
Right.
So the mathematical aspect of this, basically, there is a lot of variables that can be calculated.
But personally, I think there is two very important formulas that we will talk about right now.
Once again, you don't necessarily need to understand them 100 percent.
This is just an introduction.
So the first one would be the formula of entropy.
And basically entropy will be used to calculate the homogeneity of a sample or the integrity of a data
base.
So basically, basically, usually entropy is is used when you work with the databases and you build
up your decision tree with those databases, where you need to understand is that if entropy is zero,
the sample is homogeneous.
So like this, for example, and if entropy is one, it's equally divided.
So everything is equally divided.
What's important also to understand is that entropy is somewhere between zero and one.
It could it can be higher than one.
Usually it's between zero and one.
Also, entropy is very important to make the second part of our calculation because it controls how
a decision tree will decide well, how a decision tree decides to split the data and will have an effect
on how the tree will grow is the basis.
So basically, this would be for entropy, so entropy is the first part of the calculation, and after
that we have another part that is very, very important, which is the information game.
So entropy help us after that work with information.
Information gain would be another mathematical calculation that we will make to be able to choose the
right path to take inside of our district.
So here is our information gain formula.
Basically, it looks a bit complicated, but it's pretty simple.
Right here.
We have our complete entropy.
So basically the whole entropy of the whole decision tree and here will have, for example, the entropy
of the left side and the right side, for example, usually the calculations a bit more complex because
you have the sum of all entropy.
But and this would be the entropy of the whole decision tree.
But once again, this is just one left and one left and one right side.
What does information gain?
It will give us how much information a certain variable give us about the outcome and to interpret it,
it's pretty simple.
So a high information gain allow us to create a decision tree.
There will be more representative of the results that we expect to have.
So basically, if we come back to our decision tree right here and let's say this outcome right here
has an information gain of zero point nine.
Well, this would be the outcome.
This would be the outcome that we will look for to basically the higher the information gain is, the
higher will.
Well, this would be the decision part of the decision tree that we will take because the results will
be more representative of what we are looking for to have added.
So besides that, those are the two formulas that are pretty important.
So we have the entropy and information gain.
But in all this course, what I want you guys to understand is that the concept of decision tree.
So this is a really, really important concept to understand the mathematical part.
Once again, you are not professional right now and it will take time for you to understand that as
a whole.
But the concept of decision, as I said, it works pretty much as the human brain.
So basically, when you want to make a decision, you will automatically create a decision tree inside
of your head.
You don't even need to think about it and that this works pretty much the same way.
So, for example, if you are building up a certain algorithm, that will not go in.
But let's say, for example, you are building up a machine that will think by itself what you will
add to it.
You add an algorithm that will that will learn by its experiences.
So it will try, for example, something.
So I don't know.
You want to you want to create an algorithm that will identify faces.
So you will give you will give to this algorithm out on the one hundred thousand faces and this algorithm
will work.
Well, we'll see how each piece, what each face looks like and that it will learn by itself.
And after that, when you will give a face this algorithm and tell him, OK, is there a face yes or
no?
And what the algorithm will do, it will build up a basic decision tree to be well, it will be more
complex than the basic one to be able to tell you at the end.
If this is a face based on this, based on variable, one, two, three, four or five.
So it will test some some things.
For example, number of pixels, number of well, a lot of things that will test a lot of things.
But at the end of the day, it will have a certain decision tree to be able to tell you, yes, this
is a face or no, this is not a face.
This is I don't know, for example, this is a water bottle, for example.
So I hope you guys understand that the whole concept of decision tree and the two formulas that comes
with.
So that's it for this class, guys, Insu in our next class where we are still talking, where we will
still talk about some other statistical concepts.