All right, my friends.
Are you ready?
Are you ready to build that dendrogram and use it to find the
optimal number of clusters?
Here we go.
Let's implement the solution together.
So let's start by creating a new code cell.
And now what will be the first step?
Well, as usual you know we want to implement this efficiently.
So we're going to use a function.
But this time exceptionally this function won't be imported from scikit learn.
It will actually be imported from another very popular library in data science
I would say, you know, it is in the top three most popular libraries.
I put scikit learn first, of course, and TensorFlow for deep learning.
But that other library is SciPy.
SciPy contains a lot of great tools when building
machine learning models and well for hierarchical clustering.
It indeed contains a function which is called dendrogram
and which will return the dendrogram itself.
You know the plot of the dendrogram.
So let's do this.
Let's directly import that, you know, module first that contains this function.
And as we said this model is taken first from SciPy,
then from the module cluster
and then from the submodule hierarchy.
Right. Google collab guesses it perfectly.
The other way to write this is of course to say from SciPy import
cluster dot hierarchy.
So that's just another writing.
And then we're going to add of course a shortcut to this.
Otherwise we would need to call all of this again.
And the shortcut will be as c h right.
For SciPy cluster hierarchy okay.
So that's the module which contains the function we want to use
in order to build our dendrogram.
So now next step.
Well the next step is to use that function which we can now access
from that hierarchy module which we just imported.
And since this function returns directly the dendrogram itself,
well we are going to create a new variable here which we're going to call.
Then draw grab as simple as that.
And this dendrogram variable will be the output
of this dendrogram function which we're about to use
from the hierarchy submodule by the cluster module from the side by library.
All right. So let's do this.
Since this function belongs to all of this here in the hierarchy module.
Well we have to call first the shortcut leading to that module.
And from which.
Now we can call this then draw gram function.
Perfect. Thank you so much Google Collab.
And now indeed in the parenthesis we have to input some arguments.
And now you can't really guess what the arguments will be.
So I'm just going to write it and then I will explain what this means.
So first we actually have to call quote H.
Again you know the hierarchy module from the cluster module from the SciPy library.
So from which this time
we're going to call another function which is the linkage function.
And this linkage function will take as input two arguments.
First well your matrix of features inside which you want to identify the clusters.
And that's of course x.
And then the second argument is the clustering technique.
And in hierarchical clustering the most recommended method
and the one that brings the most relevant results.
And the most relevant clusters is the method of minimum variance,
which is a technique that will result in having clusters inside
which you know the observation points.
Don't worry too much.
You know, have among all of them a low variance.
And that's what it means.
You know, the method of minimum variance consists of minimizing the variance
in each of the clusters resulting from hierarchical clustering.
And so this is really the method that I recommend.
And speaking of this method,
that's exactly the next argument of this linkage function.
We have two input here.
And the name of that parameter is method.
And the name of that minimum variance method is not called minimum variance.
But ward Ward.
You can actually check this on Wikipedia.
There is a whole page on the ward.
And you will see that indeed it
consists of minimized the variance inside your clusters.
All right.
And that's it for the whole dendrogram function here it only expects one argument
which is basically the method you choose
for your clustering that you link to your matrix.
If you just x in which you want to identify the clusters.
So that's all you need to input here in this dendrogram.
And there you go.
This will already return the dendrogram itself.
You know the plot of the dendrogram.
But as usual we want to make it nice.
So we're just going to add a title an x label and y label.
And then we will show it.
And now I will teach you
how to read the dendrogram to indeed find that optimal number of clusters.