Hi. In this video we will learn how to manage Jupiter lab extensions
and we will try two of them, variable inspector and a link extension.
When
we go to the extension step, we can click enable.
We will see what extensions we have installed and uh, order, uh, which,
uh, we can, uh, install and discover.
For example, I have already installed plot Extension,
extension and others.
And let's look for the variable inspector.
Feel free to explore other extensions,
but be aware that these are third party extensions and you might encounter
issues while using them. Uh, some extensions might not even be listed here,
but had to be installed like a regular Python package.
Now let's install this variable inspector. The first one.
Now we see that a build is needed to include the latest changes.
Click, rebuild and wait for a few minutes, 10 minutes or so. Uh,
this will take some time and the Jupiter lab will seem like, uh,
nothing is happening, but be patient. Uh,
we can monitor the process, what is happening
with the top command.
Okay, rebuilding Jupyter Lab, uh, has been done. Uh,
click save and remote.
Let's go to our notebook.
Right click on the notebook. And, uh, here is Open Variable Inspector.
Let's arrange it this way. And you see that, uh,
when you around the notebook,
we'll able to see all the variables created in the variable inspector.
We see their name, type tape and also content.
And you can also easily delete the variables by clicking on the cross.
Now, uh, let's try the other extension, a link from Mackin Rocks.
I found this extension just recently and I was waiting for this kind of
extension for a very long time. Uh,
so I have to share it with you because it adds a lot of benefits.
Maybe you have already noticed that the notebooks can be sometimes very messy.
Table of contents help us to structure the notebook,
but when you are thinking about the codependency with this,
I mean the dependencies between code cells in the notebook, it doesn't help
frequently when working with the notebook,
you do an extensive analysis and after that,
you build the model and try also here different approaches.
Then you continue fitting the model on the winning approach. And,
uh, when designing the notebook this way you have to know, uh,
which parts of the notebook must be executed sequentially and other,
which can be executed in parallel and do not depend on other cells.
In this help, exactly this extension,
it allows to create code pipelines where a pipeline consists of
multiple cells with different branches,
different branches of the pipeline can be executed in parallel.
Steps of the pipeline can be cached, saved and shared.
A whole pipeline or parts of the pipeline can be executed on remote servers.
And also you can set up parameters and optimize them.
Using inbuilt optimizer, you can do everything,
which I mentioned and nicely, visually or automatically create a diagram.
And also you can access, modify, and create these pipelines,
not only directly in the notebook, but from Python code as well.
I'll show you the basics, how it works in our notebook. Uh,
first let's install it. We
have the instructions here, how to install it,
and we see it has to be installed as a regular Python package via peep.
Let's go and install it into our base environment.
Okay, the installation was successful, but we see here, uh,
that we have some dependency conflicts. Uh, but let's try now our extension.
Anyway, our base environment is really big. Uh,
so if you experience issues in other features, uh, like, uh,
when using widgets, exporting the notebook, using voila orders,
I recommend you to use other Jupyter image where the base environment is more
lightweight or create your own Docker image by yourself.
You will have the skills or install Con and Python directly into our machine and
install packages, which you want and need without Docker.
Now let's restart the container.
I have restarted it on another screen,
but now it should, uh, run.
Okay, now it's recommended to rebuild the Jupyter app again. Uh,
so let's click build
and let's wait.
Okay, this being a little bit shorter now. Click seven, reload.
I have already approved key, uh, so I will just input it here.
Now let's open our notebook in the module five folder and click on the lean
extension here.
And now let's create our pipeline. Uh, it's uh, very easy.
You see that each of the cell has a plus button here,
and when we click on the plus button,
we can specify the component name and its parents last name.
This one packages,
it has no parents.
This one might be debugger test death
also without any parents.
This one has, uh, the parent, uh, the debugger test, test
Pre-process data has the parent, uh, packages,
target class distribution
has the parent, uh, pre-processing data.
This one as well,
the Plotly bar chart.
It has parent the target class distribution cell
and test split.
It has parent, uh, reprocessing data,
logistic regression as parents. Um,
trend test split
confusion matrix,
cumulative gain plot.
We'll do the same for the random forest classifier.
I have to edit this one.
Fusion matrix, uh, cumulative gain plot.
Its parents are only R F D.
And finally, savings scores to disk. It has parents,
uh, all these, uh, parcel cells.
Okay, now our pipeline is ready. Uh, let's try it.
We can run all components.
Perfect. Everything is working correctly.
We can now run only part of the pipeline. For example,
this branch to
we see that only this branch has been executed.
We can set up the execution in parallel.
In most of the cases, uh, the execution of the components will run faster.
You'll see that they are executed, uh, in parallel. Now,
we can also cache specific components.
This is very useful because, uh, some pre-processing steps or, um,
training of the model.
This might be very useful because some pre-processing steps or fitting
the model may take, uh,
too much time when you right click on the logistical regression,
enable component caching
and also the random first classifier.
Now the cache is being filled,
and when we run it next time,
it'll be much faster because only this cells will be executed.
Yes.
Okay, this is it. I think you have the idea.
What is this extensions capable of.
I highly encourage to you to explore it more. In our next video,
we will see how Jupyter Lab Collaborative editing works.