Hi. In this video we will learn how to work with cells and how to execute code.
Now let's create a new empty python notebook.
We are now in the edit mode of the first empty cell.
We can recognize it by this green border and also by this icon.
When we press escape, we'll get into the command mode
and the border is now blue Press enter again,
and we are in the added mode again.
Let's import the pandas package with
shift enter, you execute current cell and move into the next cell.
Now let's process the variable names.
We have now all the lines in the file in a list,
and we are interested only in the part with the variable names.
And these are starting with, uh, the list index of 33.
Now let's extract only this part with the variable name. Uh,
this is the part before the colon.
And let's create a new variable for this called call names
alt Enter. You execute current cell and a new MT cell below will be created.
Now we have one thing missing, and this is the last column, uh,
which basically should tell us if the email was marked as spam or not.
So let's append this to this list.
Okay, and this is exactly what you wanted.
Next, I would like to show you, uh,
some useful shortcuts on how to word these cells.
Uh, we can, uh, move the cells up and down. You just select, uh,
thumb cell one or more cells. When you want to select more cells,
just press, uh, shift and use your arrows or mouse.
Mouse. And with this button, we can move it up or down.
Now we can, uh,
do basic operations with the cells like copying them the shortcut.
For this C in the command mode, press C
and now press B. To paste it,
we can cut the cells
and PIs them below.
Uh, when you press double D, uh, the cell will be deleted
when you press A, a new T cell above will be created
when you press b and t cell below will be created.
And one useful thing is, uh, to change the type of the cells,
uh, we will use only markdown or code.
The shortcut for this are m for markdown or y for code.
We can also execute, um, whole notebook, for example.
Now let's open the command palette with P
and we can run all cells.
We can execute, just, uh, just cut.
We can execute just parts of the notebook. Now, for example,
let's execute the part, uh, below the current cell.
Open the command palette.
Give your option to run all cells about or run all cells below.
We also see here on the left this number,
and this is basically the order in which the cells were executed.
Now let's read our data file into pandas data frame.
We can do it very easily using pandas, uh, method called Read, C s,
V,
and we can print the first few rows with the head method.
Now let's add the column names to this data frame.
Okay, we have now our data frame prepared just to check, uh,
what is the shape of this data frame.
Our data frame has 58 columns and 4,601 rows.
Let's do the correlation analysis of our dataset. What is correlation?
Correlation measures the strength of a linear relationship between
variables. It can be in a range of minus one,
and one value of one indicates strongest positive relationship.
That means when one variable increases, the other one increases as well.
Zero indicates no relationship at all,
and minus one indicates negative relationship.
That means when one variable increases, the other one decreases.
To get correlations for each pair of the variables is very easy. In pandas,
we just use data frame dot core method.
We have now our correlation data frame, but let's assign it to a variable
to make some conclusions when seeing just the data frame is a little bit
tedious. Uh,
but we can make it much easier to read using a heat map and
we can make it with plotly expressed. But first,
we have to install the plotly package into our base environment.
Let's open our terminal.
Okay, Plotly is installed. We have to import it.
We will use Plotly express for it.
And this is the syntax to plot the heat map.
Now we have overview of the whole dataset on
X axis. Uh, we have variable names, same as on the Y axis.
In the middle we see a yellow line,
but this is because the correlation of a variable of it itself is one.
When we hover over the chat, we can see a popup,
which tells us the exact names of the variables and the correlation value.
We see that the correlation values are shown as a color gradient.
Uh, yellow means strong positive correlation,
and the dark blue are negative correlations.
Ly charts are interactive, so we can zoom in.
Now let's, for example, zoom in into this part, uh,
of the dataset where the correlations are higher,
and, uh, how to interpret the correlations in our case. Uh,
when the correlation is higher between, uh, the occurrences of these two words.
For example, the occurrence of word laps and telnet.
Uh,
the higher occurrences of one word means higher occurrence of the other word in
the email as well, but there is no other relationship.
We double click, uh, using Zoom out.
In our next video, we will learn how to work with markdown.