Hi.
In this video we will learn how to plot our data using data shader.
Data Shader is an excellent library, which allows us to plot, uh,
even, uh, multiple billion of data points very effectively into one chart.
We have seen that, uh, when we want to make a scatter plot,
where are too many points plotted to gain meaningful insights might be a little
bit tricky. Uh, now let's make a scatter plot of, uh,
source port and, uh, length column, uh, like in this cell.
We can do it this way. Uh, first we have to make a canvas.
Uh, this is the place where all the plots will be plotted. Uh,
then we specify that we want to plot the points.
Our source is the data frame. We specify the X and the Y variable,
and we want that. Uh, when some points overlap,
uh, we want to calculate their counts.
The most time consuming operation is calculating the canvas dot point object.
Here in the next cell,
we color the points using default color map and with equalizing histogram.
Color mapping, uh,
this type of color mapping attempts to use the full range of colors
equally often to highlight both large and small features in the data.
And, uh, here is the result.
Now we can do the same, uh, but now let's do it, uh, on the length and, uh,
uh, T c P length column
here. Uh, I also used a different color map, uh, called fire.
And here we see the relationship between these two variables. And we also see,
uh, this area is, uh, more orange and yellow. Like that means it has, uh,
much more higher density than the rest.
Unfortunately, in data shader,
we cannot plot access information or legends. Uh, but, uh,
it is anyway very useful. In particular in plotting maps. Uh, for our use case,
uh, the access info and legend might be added, uh, via another package,
uh, called to use, uh,
which has the advantage of setting plotting backend like bouquet, mud,
plot leap or plotly or others.
Here we create first the point element, uh,
so hollow use novels. We want to plot a scatter plot.
Then we specify the options for the hollow use plot.
We want the color bar in the bottom position. Uh,
we want the fire color map, uh, color mapping, uh,
width and head of the plot. And then we color ize,
which is essentially, uh, conversion the of the data points to pixels.
And that's it, uh, on the whole data frame.
It'll take some time to calculate, uh, usually around, uh,
three to seven minutes maybe. Uh,
so the charts, uh, is, uh,
plotted via boke and is also interactive. Uh,
you can basically zoom, can, um, hover over the chart to see,
uh, how many points, uh, there are and so on.
Okay, uh, this is it. Uh, for this module. Uh, we can close the dust cluster,
uh, with the command client close.
Now, uh, let's summarize this module. We have learned, uh,
briefly what are the strategies on how to deal with data sets,
which are two big, uh, to fit into our ram.
We have learned also what are the differences and common features between, uh,
dusk and pandas data frame.
We learned how to start local dusk cluster on our computer and how to
perform our calculations on it.
We have learned also what is, uh,
the graph and how to read it.
We have learned how to use the D dashboard. And lastly,
we have learned, uh,
how to plot big data in our dust data frame using data shader and all of use.
Next, we'll start our next module, uh, focused on machine learning.