Hi. In this video we will introduce our dataset
for this module. I chose dataset, which I created using Shar,
where I recorded all the packets on my wifi interface.
It's total of, uh, 56 million of packet,
which I recorded over a 26 hour period. It's total size. Uh,
when you put into memory is, uh, about 13 gigabytes, uh,
but compressed on disk. It takes around one gigabyte.
Uh, please, uh,
download it from provided link and extract it into the data folder.
Uh, you will see 500 packet files.
Originally I had only 142 C S V files, uh,
output from Wireshark,
but I had to process it a little bit like changing the IP address. And, uh,
I also added the ID of the captured file as well.
I also them to have 500 smaller files.
I did this, uh, processing in, uh, this file, uh,
module zero eight prep notebook. You can explore it, uh, if you want,
uh, after I finished this lecture. Uh, but, uh, the main notebook,
which we will use is, uh, this one
we can open it.
The dataset contains, uh, 14 columns, uh,
which have the basic information about the packet, like source and destination,
IP address, and port, uh, logged in, buy timestamp, and so on.
Uh, one thought, the parking file format, uh,
this format is much more efficient in terms of storage and in, uh,
input and output operations than C S V.
So the reading into dusk will be much faster.
In our next video, we'll start to work with dusk and explain the dusk graph.