Hello and welcome to the very first session of our New York Trip Data Analysis Project.
So this is exactly the very first session of our project.
And in this session, we have this assignment to which we have to deal with that.
So the very first problem statement is exactly we have to load and collect our entire data so that we
can perform some sort of analysis on our data and then we will end up having some conclusions from our
data.
So let me minimize this and let me open my chapter notebook.
And this is exactly my Jupiter notebook.
Very first, I have to import some basic modules like what is bonders number and all the basic modules
that will definitely help us to create some beautiful data, to manipulate data, to analyze data in
much more efficient way.
So let me very first import my phone numbers, which is extensively used for data manipulation, data
extraction, data filtering, and much, much more so that if I'm just going to import my bonders and
I'm going to create its allies as peaty after it, we have to import my nonmarital python, which is
exactly my name by which is extensively used for numerical computations after we have to import some
data library like matplotlib dot plot and I have to create its Eliade BLT.
After that, I'm also going to import my Seabourne module, which definitely helps for battle validation
for user friendly visualization.
And I'm going to create its Elías as essense.
Just execute the cell.
It will take some couple of seconds and operate very close to what you have to do.
You have to video data so you will observe over here.
This is that part where I have all this data for my project.
So let me board another module, which is exactly my OS module that definitely helps us whenever we
have to deal with our operating systems.
I'm just waiting for this OS to from this OS if I'm going to call a function, which is exactly my list
that actory, which is exactly my list.
So if you would shift plus tab, you will get documentation of this function here.
You have to mention your part.
So I'm just going to copy this entire path and I'm just going to do it here.
So whatever path you will based over here, it will exactly return what our stuff's available at that
particular path.
And after that you have to add your art with exactly the to draw string, which tells Python, Yeah,
this is exactly my path and it's Python.
This is my path.
And it is not an exact backslash.
It is exactly my entire path.
Otherwise, in some of the scenarios, if you're not going to append this string in some of the scenarios,
you will get some books.
So if I'm going to execute this well, it will return.
All this stuff's available at this particular spot.
So you will also hear it will return all the data sets available at this path.
Let's say I need this last seven datasets for my analysis purposes fill out.
So what I'm going do I have to access this?
So here I'm just going to say it is exactly my list.
So to access the last seven entry, I'm going to say minus seven to this one just executed.
It will determine my these seven dataset.
Let me store this data.
Let me store this list somewhere in files.
And if I'm going to print my files, that's OK.
Let's say I don't want this data set still.
Let's say I'm going to consider this data set for later analysis.
So what I'm going to do, I'm just going to say why door remove?
I have to remove this dataset.
So I'm going to say I have to remove this one just executed.
And if, again, I'm going to break my files, you will see I have six datasets over here and we have
to collect the entire data from this six dataset and we have to concatenate all the datasets.
And at the end, we will end up having our complete data on what we have to perform some kind of analysis
depending upon what problem statement we have.
So let's see what I'm going to do over here.
Now, let's say very first, I'm going to say I had to just I create on this files and whatever data
form it will return me, I'm just going to contaminate it.
So I'm going to see for file in five.
And now whatever file you have, you have to create a data stream of that.
So I'm going to say dot reach this way.
And here you have a function which is exactly my read underscores the easy and very first here.
If I'm going to pass Jessica's tab here, you will see here you have a file path.
So let me first mention what exactly my path is.
So let me just copy my entire path from here and I'm just going to copy from here.
And let's say I'm going to define a variable over here and here.
I'm just going to come up with it and that board and let me up and over here, which is exactly my roistering.
So once defined that variable, you have to say here, I'm going to very first mention my part, then
I have to upend my backslash.
So here I'm going to say I have to spend my days over here, which is exactly this one, which is exactly
this one guy after it.
What I have to do, I have to read this file.
So I'm going to say concatenate file just with that.
And after it, I have to mention some encoding technique as well.
So I'm going to say, let's see, my encoding technique is exactly my UTF.
It has almost 95 percent of the scenarios.
You will have this encoding technique as UTF eight because this is the most famous encoding technique.
So it will exactly return with some data.
So let me store it in once I'm going to store that.
What I'm going to do, I'm just going to say, oh, BDI dot concat.
So I have to concatenate each of the data frame from this iterations before this.
I'm going to say beedi dot concat and let me define a new data frame over here.
So here I'm going to say final is it goes to nothing but b'day dot data frame and this is exactly the
data frame function.
So final is exactly a blank data frame.
And I'm basically going to append my final with this data.
So I'm going to say final comedia once I will do all the stuff I'm going to say final.
It goes to all this.
It means in this final data frame, I have my all these entire data.
That's what I'm going to do.
There are basically a baby approach to collect this entire huge chunk of data.
Now you have to just execute this.
It will take some couple of seconds depending upon what a specification you have now.
All your stuff gets executed.
And if you guys are not that much confortable will with this logic.
So there is an alternative approach for you.
What you can do, you can create a separate set of frame of this one, this one and all these different
ones.
You created a frame of each and every data set then you can concatenate using this Beedie or concat
function.
So that's basically a little bit Landi approach.
But that's a smarter approach and this is exactly that approach that you have to follow in your real
words.
And adding, that's why I'm going to mention this one approach in front of you.
So let me check what exactly the dimension of this new data frame which I have created over here.
So let me save final shape now.
You will see what it has that much number of rows.
So just amazing how much huge amount of data we have.
So that's basically your exact data, your real world data on which you can perform your analysis.
That's all about the session.
Hopefully you will love this approach.
How?
I have just using some iteration, how I have collected this huge amount of data in just a single data
of him.
So I hope you will love the session and definitely this session.
Really very helpful for you.
If you are going to follow this approach.
I hope you will love this.
Thank you.
Have a nice day.
Keep learning.
Keep growing.
Keep practicing.