Everyone, and welcome back to the knowledgeable video series.
Now, today, we are going to talk about a quite important topic specifically for the users who downloads
a lot, and that is multipart uploads and downloads.
So I'll tell you a story a long time back around seven, eight years back in India, we had a speed
of 256 kbps, the Internet speed, and that was considered as a lot.
And for the people who like to take a lot of risk like me, we try to download one GB file directly
from browser.
So one of the problems that happens is if your Internet disconnects, then you have to download the
entire file again.
And that used to be a big challenge.
I still remember one time I was downloading a one device with 256 kbps Connington and at 99 percent,
the rain gods decided to show up and the Internet disconnected.
And at 99 percent, the entire file went away.
So I had to download the entire file again.
And it used to take like a day to download.
So that used to be a quite big bill.
So that is for download, even for uploading.
So whenever you're uploading a large file and if your Internet disconnects, then the entire uploads
begin from the start.
And this was a big bill.
So in order to save our computers lives, there has to be a good technology which allows better solution.
And this is where the multipart upload and download comes into picture.
So Multipack, as the name suggests, basically, is that there are multiple parts of a single file.
So multi part upload is a way in which we applaud the entire file in form of small individual chunks
to the storage device.
So let's assume that we have this original object here.
So what we do is we divide this object and do multiple small, small parts and then we upload these
small, small parts to the S3.
So even if your Internet disconnects, what we have to do is we only have to again re upload that small
chunk of data to the storage device.
We don't really have to begin from this thought.
So this original object is divided into two parts and you have three, six, nine and 12.
So let's assume you uploaded the nine bytes and while you were uploading the tenth part, your indented
disconnected.
So after your Internet starts back again, you don't really have to begin uploading from start.
You can resume your upload from the text file and complete the registry.
Now, after you upload all of these to S3, the S3 will combine all of them together to form the original
object again.
So this is called as the multi part based technology.
Now, one important thing to remember is when we are uploading data via multipart, we need to specify
the number and its position in the uploaded object.
Now, this is very important because this will help Idalis reconstruct the data.
So we have to tell it of this S3 that this is let's assume this is part one.
This is part two.
This is part three.
This is four, five, six, seven, eight, nine, 10, 11, 12.
So when.
The S3 receives all the parts it needs to reconstruct them and it will reconstruct based on the number
that we assign.
This is the part number that we assign to each and every object that we upload in history.
So this is the basic about multipart upload.
Let's do one thing.
Let's create a practical and see on how exactly it would really work.
So I am in my two instance, so I have an easy two instance which is running and I'm connected to the
easy to an instant.
Now if I do a double S3, Alice.
There are four or five or four as three buckets which are present right now.
So what will be doing is we will upload a file to one of these three buckets via multipart upload.
So let's do one thing, since we need a good amount of files, say so.
Recommended way of multipart upload is if you have a file size, which is greater than 100 and B, then
you should be doing a multiple uploads.
So let's create a file.
So I do it and say input file.
Is there view random output file would be the big file.
Dart picks the block sizes one meet and count.
Is, let's assume 150.
Hoopes.
So what this will do is basically this will be create a file of around 150 M.B, so as we already discussed,
it is recommended to have a file size of 100 greater than 100.
And if you are doing a multipart upload now, you have a big file.
So if you're do you have an asset on big file, it is offsite 150 and now there are two ways.
As we already discussed, there are two ways.
One is you directly upload the file without multiple upload.
And during that case, if it breaks, then you have to upload everything from beginning.
And the second way is you divide the file and you upload the files in multiple chunks.
So I have a nice little script called Multi Hyphen Pardot Pibor, and this script basically uploads
this file in multiple parts.
So let's try this out.
So I'll run multiple hyphen parts would be why I specified the big finder to exist.
This is the file that we want to upload and we'll specify the bucket where we have to upload it with
the escape website and fill.
So when we press and what will happen is this python file, you see, it will divide the file into multiple
chunks and at the time it will applaud the individual John.
So this is junk number zero.
So once this chunk gets.
Applauded, then you have the junk number one.
Let's wait.
Perfect.
So now what do you have seen, the entire fight got divided into two chunks and forced the upload was
of first chunk, then upload was off second, and now you can definitely divide and do much more smaller
parts.
So since there are two chunks, you can assume this one to be 75 and beat, this one to be 70 Fahmi.
But you can actually reduce it more further depending upon your requirements.
Now, if you just want to quickly, if I do a WASC Alice Capi LAVs hyphen.
Feel what?
And you see, I have a big fire, a lot of size 157 m.b now this is the same file size.
This is the same fight.
So although it got uploaded in chunks in S3, it got combined together that just got combined together
and you get a single big fight.
So this is it about the multi part upload.
I will be uploading this script in Artforum sections so you can go ahead and download and you can try
it out in your labs.
So this is it about this lecture.
I hope this has been informative for you.
And if you have any doubts, questions, ideations, feel free to connect us at Twitter, Facebook and
LinkedIn or preffered.
We may as instructors are very capable of certain types of watching.