Though we have to
talk about the S3 Baseline Performance.
So by default, Amazon S3 automatically scales
to a very, very high number of requests and
has a very very low latency between
100 and 200 milliseconds to get the first bites out of S3
So this is quite fast and in terms of
how many requests per second you can get
you can get 3,500 PUT/COPY/POST/DELETE per second,
per prefix and 5,500 GET/HEAD requests
per second per prefix in a bucket.
So this is something you can get on the website and
I think it's not very clear so I'll explain to you
what per second per prefix means.
But what that means (indistinct) is that it's
really, really high-performance.
And there's no limits to the
number of prefixes in your bucket.
So let's take an example of four objects named file and
let's analyze the prefix for that object.
The first one is in your bucket
in folder one sub folder one slash file.
In this case, the prefix is going to be
anything between the bucket and the file.
So in this case, it is slash folder one slash sub one.
So that means that for this file in this prefix
you can get 3,500 PUTs and 5,500 GETs per second.
Now, if we have another folder one and then sub two,
the prefix is any anything between bucket and file.
So slash folder one slash sub two.
And so we get also 3,500 PUTs and 5,500 GETs
for that one prefix and so on.
So if I have one and two, we have different prefixes.
And so it's easy now to understand what a prefix is.
And so it's easy to understand the rule of
3,500 PUTs and 5,500 GETs per second
per prefix in a bucket.
So that means that if you spread reads
across all the four prefixes above evenly
you can achieve 22,000 requests per second
for HEAD and GETs, so that's perfect.
Next, let's talk about
KMS as a Limitation to S3 performance.
So if you have KMS encryption on your objects
using SSE-KMS, then you may be impacted by the KMS limits.
When you upload a file, it will call S3 on your behalf
The GenerateDataKey KMS API.
And when you download a file from S3
using SSE-KMS (indistinct) itself
will call the Decrypt KMS-API.
And so these two requests will
count towards the KMS quota.
So to set an example, our users connect their S3 bucket and
they want to upload or download a file
using SSE-KMS encryption.
And so S3 bucket will perform an API call
either generated a key or decrypt
to a KMS key and get the result from it.
And so by default KMS has a quota
of number of requests per second and
based on the region you're in, it could be
5,500 per second, 10,000 per second, or
30,000 per second request.
And you can request a quota increase
Using the Service Quotas console if you need to go over that limit.
So what this means is that
if you have more than 10,000 requests per second
in a specific region that only supports 5,500 requests
per second for KMS, then you will be throttled.
So you need to ensure that KMS
doesn't block your performance on S3.
Now these quotas are pretty big for normal usage
but still good to know if you have many, many files and
a high usage of your S3 bucket.
Now let's talk about S3 performance.
How we can optimize it.
The first one is multi-part upload.
So it is recommended to use a multi-part upload
for files that are over 100 megabytes and
it must be used for files that are over five gigabytes.
And what multi-part upload does
is that it parallelize uploads and
that will help us speed up the transfers
to maximize the bandwidth.
So as a diagram that always makes more sense.
So we have a big file and we want to
upload that file into Amazon S3.
We will divide it in parts, so smaller chunks
of that files and each of these parts will be uploaded
in parallel to Amazon S3.
In Amazon S3, once all the parts have been uploaded
it's smart enough to put them together
back into the big file.
Okay, very important.
Now we have S3 Transfer Acceleration
which is for upload and download and
it is to increase the transfer speed by
transferring a file to an AWS edge location
which will forward then the data
to the S3 bucket in the target region.
So edge locations that are more than regions,
there are about 200 over 200
edge locations today and it's growing.
And let me show you in the graph what that means.
And that's of Transfer Acceleration
is compatible with multi-part upload.
So let's have a look.
We have a file in the United States of America,
and we want to upload it to an S3 bucket in Australia.
So what this will do is that we will upload that file
through an edge location in the United States.
We should be very, very quick and
there we'll be using the public internet.
And then from that edge location
to the Amazon S3 bucket in Australia
the edge location will transfer it
over the fast private AWS network.
So this is called Transfer Acceleration
because we minimized the amount
of public internet that we go through.
And we maximized the amount of
private AWS network that we go through.
So Transfer Acceleration is a great way
to speed up transfers.
Okay. Now how about getting files?
How about reading a file in the most efficient way?
We have something called an S3 Byte-Range Fetches.
And so it is to paralyze GETs by
getting specific bite ranges for your files.
So it's also in case you have a failure
to get a specific byte-range then you can retry
a smaller byte-range and you have
better resilience in case of failures.
So it can be used to speed up downloads this time.
So let's, I have a file S3 it's
really, really big and this is the file.
Maybe you want to request a first part
which is the first few bites of the file.
And then the second part, and then the end parts.
So we request all these parts
as specific Byte-Range Fetches.
That's why it's called Byte-Range
because we only request a specific range
of the file and all these requests can be made in parallel.
So the idea is that we can
parallelize the GETs and speed up the downloads.
The second use case is to only retrieve
a partial amount of the file.
For example, if you know that the first 50 bytes
of the file in S3 are a header and
give you some information about the file,
then you can just issue a header request,
a Byte-Range request for the headers using the first, say,
50 bytes and you would get that information very quickly.
All right.
So that's it for S3 Performance.
We've seen how to speed up uploads, downloads.
We've seen the baseline performance
and we've seen the KMS limits.
So make sure you know those into going into the exam.
And I will see you in the next lecture.