Hey, Cloud Gurus. Welcome back.
In this lesson, we'll be talking about partitioning data.
During the course of the lesson, we'll go over
why partitioning matters,
as well as a couple of different strategies,
vertical partitioning, and functional partitioning.
This will be mostly an introduction of partitioning
concepts, helping you get familiar with the strategies.
And then in the next couple of lessons,
we'll get into some best practices for partitioning,
but we'll wrap this lesson up with a review.
To start out, why does partitioning matter?
Well, here are 5 reasons.
One, it improves scalability.
As our data grows, we can split it out
into more usable chunks and still access it quickly.
This is not to be confused with the same type of splitting
out across multiple computers that you will encounter
with sharding, but we'll get to that in a future lesson.
It improves performance. Think of it like a filing cabinet.
If you can pull 1 file and read the contents of it
instead of the entire filing cabinet,
it's going to go a lot faster.
We improve security.
You have an opportunity to place data
with different security needs on different partitions.
You improve availability.
Having the data spread out avoids single points of failure,
and you can improve cost savings by taking advantage
of opportunities to place lower priority data
on cheaper storage.
We have a couple of options when it comes
to partitioning strategies.
The first is vertical, where you distribute the data
according to its pattern of use, and functional,
where you aggregate data by purpose
within a bounded context.
Let's jump first into vertical.
Here, the partitions hold a subset of the fields,
and they're divided according to the pattern of use.
If we take a look at this visually,
let's say we have a table of students with a few different
fields: ID, Name, Topic, and Hours Watched.
We can take the most frequently used fields, such as ID,
Name, and Topic, and put them on 1 partition,
and then take the less frequently used Hours Watched column
and put that on a different partition,
still matched up by the ID.
Think of this kind of like a chef
who has to have the most common orders ready faster.
This chef is going to keep on hand the ingredients
for their most frequently requested meals.
Maybe if their specialty is hamburgers,
they've always got hamburgers ready to go,
and then maybe just 1 or 2 hot dogs,
because, I mean, really. Who wants a hot dog
when you can have a hamburger?
But they're prioritizing their most frequently
requested meals over the lesser used ones.
This strategy also allows you to separate static
and dynamic data.
Slow moving data can be cached in memory by the application,
improving its performance.
You can also add security,
placing sensitive data on separate partitions
with higher levels of security.
Our second strategy is functional.
And in this case, we're aggregating by use,
finding bounded contexts to determine our partitions.
Think of it much like offices.
The sales team is going to be in 1 office
and IT is in another because they're working on
completely different things.
If we take a look at this,
let's say we have 2 sets of data.
One is a list of students and the topics they're watching,
much like we had before. The other is customer information,
or the businesses that employ the students
that are watching the courses.
And so we'd have our student information on 1 partition
and our customer information on a different partition,
2 different bounded contexts
placed on 2 different partitions.
With this strategy, we can also separate by workload types,
placing our read-only data in a separate read partition.
By way of review, partitioning improves scalability,
performance, security, availability, and cost savings.
There are 2 strategies for partitioning,
either vertical or functional.
Both of these are methods of taking a single database
instance and splitting out the data
into multiple workable chunks.
The difference is that one does it by the pattern of use,
and the other does it by bounded contexts.
And of course, partition decisions will depend
on the type of data you have, how you want to use it,
and what the workload will be.
This is really what's going to decide
which of these partition strategies
is the most effective for you.
Thank you for joining me in this video on partitioning.
As I said before, in the next couple of videos,
we'll dive into some specific strategies
around partitioning both on data lakes
and Azure Synapse analytics.
When you're ready, I'll see you there gurus.