So in this lesson,
we are going to talk about "Configuring Data Retention".
Specifically, we're going to be looking
at some data retention policy basics.
So, I'll explain how data retention works
at a super high level.
And then we'll talk about
configuring data retention in Azure.
And then, of course, we'll jump into the portal,
and we'll take a quick look
at how you would set data retention.
So, data retention policy basics.
The first thing you need to do
is determine your regulatory requirements.
So this is your legal requirements, right?
Every business is going to be different.
Some businesses are 7 years,
some businesses are 3 years,
some businesses don't have legal requirements.
But as a data engineer, you need to understand
what the regulatory requirements are for your data.
So start there.
Once you've done that,
you're going to determine a retention policy.
So, based upon the business needs,
and it's not just regulatory requirements,
there's also business needs here.
So it might be that we want 5 years of records
and legally, we have to keep 3 years of records.
So, for our data retention policy, it's 5 years.
And after that, we purge, right?
That could be a data retention policy,
or it could be one of those things
where you just have a business need
but no legal requirement or regulatory requirement.
So, determine your regulatory requirement,
then determine your retention policy
based upon your business needs
and that regulatory requirement.
Third, that's going to give you your data retention period.
How long does your organization hold onto its data?
That's what that is.
Once that's done, generally, you would purge that data,
or you could move it somewhere else as needed.
And then fourth, you are going to implement
or start your data retention period,
and set up a periodic review.
Generally, this is going to be something
that's an annual review or something like that,
because in general, regulatory requirements
and business requirements
-: -at least as it's concerning data--
doesn't change that often
as far as on how long to hold records.
Next, let's talk about configuring
our data retention policy.
So, this has to be configured by Azure service.
And so, for the DP-203, we're going to focus on Synapse
so you get a look at how that would be set.
You're going to implement at the SQL-pool level.
And I'm going to jump in and show you this in just a second,
but I'm walking you through the process right now.
So, we go into the Audit tab, we configure
our data retention period,
and that's going to be set for SQL pool events.
Then, we're going to define our retention period.
This is anywhere from a day to 9 years,
and that's somewhat dependent upon
the service that we're using.
And so with that, let's just go ahead
and jump into the portal, and let me show you
what this looks like.
So, here we find ourself in Azure Synapse.
And I have scrolled down from Overview down to Security,
and then Azure SQL Auditing.
And this is what we are interested in.
And you can see here, like I said,
Azure SQL Auditing is going to track your SQL pool event,
and then it's going to write them somewhere.
So let's go ahead and turn that on,
because it's not on by default.
And then we get to choose where we send this thing.
So let's just say
that we want to send it to a storage account.
So, I pick a storage account.
And then you can see here, under Advanced Properties,
it gives me a choice,
anywhere from 0 days up to 9 years.
So, I can choose that.
And in addition to that,
I can also send it to Log Analytics or Event Hub as well.
So once that's all set, I just click on Save.
It'll take a couple of seconds.
And then, Azure SQL Auditing will be turned on.
That simple.
So with that, let's go ahead and wrap up this lesson.
So, this looks like an auditing lesson, not data retention.
Yes, it does look like auditing.
However, the key is the purpose, right?
From that data retention, we can run auditing
or we can store it for regulatory requirements
or do a whole host of things.
Next, don't just look at storage.
Log Analytics is very useful for querying.
So if you just need to store it
and you don't ever plan on using it,
hey, storage might be fine.
If you think you're going to be using that in your analysis,
Log Analytics is an excellent solution.
That's it for this lesson.
Super easy, super fast.
I'll see you in the next.