-: Hey, what's up, gurus.
Congratulations for finishing another section.
As we move along through the course,
you'll notice that our drive and view
is getting better and better,
because we are almost done
and almost ready to take that DP-203 exam
and grab a new certification.
But first we gotta take just a minute
to look in the rear view mirror
and see what we learned in this section.
So in this section, we are going to be talking about a
review. Keep in mind not everything is reviewed,
but we are going to review some of the core concepts
that you need to know.
It's all around the DP-203,
so focus on that.
There are other topics and deeper dives
that we are not doing
that you might need for a data engineer,
simply because my goal is to get you to pass that DP-203.
Finally, if you don't know something,
hey, go back and review.
And if it seems like you've been here before, fantastic!
That means you are getting close to the finish.
So let's talk about how we start collecting data.
If you'll remember, there's great news,
we already are collecting data.
Through Azure Monitor, all Azure services
are generating activity logs, platform metrics,
resource logs, and VM guest metrics and logs.
We just need to tap into that
and pull the data that we need out of it.
We also talked about
how we measure performance of data movement.
If you remember, we estimate what we should get,
we compare that to what we are getting,
and then we optimize as needed.
And because of a cool concept known as full utilization,
we can estimate our throughput because of that.
If you remember that chart that we had to look at
and figure out what kind of performance
we should be getting from data movement.
And finally, we use our monitor copy data activity
to measure what we are actually getting.
So don't forget about that as well.
We also talked about the synapse query store.
What is the synapse query store?
Well, you should say
it's a tool that provides insight
on query plan choice and performance,
and allows us to capture queries and plans,
and runtime statistics.
But, you gotta turn it on.
If you remember, I threw that script in there
to show you how to do that.
And there were 3 different kinds of stores.
There's our plan store,
our runtime stats store,
and our wait stats store as well.
We also talked about measuring HDInsight clusters
and remember, it all starts with Apache Ambari.
We can use this to create alerts
and manage and monitor HDInsight clusters.
And here's a couple of pictures
if you remember what it looks like,
and we access it by going to
whatever our cluster name is .azurehdinsight.net.
So don't forget about Apache Ambari.
So, summarizing all of this together.
Hey, think scenarios.
Scenarios will make you a better engineer.
So be thinking about how these concepts tie into
the things you're doing at work
or other scenarios that might fit that application.
Don't forget about the core services.
In this section, there were quite
a few concepts and services,
so just keep those concepts in mind
with which service they live.
And as always, don't forget about the labs,
you'll need those for your exam and your career.
If you are enjoying these lessons
make sure you do me a favor
and smash that thumbs up button.
I greatly appreciate it.
Congratulations on finishing another section
and I will see you in our last big section.
All right, I'll see you there.