Hey, what's up, Gurus?
Brian back with you
to talk about Performing Data Exploratory Analysis.
So in this lesson, we are going to talk a little bit
about what data exploratory analysis is.
I'm going to give you just a brief code example,
and then we're going to wrap up this lesson.
It's going to be a pretty short lesson
because the focus for this is not going to be on the code.
It's just going to be on understanding the overall concept.
So with that, let's dive in and get started.
So at a high level, data exploratory analysis,
it's just talking about initial investigations on data.
So we talk about taking a look at data,
without diving too far in.
So what we want to be able to do is test a hypothesis
or write some code and see if we're in the right ballpark.
And so, exploratory data analysis allows us to do that.
And it helps us
through something called Kusto query language,
to give a summary of statistics
and graphical representations of our data.
It's actually not that far off from Databricks,
which we'll cover later in the course,
but as you build cells, you can actually, in Databricks,
take a look at an individual snippet of code
and pull up graphs or charts
to kind of see what's going on with the data
that way, as well. Similar concept.
So the takeaway from this is it's making sense
of the data you have, before going too deep with it.
Now, in order to look at exploratory data analysis,
we're actually going to talk about Data Explorer,
which is an Azure tool that you can use
to take a look at code,
and we're going to use just a few basic queries
in order to do that.
So, here's a code example.
Let's say that we have traffic events,
and we're taking a look at traffic events
across the United States.
So we could start off by sorting by the start time
of that traffic event,
and just pulling 10 results to kind of see what we have.
So once we have that,
we can then go on and get a little bit more information.
So we could say, let's take a look at the traffic events
and based on what we saw, let's sort by the start time.
But instead of getting a hundred columns,
I'm only interested in the start time, the end time,
the state that it occurred in, the type of event,
so maybe that was a single car crash,
and then the damaged property detail, right?
So I could specify those columns
by using a project statement in Kusto.
And then again, I could say
I'm only interested in taking a look at 10 results.
So what I'm doing
is I'm performing that exploratory data analysis
by quickly running through
and looking at little bits of data,
and kind of seeing what's going on with that data.
So once I get that information, I could then go on and say,
all right, let's summarize all of this.
I could summarize the event,
take a look at an average, sort it by state,
and you can here.
And then I could render that in a column chart.
So I could move
and actually start to take a look at the data
by looking at a column.
So now, within the span of--what's that--
maybe 10 lines of code,
I've been able to pull some data out of my database.
I've been able to look at what's going on
and build some pretty basic charts.
That gives me a decent idea of where we're at.
For instance, why is Florida so much lower than California?
So you can search a look at anomalies and start to get ideas
on the data itself to figure out what's happening.
So at its core, that's what exploratory data analysis is.
So, kind of wrapping all this back up together,
exploratory data analysis,
it lets you make sense of the data you have
before you build giant complex processes around it.
And using Azure Data Explorer,
it allows you to perform that real-time analysis
on large volumes of data very quickly.
And we use Kusto queries.
And that's how we, just in the example of the code,
that's how we interact with and explore the data.
Really, that's all there is to this.
We don't need to dive any farther into it.
So I'm not going to give you tons of examples on code.
This is what you need to understand for the DP-203.
I hope this has been helpful.
And with that, I'll see you in the next lesson.