In this video, you'll learn some basics of monitoring on a
Lennox server.
Plans to do a brief overview of Monitoring Solutions that come with Linux you
will use a few commands to probe your Linux host
to check CPU load and memory.
Finally you head over to AWS console and
check out cloudwatch metrics for your ec2 instance. Okay,
let's get started.
There are many ways you can monitor Olympics hosts.
In fact, there's a whole industry built around
this but for this course, I'm Gonna Keep it very simple.
So let's we're going to think about monitoring. We're gonna think about it in three
stages. These three stages are gathering
reporting and alerting. So Gathering is
recording the metrics or state of your system. Yeah, I
think logs tracing time series data reporting is
the reviewing of the data and displaying
it, you know think about graphs on a wall.
And the final stage of learning alerting when
they threshold is reached either
automatically fix the problem or page someone to take
a look at it. So we're going to cover the first two categories of
modernify inspecting and Gathering data of the
state of our lab host.
Then I will show you how to create a very simple metrics dashboard
on the AWS console using cloudwatch metrics
that are already provided to you by idios by default.
We're going to explore a few ways to
inspect your host using commands. You can find on any Linux server.
There are numerous commands to inspect your system.
So to keep this digestible we're going to
only focus on two. The two commands are top
in VM stat. You can find these commands in
the proc psng package.
On our AWS house though. They should already be installed by
fall.
So the First Command we're going to use like I said is top.
And so the top command will display all the running
process information about all the running processes on your system.
And you could do overviews like CPU memory
and load. So let's go ahead and run that so in your
terminal type in the top command.
And as you can see this brings up a lot of
information and every five seconds by default, this
page will refresh with new information as it pulls it.
So let's just start from the top down.
The top line here is basically the
uptime. So how long this system has been running and here
it's been up for 18 minutes. This is because I we
did the reboot in the previous video this other column
here tells how many oh, I'm a users are logged in the system. And
here's your load average. So load averages split up
into three different sections. We have one minute five
minute 15.
This tracks your load on your system over those three times
fans.
Load average basically takes into account a
bunch of different data points of CPU memory
disk I/O Network IO and it
tries to give you a good indication of how busy
your system is now. It is not a
end all be all because you could have a very high
load on a system.
And the system be performing fine and responses are
fine. So but what this is is a good indicator that
maybe you have stuff waiting for CPU time.
If you have a high load, so just use it
as a starting point.
The next line down here is all about the tasks. So basically all the
tasks that are running whether they're what state they're in that
they stop they sleeping or running.
This other line down here talks about the CPU basically
shows what percentage of the work is
being done in the user space. They're kernel or
if it's the cpu's idol
or it's waiting on I/O task to finish and then
these last two Fields down here all about
memory and swap. So memory you
got basically the total the free to use and then what do
you have in buffers in cash this last line here is Swap
and if you're unfamiliar with swap it is a way for your Lennox host
to write stuff out that
it usually would put in the physical memory on to disk
if there's a lot of back pressure or your system is busy obviously
swap writing writing
memory just to disc is a lot slower than
actually writing it to physical RAM. So just keep that
in mind. Okay, so everything down after this does top
levels is all about processes.
And here we have a PID which is
the process ID. The user that that process
is running at don't worry about the pr and I
columns we're not going to discuss them. So you
have vert which is the virtual memory you have
res which is the resident memory or basically how much
memory that process is using of the physical of your
physical memory.
Memory that is shared not only with your process but could
be shared with other processes and the virtual is
basically all those put together.
Thing you also have here this s column
here is basically the state of your process whether
it's sleeping interrupted running that
sort of thing. You have percent CPU and percent
memory which basically shows how much percentage of
the process or how much percentage of the memory that you're
process is taking up or the this process taking up
and then the last column here is the command and so if
you watch it long enough, you see there's the top command we ran.
The PID number one which is system D and is
this this will come back and forth as each of these processes
have their time with
the kernel and CPU to do work. Obviously. This
host is not doing anything. So as you can tell
there's no load. There's no CPU as you can come up here.
Pretty much. Our CPU is almost a hundred percent. I
is a hundred percent of the time is Idle because we're not really
doing anything. So top is good for a quick overview of
a system at a glance. If a process
is using a lot of CPU or memory topkin show
that if your host has high load talking show that too
but I usually use top as a starting point if
I'm jumping on a system and I I think there's something wrong and I'm
starting to troubleshoot then I usually start
with top
Okay to quit top you have to do control C
and that'll get you out of that polling page and
let's clear the screen. Okay. So the
next command is VM stat the vmstat command
reports info about processes memory block
IO and disk and CPU activity.
So vmstat command takes many flags,
but we're only going to focus on two arguments
that we're gonna pass to it the first argument tells VM
stat.
How often to pull the system for the for this
data and then the second argument we're going to pass to it is
how many times should it do that? So in our case we are
going to just do one in five which means pull the
system every second but only do this for a
count of five times then quit.
So go ahead and type VM stat space one space five
hit return and we'll let it run and as
you can see we did the five times.
And so the output of vmset is broken up
into section. So the first section shows about processes like
kind of whether they're running a blocked. The second is
memory third to swap the fourth category.
There is I/O fifth is system and six to
CPU. So we'll go through a few
of these in just kind of talk briefly about what each of these do so
If you have tasks waiting to run and are
blocked then the First Column would show that so R would
be they're running and B would be blocked. So if you had a high
number if you had a high number in the blocked, that
means you do have processes kind of waiting for something
so, that'd be a good indication of something. So if
you look at the memory category,
You have swapped free buffer and cash columns swapped.
Basically this this column right
here basically just talks about what's what's being swapped in and out of
disk.
How much is being swapped I should say from the
memory and then here you have the free
column and this shows you all the free unallocated
memory on your system.
And then you have here the buffer.
buff shows the actual amount of memory in
usage
and you have here as the cash column.
The cash column is used memory that could be either like
a cut the cash column shows memory
that's already used but it could be either swapped out
the disc or giving back to the system if if
they need it if there's any kind of memory pressure.
This swap category where I talked about this is kind of just swapping in and
out and bytes which being the disc.
IOS the same thing shows up
like activity for this like you have bites in bytes
out. So
if you're experiencing slow disk rights might be
good idea to look in here. If you have a lot of high numbers
here that might be an indication of what is going on
and give you a breadcrumb to some other tool or
process to look at.
System system is just kind of like how many tasks and
like how many things are being asked to do
you think of like contexts and so,
you know, this was really high then may indicate
that your system is over busy. And then the
last category is CPU. And so this shows you
information like user space your kernel
space What percentage they're running in it? How
idle is your CPU and then
you have weight right here wait, meaning that the
CPU is waiting to do work, but it's waiting on I/O task
to finish before we can if you ever have a high CPU
or a high load, but you can't figure out what a culprit is
go ahead and you should look at the weight column here because if you have
a slow disc or slow network connection that may be slowing backing
everything up and this would be a good indication of it.
Okay. So what we'll do here is
we'll stop here. We've we've talked about the top of
the VMS document the next being that we're in the
cloud. We get a benefit of a whole bunch of metrics provided
to us out of the box. I wanted to start off with
showing you some Basics that are on a Linux
host. But now let's go. Look what AWS offers for our ec2
is and so let's head over to the browser.
Okay. So here we are in the browser and the first place we're
gonna go look at is over our ec2 instance. There's
a monitoring tab if you recall from the
ec2 console here, so let's go here click on it.
And as you can see what we'll go on the instances running.
And if you recall this is our lab one.
Go ahead and click the checkbox here. And this should
bring up our detailed view. I'll scroll this
up. So it's easy to see and go over here to the monitoring tab.
So click on monitoring and as you can see, let me
scroll up just a little bit easier to see we have a whole bunch of
small little graphs everything from CPU utilization
to the status checks and basically the status check
someone Amazon brings up an ec2 instance. It makes sure that
it's up and running and that these fail safes pass
before it'll allow you to say that hey there's a
problem with this host or not.
So you have Network in bytes, you have the packets in
and out all the disc reads disc operations
did how many times we're writing to the
disc? And then you have this down here, which is called CPU credit
and CPU credit balance. So basically
what CPU credits are when you use a free tier and Amazon Linux CPU
credit allows you to do as
many tasks as you possibly can up until
certain point and when that happens you have
a balance in the more busy your free small
host becomes you start depleting your
balance of CPU credits. And if you hit zero after
a certain amount of time Amazon will actually throttle your
your free host to make it even slower.
So just be aware. That's why these are really good for
testing and for prototyping but you wouldn't want to run this free
tier this this T2 small.
The micro here as a production host. Okay, so
let me scroll back up here.
Another option you have here is you can look at like hey.
What were the metrics for these hosts a
day ago and you can click over here to the one day into
three weeks we can do a custom time frame as you
can see right here. We had a spike right here on the CPU utilization. I
most certainly assume that is
when we rebooted the host, you know coming back up
we get a little Spike and
The network or you know, that could also been when we
did the updated self and the young package
manager was using more CPU.
So you can play around with that all you want and then you
come down here and you can also do you can do an auto
load where these this page refreshes every second. So this is great. If
you're already in this page, but Amazon provides a
whole product called cloudwatch metrics and
in my opinion, it's a much better experience in
this. This is good for a quick look but so what we can do
is we can click on the ad to dashboard here and what this
will do. This will bring us over to the cloud watch metrics and create a
dashboard for this specific instance for us. So I'm
gonna go ahead and click add to dashboard.
Okay, so what it does here it says add it's popping
at us up asking a static dashboard since we've
never created. We don't have any dashboards. There's nothing to be found.
So we have to create a new one. So we create click create
new.
And I am just gonna call this Linux lab, but
you could call whatever you want. You can't use
any spaces. So just be be aware and then click create.
One now that's created it highlights us to a select dashboard force
and then you just click add to dashboard.
If you go ahead and click save at the top here, this will
go ahead and save the dashboard. And as you can see
now we're under This Cloud watch metrics run
to Cloud watch. We're on the dashboards and that way the Linux
lab. So it's basically all those that we just
saw there except that we have a little bit
more a one central place to look at them. So we could you
could imagine putting a dashboard for all your ec2 instances
or you know for your database running
in AWS, whatever the case may be.
Obviously here you can change the the resolution
as well. You know, we want to look back last 12 hours.
You can do other things here too as well. You can you know,
share this dashboard. You can delete it. You can rename it those sort
of things. You can also click this little star little favorite which
will put it as one of your quick dashboards to look at if you
go over here to this little hamburger menu and if you click on
that
What this will do is this will open up all the other
stuff that cloud watch offers. And if
you look there's alarms so you could go create an alarm based
on CPU utilization. You could see yourself saying hey
if the CPU gets above 80% page me or
send me an email.
This is obviously logs. So you could go create a
law group and pull in all the logs from all your
ec2 instances and you could write an alarm
an alert on a log base if you see something in
a lot a log line you could send an alert.
You also have metrics and so you you have
a metrics all metrics and metrics explore. And what
you could do is you could go look at all the metrics for all your accounts
and that you have access to this is
also an easy way to do it here as a quick way. If you go back over
here, let's go to CP utilization and you click on the three dots you
can see this link. Dennis's view this
symmetrics and what this will do is this will bring this exact metric
up in the metric Explorer and I'll
click on this now so you can see it.
And so as you can see, we're in metrics basically all
the metrics. You have the graph up top. You have
the graphed and see how it already highlighted our ec2 instance
for us and it tells us let me move
this over it tells us what the metric is. It's CPU
utilization. The statistic is
doing an average and it's looking over a five minute period
And so you can adjust these as you go
to.
but I wanted you to have a basic idea of
What also you could provide that AWS provides
for ec2 instances, there is a lot more you can do here
in like anomaly detection and insights and
alerting. It's really is a One-Stop shop for you. So while
it is great to know how to do things on the command line.
On Linux to be able to troubleshoot a host.
It is great that because you're in a cloud environment
that you have access to all these other tools basically out
of the box for you that you can turn on and
start using once you spin up an instance.
Okay, so that'll cover our AWS dashboard.
Let's head back over and let's do a recap of everything we
did in this video.
Okay, so let's talk about what we did in this video. You learned
about basic monitoring from the command line on
a Linux host you use the top in VM
stack commands to validate the health of your Linux instance.
Finally you visited the AWS console and looked at
cloudwatch metrics and created a simple dashboard for monitoring
your ec2 instance.
In the next video, you'll explore networking inside your
Lennox host.