System is running slow.
OK, this is a lecture where I am going to cover a little command, so sit tight and make sure you are
ready to go over all the commands, because many, many, many times you have this issue and people
say system is running slow and you are you are being reported this problem by someone else, either
in your group or somewhere else, or actually you find out your own system is running slow and what
are the things that you have to do?
Well, first of all, you need to understand the problem, as I discussed that earlier as well, understanding
the problem.
Where exactly is it coming from?
This is the more important thing before you actually start troubleshooting any issue.
So you need to find out whether it's process related, due process is the one that's running slow or
what exactly the problem is with the desk is that the disk is causing the issue.
Maybe you have external disk where your system is writing or you have your internal disk that probably
one of the disk is has issues or bad blocks or degraded.
You need to find that out.
After that, you have to look into the networking to like file transfer.
Like if someone is transferring files from your system to another system or anywhere within the network,
you have to find out whether that transfer is causing by processing disk issue or networking itself
a memory that is actually in use.
So memory will be the part of the processing as well.
Then at the end, we have to know whether it is hardware related.
So these are the few things you have to keep in mind, whether it's processing or memory disk issues,
networking issue or hardware issue.
So we have different tools for every one of them.
Well, when when we are troubleshooting the system slowness issues.
So we'll talk about the troubleshooting steps now, the troubleshooting step.
The first thing is always check if the right system is reported or you are on the right system.
Yes, when someone reports there's a problem with system ABC and that when in fact it's not ABC, it's
it's X, Y, Z.
So you need to find out what system they are reporting.
The problem on whether it's a database or it's a system server, a web server.
Once you find that out, when you log in to that system, of course, always the hostname to check.
You are on the right system before you start troubleshooting and start digging into it, because I don't
want you to spend like hours and at the end finding out you are not on the right system.
Then we'll check the disk space, the disk space.
The first thing you'll check is if the disk is full, because most of the time and the disk is full,
it will cause problem for everything that is processing, memory, networking or or disk related issues.
Everything will be actually end up at the desk.
If your desk is ninety nine percent full, your memory processor cannot do anything at that point because
they cannot write to the disk because it's full.
So the commands that will learn is about DB and NDU, which we are already covered by quickly quickly
cover in this lecture as well.
Then about the check, the processing and memory of course, and the commands that will cover his top
free list memory.
We'll check the proc memory info that is currently in use.
We'll check the virtual memory status P map, which will tell me which process individual process is
using, how much memory, not just the entire system.
DMI Decode, which will tell me some information about the hardware, what type of hardware I have installed
in my or add it to my system.
Less CPU house.
What the is listed.
How much is this use.
How much it is used by other processing.
Also the CPU info you could use the same thing that will give you the same output as less CPU that will
cover about the check disk issues.
And the main command that I always use is IO stat and I have it every five second interval intervals.
So while you are troubleshooting issue, you can run the state and check the disk status, meaning what
is going out, what is coming into your system and what's the status on that traffic going in and out.
Then we'll also cover the LSH stands for list open files.
Any files that you are trying, any files that are that are up and open at this point that is taking
up the memory or CPU?
What are those files that are open?
We'll find those out that a file comen is also used sometimes when you're trying to unmowed a file system
and you cannot because it's telling you is busy, but you don't know how it is busy.
Maybe there's a script in it.
Maybe there's a file in it and the mounted file system that is used by another process or another user.
So you will use less to find that out and you could kill that process, then comes to the check networking
part.
So the networking commands that we will be using is the main one is the TCP dump with minus ie optional
use on the interface for interface also at the same time will use a less o f to see what is listening,
what the port is open and we'll also grep for the listen what the files are open, what protocol, what
sockets are open and what is it listening to.
We'll use Usenet's Stad minus P, L and T and by the way Netz that has been replaced in the new version
of Linux system and S.O.S and RedHat is a place with S.
S as in socket and same options can be used that was used in nets that then there's another command
F top, which is a very useful when you are troubleshooting network related issue.
If top does not come a preinstalled with your operating system, you will have to actually install a
couple of packages like youm install Apple release and then you will install youm install if top, then
this command will be available to you.
It's like an add on command, but it is very useful that will tell you exactly what is going coming
in, what is going out, what is being exported, transferred, what is the transfer rate.
If it's high to low, then we'll also check the system uptime because the system is running slowmo.
Sometimes what happens, system crashed and it just you don't know if it's crashed or not.
So you should check whether the system just an update, a boot it up or not.
So update will give you the time when it's been up and running.
And it also gave you the system load every five, 10 and 15 minutes.
That is pretty much the same with the load that you see on the top command.
We will also check the logs log as one of the main resource that you have that where you could actually
go and and check if there is something wrong with your operating system, kernel, memory, CPU, hardware,
any type of errors that are reported will be reported.
And your logs.
Then there are other things that you could check on the hardware side that are available to your operating
system.
And if you cannot get to the hardware issues, you need to actually go into the system console to actually
find that information out.
Then there are other tools that you could install against these tools are not added.
When you install the operating system, you would have to install these using a regime command.
And some of these tools are top, high or top PF traffic account.
It's related to the user account status.
So these are the few commands.
If you want to use it, keep them handy, keep them, keep them in your mind.
So these are the few commands that you need to know for being a good system administrator, how you
will use them, how you will run them in the system.
And when you get the output, how are you going to read them?
So let's log into my Linux machine and I will show you each one of these commands one by one, and hopefully
will cover pretty much all of them.
So, as I said, first check, you are on the right system or their system that is reported to you is
the right system.
So for that, you always run the hostname, which will tell you, oh, yes, this is the right hostname,
because many times in a bigger organization, you have servers, 10, 20 servers using the same hostname,
but they have zero one zero two or incremental numbers at the end.
So just make sure you have you are logged in to the right system, then check the disk space.
That's that's the first thing you should always do.
And to run that you do D.F. minus H sorry.
DF minus H h for human readable.
So this is going to tell you the file system on the left side, the size.
What is use.
What is available in percentage and what is it mounted on, all the temp efforts can be ignored because
these are the memory files system.
You could run down minus H and you could grep minus V to exclude temp first to exactly see the status.
So if you look at my system status, I have a pretty healthy system in terms of, um, the route is
59 percent as the one is at one and the boot is 17.
So the system at the disk level, the usage of the disk looks OK.
If you see anything here that says 99 percent or 100 percent or any threshold that you think has exceeded
the normal threshold, then you have to check where why it is using that much.
And for that, you will have to use the Kamandi you when you do desu minus a to all and specify which
file system that you're trying to look for.
You'll see all the file system that's come, that all the files in the system that will show up here
and it will tell you every file and its size.
You could do, um, the minus H for human readable as well.
And it will tell you in by megabyte or gigabyte as well.
If you want to sort this out, you could do short in the reverse order.
So you will see that the highest usage of file and the top, you will do this and it will sort everything
for you.
But of course it's going to go so fast that you will have to actually Morad.
So pipe it to more.
So it is actually going to the proc, which is just a memory once and no such file while you're running
it.
Don't worry about it.
You could ignore any of that that comes up and says cannot access.
But if you are getting an error message there saying, hey, permission denied, then it means definitely
means that you are not root.
So make sure you are rude before you run the D u minus a command or H command on any file system.
So I could go into every file without getting their permission denied error.
Let's clear the screen then.
The next step is about check processing.
So let me make this window a little smaller so we could go over the commands one by one.
The first one we have top, which we have already covered previously as well.
I'm just going to go quickly.
Again, the first line tells you about the system up time.
How many users are logged in and the load average in every five, 10 and 15 minutes.
This one tells you total tasks that are running.
How many sleeping?
This line tells you about the CPU status.
This line tells you about the memory status.
This line tells you about the swap status then coming down.
This is the list of every process that you are running in your system, mostly the high utilize process
you will see at the top.
The first column is the process ID who is using it?
The virtualization raises a reservation, shared CPU memory time command and has a lot of information
as well.
Then we'll go into the next command, which is free.
Free is specifically for memory utilization.
You could also do free minus M for if you wanted to see in megabytes.
So this will tell you how much memory you have total, how much use, how much free share, how much
is the buffer and how much is available to be used by the other, uh, other processes.
Yes.
Other scripts or other programs.
OK, now next one, we have less memory, less memory.
This is tell you exactly how bad the memory block size total online memory that we have available to
the system.
It is one gig, total offline memory that is sitting that we are now using, which is zero.
We don't have any.
So this is going to tell you about your memory as well.
Same thing pretty much when you do proc memory input cat.
Sorry, proc.
And so this is going to tell you exactly what is going on in the system right now, total Membrey,
free memory memory available, it has a bunch of information about your memory and each and how memory
is allocated to every process, every pages in your system.
So you have to read through it, every one of them.
If you do not if you really wanted to know exactly what each of them means, if I start explaining them,
it's going to take me forever.
So I will recommend you.
You could just go to Google and type memory info and it will give you a lot of information about Memento
then PVM Stat, which is a virtual memory status.
This is telling you about your swap space swap is pretty much the same as a virtual memory.
How much it has been assigned or how much is it free, how much is it buffer?
This would also tell you if you have memory.
So if you are running free minus M and you see the total is 991 and used is 991, nothing is used,
then the next thing you have to find out is whether you have the same issue with your swap or virtual
machine.
So virtual memory as well.
By the way, free can also give you both as well, the regular memory and swap memory as well.
So check the memory status, make sure the memory is not used.
If it is used, then you actually have to assign more memory to this machine because it is using up
a lot of processes.
If you are on top again and if you see a lot of processes here and that are using this column right
here, this column is used up a lot of memory, but a lot of different processes.
Then you actually have to add more memory.
Or you could also use a new command that I'm going to show you, which is a map to right here.
If you notice, the top command has a process.
It is six six one zero.
So if I do open up another session.
And I type map and put it in the process, and then it's telling me this is a total process that is
being used by the top command that are the top process that I'm running.
So just that specific information about the map, you can also find out about the other process if you
want to know, like, let's say process.
I'd won this one.
I could give you the process information, sorry, the memory information of one particular process.
Instead of going around and finding out every process, you could check it over here.
OK, so.
Next one is DMI, Decode, DMI, Decode, this is going to give you a lot of information about your
hardware, what is inside of your computer.
So when you run this and if you notice you have an issue with the CPU or memory and you have to actually
call the vendor and the vendor asks you some information about give me the information about the CPU
serial number, a memory serial number or what's the size or what's the release or what's a version
information.
You will find all the information right here.
As you can see, the manufacturer information, the product name, the Chassy information says manufacturer
is Oracle Corporation.
So you know who to call when you are trying to open up a case and with who.
So if you do not know which server that you are on, this is that the MD code will tell you whether
it's HP Oracle machine, Sparke machine.
So that sparks they have different ones, um, like any other hardware like Dell or any of the hardware
that run 64.
But operating systems.
OK, then Al C.P.U, which is kind of similar to as men, it give you the information about your CPU
architecture.
CPU architecture is X eighty six 64, but it actually supports 32 and 64 bit models.
The bite size CPU is one any socket, any core per socket.
Um the vendor CPU family model model name Intel's CPU megahertz.
Everything that you need to know about CPU can also be fine in LCP.
You OK?
Next one is proc CPU, which is again similar to.
Less, less CPU, prox, CPU info, if you notice, pretty much the same as the other one, but probably
it is rearranged, but it gives you similar information to what you would get with LCP.
You OK?
Let's clear the screen so you have find out so many things about CPU, you know, your memory information,
your disconfirmation.
Now, moving on to the disk status.
You know the disk.
When you run D.F. minus H, you say you see that the disk usage is good and it's actually fifty nine
percent.
So there isn't a disk in terms of disk capacity issue, but maybe the disk input output disk itself
is running and had degraded mode.
So for that to find out you have to run.
I was that any input output statistics minus Y with five.
I always use that because I, I kind of used to it.
I'll say that this way there are many of the options that you could use with Stat as well.
I would say there are many, many options with every command that I'm showing you.
So the best way is that you run a man on every command and find out every option that is available to
you.
Anyway, let's run.
I was stat minus Y with every five seconds.
Now this is actually going to the desk and it's going to ask a desk, give me a statistics every five
minutes, sorry, every five seconds.
And here it's telling you your disks.
You have to disk stay SDP and it's read kilobyte read.
Kilobyte written how much it is reading how much it is writing it.
So by looking at my system, my system is running a very, very efficient because it's pretty much zero
point zero.
Nothing is, uh, I don't run any production type of application on my system.
So that's why you see that everything is zero.
But if you are running, you will see how it is going through.
How much is KBE reading, how much it is writing, how much it is transferring per second?
So all that information will show up here.
If you see the number very high, it means either you are of course writing so much or reading so much,
or it's just your desk is simply having some issues, reading or writing or actually queuing up the
the the the entire transactions.
So this is about Iooss that.
Now let's get into networking part.
Now you notice you confirmed nothing is wrong with the disk.
Everything looks good at the processor.
Everything looks good on the memory side.
Now let's look at the networking.
If you are actually if your programs are actually involved into transferring or actually receiving,
you would have to check the networking.
So first command that you will run as TCP dump is TCP IP.
Every protocol that goes in and out is mostly on TCP.
So with minus I option, it's interface.
First you need to find out what interface you're going to run TCP dump on.
So for that you have to run F config and you notice my interfaces NPC or S3.
So I'm going to run on my TCP dump minus I ENPI S03 and when I run that enpi of wrong interface spelled
wrong.
OK, now you see it is going so fast because I am at my PC where I'm running my machine or VM actually
showing you this whole lecture.
It is communicating to this virtual machine.
That is why it's going back and forth, back and forth, back and forth.
So if I just closed this or control see out of it and out and open up another window, by the way,
you have to be rude to run most of these commands, so always log in route.
Now, if I try to do Pangu, let's say Google, Google dot com, it is pinging.
But now as it's pinging, I could actually check this traffic on my TCP dump.
So let's just grep for let's say.
Google.
So as it going down?
No, actually, it is actually not going to Google, it's actually going to the IP address.
So Tsipi dump actually does not is not going to give you Google is actually going to give you the IP
address.
So let me grab for one sixty two to forty three.
Ten one fifty one.
And there you go.
See it's telling me my machine which sname is my first VM is going to the Google with the Perkel ICMP
any time you paying a machine.
That Perl call is known as ICMP.
And so this machine, my machine is using ICMP protocol and pinging this machine, which is Google.
The Perl call is listed at request.
And again, this machine, which is Google coming back to my machine and acknowledging that traffic
is getting a reply.
So I am look at this very carefully.
This is my machine going to Google with this protocol and requesting to send the sequence the length
of 64 bit, 64 bytes right here, sequence time ICMP.
That's what is telling you.
So then once when it goes there, the Google is coming back to me and say, hey, I got it.
You are the one who's who's pinging me and I'm replying back to you and telling you, yes, I'm alive,
I'm available.
So it's just a simple way of doing Tsipi dump.
If you are trying to troubleshoot an issue where your machine is talking to a database server, then
I would say put that database server IP address or use the protocol and the TCP dump.
There are different options to use a protocol specifically for that.
Um, sorry.
Um, for that port.
Sorry, not the protocol I meant to say the port, if you are going to let's say one night, two one
six eight one 13 and that is your database server and that is using a protocol let's say two zero four
nine.
Um, no, sorry.
The database protocol is a sorry.
The port number that is used on database is anyway, I forgot, but I hope you understand what I'm trying
to say.
So you could actually specify that port number to listen whether this is listening on that port.
No or not.
So TCP dump is a very long kind of discussion that I could go into, but I want you to try it on and
this will give you a lot of information when you're troubleshooting traffic issues.
The next one is also F list open files, one minus I, minus P and minus.
And this will give you what is open, what is listening.
So let's say if my machine, this is the one is actually running a, um, a service, let's say, uh,
Secich and I cannot connect from this machine to this machine using SSL.
Then I will come into this machine, I'll run that command and I will grab for twenty two.
And as you could see right here, this will list all the established connection and also listening connection.
Right Hearsays S.H. is listening on for twenty two so I could tell hey my machine is listening.
I have no problem.
Is something with your machine that's not listening.
Or maybe you have a listening.
Maybe my machine is listening and probably I have a firewall that is blocking any traffic coming from
this machine to my machine.
So I have to check firewall as well.
OK, then the next one I have is Netz that.
That P, l and T this is pretty much gave me the same information that I just typed in a different way.
So this is another command.
Another thing that I was going to talk to you about is Netz that is now deprecated.
And the new command that is used is SS, which stands for Socket.
You could use the same options to list all your ports that are open or listening.
You can also do man on SS and you will find another utility to investigate.
Sockets that are open socket is something that runs on your computer, which actually attaches the port
number and opens up every time a traffic comes in on that port or anything that comes in and trying
to knock on the door that has twenty two label on it.
The last one is about networking and I have this command if top and as I said earlier, if top is delete,
that does not come with your Linux operating system.
You have to install that.
And to install that you have to run the command from install Apple release and then install if top.
I have already installed this comment in my system, so I'll run it if top and you will see right here,
this is telling me about the trash forever that is going from to this machine and from this machine
to any other place.
So if I'm using FTP to transfer some files from one machine to another or I'm using protocols like Nafez
or SICP protocol every time you use them, every time you run those commands, and if you wanted to
know how how much bandwidth is used and how much megabyte or kilobyte per second is being transferred,
this comment is amazing.
It will give you all that information from to the rate and the kilobyte, how much it has transferred,
how much is remaining, and a whole bunch of information.
If you want to learn more about it.
I think it when you install it, it actually also install the man pages, I believe.
Yes, it did.
And it says display bandwidth usage on an interface by host.
So use that command.
If you do not have it, I would highly recommend you to install it with the instruction I have given
you.
Next one is check system up time.
See if your system has been rebooted recently.
The time is there with the system load average.
Moving on, the check, the logs, all the logs are in the war logs and when you do ls minus l l tr
whichever the option you prefer and the best log which have all the information or log that I always
look for is messages I could always do tail minus hundred.
Give me show me last hundred lines of messages and I could easily grep for error.
There is no error so I could just take out an entire tail, just cadart and grappler errors.
And here I will see if there are any errors have been reported that might help me find the issue.
I would also say, look for all this.
Put mine aside, by the way.
I will also say put warning.
I would also say put error, I will say, but fail as in failure.
You'll find so many information that could help you troubleshoot the issue, check the hardware status
by logging into the system console.
You could do that by logging.
And if it's HP, go into Illo, open up a browser, put in the IP and you will see the logs in there.
If it's Dell put in the eye drug and if it's other system, every system does have console access.
Find that out and log in and find that hardware information or hardware message the last one or other
tools.
There are other tools that are available to you.
You have to download them as well, just like I downloaded if top.
These are the tools are also helpful.
Some of them like H2, Io top, IO top is helpful, which works the same way for disks as I use that.
Um, so these are some of the tools I want you to actually go through them one by one.
I read about them.
They will definitely help you if you are pursuing sys administration type of job or Linux related job
or any type of computer related job, you will be using these type of commands quite often.