Hey everybody welcome to part for the Linux command line mini module.
Now in this video we're gonna be working with data how to manipulate it how to transfer and translate
information.
The first thing I want to go over something we've already kind of talked about and that's cat.
So if we do a quick look in here we can see that we have a file called data.
Now we'll just take a quick look at what's in it.
So Cat prints out the data from a file doesn't necessarily need to be a text file it could be an executable.
Doesn't matter as long as it's considered a file which in this case everything is it will prints to
the screen.
Now the screen can do some funky things if you put the wrong characters or you can out a binary that
can be a problem.
It can.
It can do some weird stuff to the prompt.
So first of what does it look at the data.
So we have a good bunch of data here some duplicates some things with uppercase lowercase.
We've got a comma separated list here.
So a CSB with looks like name on one side and then some sort of digit on the other.
In this particular case what the digits represent doesn't actually matter.
So that's the data we're working with.
So first we want to do is let's say we want to find out and we want to find every line in there that
contains a specific string.
So for that we're gonna be doing grep.
Now there's two variations of this.
We're going to go over the first one which is the standard graph right.
So we're gonna look for the word red.
So we see that there's a few in there called Red Team nation.
So if you get data we can see that the output is correct.
We're just looking for the word red.
In this case.
So if we look for red team we get a few others.
Looks like it's the same.
Same bit self we look for.
Blue.
Perfect.
Now let's say we want to look for red like this we get nothing grep is case sensitive.
So if you do a dash eye that means be insensitive.
So again it can do uppercase lowercase doesn't matter as long as the text is the character itself matches.
So in this case we see we're matching red team like normal now so there's another version of this for
grep dash E which we're not going to go over in this particular video.
I'm going to leave that for you guys for homework.
But grep dash e handles regular expressions.
Now we don't have a video on regular expressions in this particular course.
However if that's something you guys want.
Feel free to hop over to the Facebook group and then let me know if that's the case we get enough support.
I'll go ahead and make a mini course on regular expressions.
We don't need that for this particular scope and to kind of get going it's not quite needed.
So that's crap.
That's how we're looking for text so that's similar to find string in windows that we have already looked
at.
Now let's look at start replacing things.
So we do our data again let's say we want to remove the comma that we had and replace it with a dash.
Right.
So we could use something called TR so typing this in a TR The first thing is we're gonna want to tell
it what we want to replace.
So in this case a comma and we want to replace it with a dash dash.
Right.
So what it will do is it will print out the first dash.
So if we see here we've replaced the commas with dashes.
Now this can be done for the new line.
So if we have a list here and we want to replace all of the new lines with a comma just to get it all
into one one giant list we can do that.
And now we're here now we've got everything in place.
So TR is good for substituting now there is something called said again we're not going to go over that
I'm going to leave said for homework for you guys as well to kind of go over and look at how to manipulate
the data in terms of substitution said also breaches into regex as well.
Now let's say we've got this set of text right and we want to grab only the stuff after the comma.
However if there's more data after let's say this is named comma age comma
title comma whatever.
Right.
Let's say we only want the second column.
So what we're gonna do is use something called cut.
So with cut we do a dash deferred the delimiter.
So how are we going to split the information.
We can do this based on a comma.
So and then F is for the field.
So if we get F one we should get red team nation Red Team Blue Team random random to random three.
We're not going to get anything after that as we can see we don't.
However if we do F two which is after the comma we're going to get just the digits after the comma.
Now this is important when you're passing out CSP ease it can make your life a lot easier.
So go ahead and play around with that.
Now let's say in our data we have a lot we want to find out the word count.
Right.
So if we cut out the data and we pipe that into something called WC we get our word count.
So the first digit here is how many lines we have which is important.
There is an easier way to get that.
I'll show you in a second which is the most common way.
The second is the amount of words and then the amount of bytes.
But in most cases when you're doing WC you're wanting to know how many lines are.
So what you would do is do a dash l dash.
I'll just give you a line count.
That's pretty important when you're trying to figure out how many instances of something you're running
into.
Right.
But there's also a different way to do this too.
But before we get into that I want to talk about sort which is exactly the same as it does on Windows.
So if we sort this data we can see now that we have ABC ABC Blue Team Blue Team random red team in testing
if we want to reverse it.
We do a dash.
Now we've got it reversed again.
Sorting is important for this next step because if we want to get the the amount that's unique.
So for example we see red team nation here's three times what the exclamation point but not with then
there's two more instances of it without the exclamation point.
Same thing for testing this for instances of testing and then there's two instances of texting too.
Now how do we want to do this.
Well the first thing you need to do is sort the list right.
And then you need to pipe that again which is why is it piping is important and is something called
Unique know what that's gonna do is it's going to provide us all the unique entries for this file.
Now it removes all the duplicates which is might be important to you in some cases it might not be.
However there is another one called Dash C now a dash C is going to do is it's going to count how many
instances of something you see.
So if you're running an engagement you want to know how many IP addresses in the report are vulnerable
to this specific thing you can do something similar to this.
So you can see ABC is in there twice.
Blue Team is in there three times.
Red team nations in there three times with the exclamation point.
Without it you see two and then testing is four.
So getting used to piping all this stuff together is very useful.
Now well we want to do now is look at the command's tail and head so in a nutshell tail grabbed the
last five I believe of five
OK the last ten last ten is by default and then head will do the opposite.
Right.
So we're now we're getting the top kind cat data we can see with head we've got the top top 10 and with
tail we've got the Bottom 10.
Now a tail and head there's a dash an option for the amount that you want to go.
So if we want to get to only the top five from data or the the the last five from data we do that.
We did a dash and five if you want to do that only the top five at the very top of the list you could
do this.
So now we've got our top five.
This is important for grabbing like for example if you're trying to scan the system and find the the
the top five globally accessible files.
This would be a way you can sort out that top five.
Now what you can do past this point is something called tail dash f data.
Now the reason it's just blinking there is because if I had another prompt let's say we'll go ahead
and openness open a new terminal.
Right so I want to I want to kind of show you that.
So what tail does it follows the file.
So it actually keeps printing it locally but only showing the instance and only showing something new.
So let's see we'll go to
desktop data.
So if I go
testing to come up five testing to come up fifty five into data don't overwrite it then do it we can
see on the far left side that now we have testing to comfort thought so it's good for viewing logs and
other bits like that.
So as we see it's at the bottom now those two are very important to understand.
Quickly follow.
Finding information.
Now what we're gonna do over.
We're going to go over the last bit of something called more and less.
So if we cut this into less
what it's going to do is it's going to provide us a quick prompt.
Now this file is very small so it only has one page.
However if you've used a larger file let's see if we can find a larger file.
Cat or a log will go over this later and grab something
log and the pipe that into less.
Right.
So as you can see we've got a bunch of information and you could scroll with your mouse wheel which
is what I'm doing now.
Use the arrow key down or hit the spacebar.
That goes by page.
So it's a good way for you to kind of sort through and look through data that might be a little bit
bigger.
Now to get out of it you just can type.
Q Just the just letter Q Same thing for more.
Very very similar but except it wants the spacebar not the down arrow again Q pulls you directly out
of it.
And understanding both of those is is very important.
Now I'm going to give you a quick overview of a file Ed called VII.
Now let's say we want to edit this file so VII data or the will not use that it will use something new
data to write enter.
Now when you get in here to start editing the file you click the letter A.
Once you have that you can start testing stuff on the screen.
I just hit enter two four five whatever random text random text Rent Tax random tax Colorado tax right.
So let's say you're done you've edited your file you've changed whatever you want you're good so you
hit escape
w q now w was right quick q is quit.
So if we just did w just write the file if we did w q this will write the file and quit.
So if we come back in here we wanted to just quit going to so it's important to understand that play
around with the v i.
There's other things such as nano whatever you're more comfortable with.
I'm just used to using the ides normally on every system so you normally don't have a problem with that.
Get used to get used to some sort of baseline text editor such as Nan or V I play around with some files
use some of the local files on the system to kind of find information that you want that you're wanting
to look at and what I want you to do for some homework is to look up regular expressions something called
said S E D and then something called OK a W.
Those are more advanced data manipulation and they can be used in a large variety of ways.
However for this of course for this particular course just understanding what they do and then kind
of how they work and at a pretty high level is more than enough to kind of get you going.