>> Hello everyone and welcome back to the course,
identifying web attacks through logs.
After a brief review of web server logs
and log importance,
in this video we'll keep talking about logs,
but I'll also give you some advice and I'll demonstrate
some common issues and mistakes that
can occur during log analysis.
The video objectives are to understand the differences
between availability and security log analysis.
Understand that some log fields
can be crafted to hide something from
the analysis and show
some mistakes that can occur when analyzing logs.
Let's start with the rise of security risks.
Recently, the presence of security staff have
increased and SOC teams have
started to play an important role in companies.
Nowadays it's common to have
a NOC and SOC team working together.
They have the same worries with different perspectives.
NOC and SOC teams want to keep things functioning,
but NOC usually worries if the systems are up
and SOC worries about security incidents.
A security incident occurring
doesn't mean that a resource is down.
Though a security incident,
can affect the resource
even if it is working as expected.
For example, consider this web server log.
What do you think it is?
Is it malicious, weird or is it just okay?
Since we have a 200 status code,
a NOC analyst could say that,
yes, it's okay.
The server is up and it's answering.
They could also check the CPU and
memory and say everything's okay.
Though for a SOC analyst,
this is suspicious behavior,
so it's better to investigate.
During this course, you will learn
that this request is an attack,
specifically an SQL injection attack.
In the previous slide,
we had our log that was related to an attack.
The logs are generated by the web server,
but how are the logs generated?
Can I trust other log information?
Logs are generated from two actions,
the client request and the web server answer.
The web server is known,
it's under our control.
The client is someone who we probably only
know the IP address and user ID of.
Usually we don't know if the IP
is from an attacker or a real client.
The conclusion we reach is that
we can't trust the client too much.
Because of this, we have our doubts.
Are the logs 100 percent trustable?
The answer is no. But let's see why.
HTTP protocol consists of basic text commands.
It is easy to craft text packets.
Remember, we have a lot of
user agent software and some of
this software can craft package with HTTP requests.
What do you think will happen with the crafted packet?
Remember, it's an HTTP request.
As soon as a crafted packet arrives at the web server,
it will be processed and answered.
The web server job answers the request.
It doesn't care who sent the request.
For example, you can say that it's
a different user agent or a different referrer.
This happens commonly during attacks
because the attacker wants to hide.
It's better to use
a normal user agent than a suspicious user agent.
Web browsers are considered normal user agents.
During this video, you'll see some examples of
suspicious user agents like curl.
During the course, you will see another example of
suspicious user agents like Python Libraries.
Interestingly, for TCP/IP communication,
the web client IP address is always true.
Since the IP HTTP transform
starts off the three-way handshake,
the IP address on the log will be
the same IP that established the connection.
One possible problem is when the user
connects to a VPN or a web proxy,
these two will hide the web clients real IP.
In this case, the VPN or
the web proxy address with the client's IP.
Again, the web server doesn't care if it's a proxy,
a VPN or an end-user and will log
the web proxy or the VPN IP address.
To get the real IP, you need
the logs from VPN or the web proxy,
and then you need to correlate them.
Another thing that can't be crafted is the status code.
Can you guess why the status code
can't be crafted by the user?
For an example quickly,
the status code should be 404,
but it'll show 200 on the log.
The status code is generated by the web server.
Though it depends on the client's request.
You can craft requests to get a 404 status code.
But to change the status code from 404 to 200,
you need to change the log
inside the web server log file.
That cannot be done during the HTTP request.
Based on what we've seen,
allow me to show you some quick examples.
Using a Linux machine,
we will perform some requests
using different user agents.
The IP address of our web server is 10.2.0.101,
and our false access will be a simple Telnet.
Check the log of the Telnet requests.
It didn't show on user agent,
but we can see the same status code, 400.
For the second, third request, we use curl.
Curl is the Linux command to Request Webpages.
In the first curl request,
it's easy to see that curl is a user agent.
However, curl has many options.
One of the options is that you
>> can change the user agent.
>> If we use the curl option to change the user agent,
the web server will log
exactly what we put in the option.
Here we used Mozilla Firefox.
There are many other options
in curl that can be used to craft
HTTP packets and lots of
other software with the same capabilities.
For the summary of this video, check out this table.
It has the key log fields,
and if it's possible for
that respective field to be crafted in a request.
On the set crafted, it's
possible to generate and manipulate
the HTTP request to hide some information
>> about the request, like the user agent.
>> Based on this, the IP address cannot be changed.
As we said before, this is
because of the three-way handshake.
Date and time depends on the web server configurations.
User ID can be crafted and the attacker can
use this to perform a brute force attack
or to steal someone's session.
Method and requested file can be crafted,
but if the requested file doesn't exist,
the web server will always answer with a 404.
We will see in the next video that
404 errors can help us identify some attacks.
HTTP status code is generated by the web server so
it cannot be crafted and it is
possible to craft the user agent.
Other client related fields can be crafted.
Crafting packets and HTTP requests
is one way to tell how a web server is compromised.
Some crafted requests can
actually trigger vulnerabilities.
Maybe you're thinking now
that I know I can't trust web server logs,
why should I use them to identify an attack?
Even if you don't trust them,
you need the logs to identify the attacks.
You always need to take care when doing analysis.
A really important thing is to know your application.
For example, if your webpage isn't
compatible with mobile phones,
well, you should not see user agents related to mobile.
I would think, as an end user and guess if
the end user will do the same thing
is you have in the log.
Try to guess if that info could be
fake and always get more logs to correlate.
This will continue in the next video.