Let's begin by making changes to Suricata's configuration.
First,
we'll enable file‑store by changing the value from no
to yes under the file‑store section.
We're also going to remove the comment in front of the stream‑depth property,
ensuring it has a value of 0.
As the notes in the configuration mention,
setting this value to 0 will ensure that Suricata performs
an unlimited read during file extraction.
Next,
we'll set a value of 0 for the response‑body‑limit
setting under the libhtp section.
We're now ready to save our configuration.
With any changes to the configuration,
we need to restart Suricata and can do that with systemd.
Keep in mind that you need to give Suricata a few minutes after
restarting before it's ready to process traffic; however,
for this demonstration,
we don't need to restart Suricata as we'll be utilizing the ingest script,
which relies on Suricata's ability to process pcaps and not the replace script,
which is utilizing Suricata as a service.
Before we're ready to extract files, we need to create custom rules.
Open the file custom.rules in the training directory.
Here, we'll add our custom rule to extract PE files from the HTTP protocol.
I've chosen PE files because of their prevalent use for
distributing malware and that a lot of organizations rely on
Windows as their main end‑user PC, as well as server infrastructure.
We'll keep this rule simple.
First, it will alert when a file matches.
We'll use the HTTP protocol and tell it to match on any
source port to any destination port.
We'll then add the filestore keyword and use the filemagic
keyword to match the lib magic value.
For this, I use the string PE32 executable.
We'll discuss where I came up with this value in just a moment.
All that is now left is to add the revision and the SID and save the file.
With our demo pcap, we'll use the ingest script.
This is due to the fact that in the ingest script,
it will tell Suricata to load not only the rules configured by suricata‑update,
but also a custom rule file we defined with a ‑s argument at the command line.
You can see this inside of the script if you'd like to take a look.
After processing the pcap,
you should see alerts that indicate that a PE file was not only downloaded,
but also extracted.
Let's go ahead down and investigate this data using EveBox.
We can use oneshot mode with EveBox.
All you need to do is to tell EveBox to run in oneshot mode and point it
to the location of an eve.json file you want to analyze. Since our ingest
script is placing log files at a temporary location, /tmp/suricata, we
can provide that path. EveBox should now open up in your browser. Ensure
that the time span is all and then navigate to Events, FileInfo. This
provides file info events for every file that was encountered in that
pcap.
Well there are several files here, let's focus on the file os2.exe.
The details of this file include a magic that should have matched in our file
extraction rule, at least in part. The State is Closed,
which means it was not only fully read, but also fully extracted.
And we have several hashes, MD5, SHA‑1, SHA‑256.
Let's take note of the first two bytes, or characters, of the SHA‑256.
These values are used to determine where to store this file during extraction.
Next, let's go ahead and click on the Flow ID. You'll see all of the events
surrounding this file to include our file extraction alerts. Remember, the flow
ID helps us to take a step backwards and oftentimes get a better, broader
context around events that occur on our network.
Let's now head to our log location at
tmp/suricata/filestore. Listing the contents of this
directory reveals several folders with two character names.
These are the first two values from the SHA‑256.
The file that we're investigating started with the characters 57.
Let's navigate into that folder.
Doing so reveals a file.
If you use the file utility, you'll see that it is a PE32 file.
So, the file utility is another way to get the
results of a file type using libmagic.
As long as the file was completely extracted,
you can now use it with other analysis tools, such as a sandbox.
When investigating files to extract,
you can now use the file utility on those file types in
order to see what the libmagic is and devise the patterns
that you want to use in your rules.
As you can see,
the results for a PE file are quite a bit longer than the
string or value that I used; however, running file across a large dataset,
a large number of PE files, I found that PE32 executable
was one of the most common patterns.
If we wanted to get more precise,
say we only wanted to extract .NET assemblies, .NET files as
you see here, then we would need to include parts of this
libmagic result that were more unique to these files, such as
the string Mono/.Net assembly.
In any case,
this gives you a really good idea as to how to configure Suricata to
perform file extraction, the rules that you need to write in order to
extract the files, where those files are stored, and how to determine
what value to use for the filemagic result. That wraps up this section
on deployment considerations. We're now ready to move on to the next
section in this course