As I mentioned previously,
there are a few prerequisites that you would need to complete before
you could fully follow along with this demonstration.
First, you need to have an Elastic cluster available, to ship your logs to.
And second, you need to have Kibana installed.
No need to worry if you don't have those set up.
This demonstration is really geared to give you more of an idea
of what you can do with the Suricata data.
As you'll see in this demo,
I have an Elasticsearch server running at a .200 IP address,
as well as Kibana running at .215 on my local network.
There are a few ways to install Filebeats,
as well as much of the key Elastic Stack components.
Since we're running on the latest Ubuntu,
I'm going to follow the instructions to use the APT package
manager on the Elastic documentation.
At this point, I've already installed the public signing key,
added the APT transport HTTPS package and updated repository definitions.
These are the key steps needed in order to be able to
install Filebeat using apt install.
At this point, all that is left to do is to update APT and then install Filebeat.
Once installation completes,
you can use systemctl to enable Filebeat as a service,
by running the command systemctl enable filebeat.service.
What this will do then is to start the Filebeat
service every time the server starts.
You can also use systemctl to check the status of the Filebeat service,
and you'll see that after installation, it is not running.
We need to first make some changes to the
configuration before we start the service.
After installation with APT,
you'll find the key configuration file under etc/filebeat/filebeat.yml.
You'll need administrative privileges to edit this file,
so let's go ahead and open this up with a text editor.
Now, there are several things to consider in the configuration file,
but I'm just going to focus on integrating with Kibana, which will
provide some default dashboards to help you immediately begin
visualizing the data and then connecting to Elastic.
If you scroll down in the configuration file, you'll get to the Kibana section.
You'll see that there is a setup.kibana property,
and then the host field, which was previously commented out.
You'll need to uncomment this,
as well as provide the IP address to the host that is running Kibana.
Next, we need to tell Filebeats about our Elasticsearch server.
If you scroll down a little bit further,
you'll find the Elasticsearch Output section.
This is already commented out,
so all we need to do is update the host value to reflect the
IP address of our Elasticsearch server.
You'll notice that this is going to connect on port 9200,
and just like you saw with the Kibana server,
both the Elasticsearch server and the Kibana service are
going to be listening on default ports.
If for some reason you set those up with nonstandard ports,
you would need to make that accommodation here.
At this point, we just want a simple setup,
so we're not going to worry about authentication or connecting over HTTPS.
There are many things to consider when setting up an Elastic Stack and the
Elastic documentation will give you a lot of ideas and suggestions for how
to get a Elastic Stack setup production ready.
Let's go ahead now and save this file and move on to our next steps.
By default, Filebeat contains a list of modules that it uses in order to
understand how to parse certain log files. You can use, now that Filebeat's
installed, the filebeat command, along with the arguments modules and list to
see what different modules are enabled.
And lucky for us, there's already a Suricata module that we can
enable. Configuration files for the modules are located under
etc/filebeat/modules.d. As you can see by looking at all of these
different configuration files, all of them, by default, come disabled.
So, we need to enable the Suricata module in order to tell Filebeat
to begin picking up this log information. To do that, we'll use the
module sub command with filebeat.
But, instead of listing the modules, we'll tell it to enable
Suricata. As you can see, we not only get a success message
that that module was enabled,
but now, if we look at the configuration files, the suricata.yml
will no longer contain the .disabled extension.
The next step is to prepare both the Elasticsearch server
as well as to import those default dashboards into Kibana
that I mentioned earlier.
Part of the setup process is to install default templates,
index templates into Elasticsearch. This provides a field mapping to
data type so that Elastic understands what data it should be
expecting to receive. This has an impact on searchability, as if the
data isn't interpreted correctly,
you may not be able to search it in the way that you intend.
Also, since we set up the Kibana server in our configuration
file, this will import the default dashboards that comes with
the Suricata module into Kibana for us.
You will have to pay attention, once you run this command.
If there are any problems connecting to Elastic or Kibana,
you'll see those omitted here.
Otherwise, give this a couple minutes, and once it's finished, then, you
should have both a template index inside of Elasticsearch, as well as the
visualizations for Kibana. So, I'm going to skip ahead and we'll wrap up
this installation here and then look at those dashboards in Kibana. Now
don't forget that we haven't started the Filebeat service. So now we can go
ahead and use systemctl to start Filebeat, and it never hurts to check the
status after running, just in case there were any errors in the
configuration file.
Now that it looks like it's running though,
we should see data being shipped to Elasticsearch, which will allow us then
to visualize it with Kibana. So, let's move over to Kibana. By default,
Kibana will not provide any authentication. There is some basic security that
you can enable, and again, I would refer you to the documentation in order to
get started with that. However,
pointing a browser to the IP and the port that your Kibana server is
listening on is enough to get started. From there, open up the navigation on
the left hand side and select Dashboards. Filebeat doesn't install sample
dashboards just for the enabled modules. It actually installs them for, it
appears to be, all of the modules. So we can use the search box to narrow
this down, and you'll find that there are two dashboards for Suricata. One
provides an alert overview. Another, an events overview. Since we just
started Filebeat,
it may take a little bit before data starts to flow in and become
visual, but you'll see here, that from some of the previous pcaps
and other traffic that we've analyzed, we already have a nice
looking Alert Overview dashboard. So, with very little work, we can
collect logs from Suricata,
get them into an Elastic Stack and begin visualizing it. From here, there
is some effort that you'll have to put into it, to ensure that you're
getting the data that you want, and that your dashboards and
visualizations provide the insight that you need. But, as you can see,
this data can be incredibly powerful.
So, we focused on the data that Suricata generates around the network.
Now let's turn our attention to Suricata's file extraction capabilities.