All right, now that we have Logstash configured,
we need to finish up and create the data source in Arkime,
then we'll validate it and check out some of our data. Now is the
time to jump into the lab and get configuring.
And so here we are in our Malcolm dashboards, and so I have two tabs pulled up.
This tab that we're on right now is our Kibana dashboard.
It's our welcome dashboard when we first log into the Kibana piece of Malcolm.
So we have this nice dashboard that gives us the log counts that we have,
it gives us the different types, the protocols that we see in the environment,
service by destination country,
and then we can go through and see all of the logs,
for example, down here.
We can see it in either table format or JSON format.
If you're familiar with how Kibana works then you're familiar with the
visualizations and dashboards that we can get with it.
We also have all of these different tabs from the Zeek side of things in here.
We can look at the common protocols, click them,
we can go look at specific logs, so like the weird logs,
for example.
I know, I always default to the weird logs for some reason.
But we can see a lot of the information about this stuff.
And we can also go to the Discover piece and actually
look at the logs that are coming in, so we have the one index pattern here,
and then we see all the different logs that we have coming in from Zeek.
And as you can see,
it's using Filebeat as the tool that's sending the logs for us.
Now I'm going to hop over to the CLI forward again,
and I'm going to go ahead and navigate to our Moloch directory,
or our Arkime directory, and, specifically,
I'm going into the WISE directory.
Now this is where we can go to actually view the views that we
create that show the mappings for Zeek to Arkime.
What I want to look at here is this source.zeeklogs.js file.
And as you can see, it's the WISE data source definitions for the Zeek logs.
Now if you're doing this integration on your own using your own
instance of Arkime and Zeek and the ELK Stack,
then you're going to need to add this WISE data source to be able to
view the Zeek logs in Arkime's dashboard properly.
In here we are doing similar things to what we did in
the Logstash pipeline configurations.
We're defining different fields,
but in this case we're calling different areas so Arkime
can map the fields to different things.
So we have this file right here, the source.zeeklogs.js file,
Arkime's config.ini file has the field definitions there,
and then also Kibana's Zeek template has them.
And so scrolling through here,
there's all of these different field definitions in here.
And so these all are defined in this WISE data source so that Arkime
can actually use them and show them in the dashboard.
So that's essentially what we mean by creating the WISE
view, or the WISE data source for us,
is defining all of the fields that Arkime will pull and then mapping them to
the different tools and stuff like that like we saw above.
And also,
what's really cool about Arkime is that we can add right‑click pivoting to the
dashboard in Arkime to be able to pivot to things like searching IANA for
services and searching VirusTotal and so on and so forth.
So we'll demonstrate this capability when we get up into the GUI to view it.
We're also using this data source to define the different
arrays and connection information as well.
And as you can see here, we have a lot of the Zeek field values here,
and you see it kind of maps them to the other data
sources or the other field definitions.
And that's it.
So if you're familiar with Arkime and how to create these WISE data sources,
then you should be good to go with configuring this yourself.
But essentially, you can pull templates for WISE data sources from Arkime.
You can go in and download Malcolm and use this prebuilt template
and just make modifications if you need to.
And so now let's go up to the web and hop over to
Arkime and view the information.
Before we actually go into these sessions and start diving into them,
if we go over to the Settings and then Views column,
if you look here, we have these views that are saved,
different logs within Arkime.
Now the most common Zeek log that we usually view in Arkime and/or
map the data to is going to be the connection log.
Because if you think about it,
connection log is tracking all of the connections that we see on the network,
Arkime's capturing the packets on the network,
so it makes sense that Zeek connection logs would fill in the blanks for
full packet capture if it's not available in that location.
So now we're going to go over to the sessions,
and if you're familiar with Arkime you can experiment with this on your own,
but I'm just going to filter on protocols == http to
pull up any information that I can.
As you can see, nothing's there.
So let's go ahead and try DNS.
How about protocols == icmp? Let's go check out some ICMP packets.
And my spelling's failing me on this one.
There we go.
All right, so we have some ICMP packets.
And if you look at this,
what's really cool about this is we can see the data from the Zeek logs,
and we can see the data coming in from Arkime.
So right here we have a pcap from Arkime,
but also right here we have the Zeek connection log.
They're showing similar information.
We can search by the community IDs here as well,
if we wanted to, and tie them to Arkime's root ID.
Let's see if that pulls up anything for us, and as you can see it did.
And so here what we just did was mapped Arkime's
session to Zeek's connection log.
And we can download the entire pcap if we wanted to,
we can download the source or destination versions of the pcap,
we can go look at all of the sessions, so there's a lot of power here,
and we were able to view the data in multiple ways.
A. And in Arkime what's cool is it has a lot of the information about a lot of
the fields and information that we would normally see in Zeek. We see the
source and destination and the number of packets,
the ports, and the IP addresses.
Obviously, ICMP's not really a good example for ports and stuff like
that. And we also have some of that enriched information in here so we
have the OUI information from our enrichment pipelines showing that
it's VMware. Now what we can do is take this information and pivot
over to Kibana using it.
So let's take this IP address, for example. If we click on this menu, and
remember how we saw some of those right‑click pivots for VirusTotal and
Kibana and reverse DNS lookups and stuff? Well, we can look at this host
in Kibana just by clicking the Kibana source IP, and it's going to send
us straight there and pull up a log search for that source type. And as
you can see, we have six hits. So we can look at the information in
Arkime,
correlate between the Zeek logs and the Arkime data, and then pivot
over to Kibana to view the full logs if we wanted to.
And so that's the power of integrating Zeek with Arkime, and
that gives us a lot more new capabilities.
So we could create automatic correlation with the community IDs and root
IDs between Zeek and Arkime. We could use some of that enrichment
information like the OUIs to help us alert on specific communication
mechanisms that we may or may not want to see.
All right,
this was a great module. We talked about a lot, and we
configured a lot more throughout the module. We were able
to ingest Zeek logs into ELK,
parse them and enrich them with Logstash in Malcolm, and
map them to Arkime's schema for correlation inside of the
tool using Malcolm as well.
I hope you learned a ton in this module and so far in this
course. We have a few more modules to go,
which means we have more integrations to learn about for Zeek. In the next
module, we're going to integrate Zeek with another tool called RockNSM, or, more
specifically, we're going to show Zeek's integration in RockNSM and also learn
about the file carving capabilities within Zeek. Take the time to practice and
explore what you've learned so far in this course, and when you're ready, join
me in the next module.