Now we're going to dive deeper into the logging framework.
We haven't quite learned the basics of scripting yet.
That's going to come in the next module.
But for now,
we'll look at the structure and some of the nuances of the
logging framework's scripting and practice with it.
So, previously I spoke about the three components of the logging framework,
the streams, filters, and writers.
When it comes to manipulating the Zeek logging infrastructure,
modifying the writers doesn't happen very often unless
you're trying to get a specific output.
Using the ASCII writer is common, and that can be converted to JSON formatting.
Zeek also has other writers that it can support,
such as the SQLite writer and ones that can be used with things
like Elasticsearch as previously mentioned.
Diving into the actual interface for the ASCII writer
so we can see some of the options,
we're able to look at and understand what they do.
For more information on the writers that come with
Zeek and how to manipulate them more, you can find them here in the Book of Zeek.
We have all of these options like detecting logs that didn't
get rotated and formatting timestamps in a certain way or to
write logs in the JSON format.
If you take a look at the docs,
Zeek defines all of these options and the attributes that you can use with them.
We'll customize our log scripts with some of these,
but we won't be going into and modifying the writers themselves.
The filters are what define parameters of what the stream will actually do.
So, a stream can have multiple filters attached to it.
One thing to keep in mind is that if we create a log
stream and that stream has no filters, no output will be written to the logs.
We can use our filters to do things like rename logs or
change the log path for the output,
as shown here by changing the name of the default filter for
the connection log to default_connection.
We can have them logged to multiple log files and
define the log rotation processes here.
What's cool is that we can use these to help us add additional
fields to the logs if we wanted to as well or rewrite the different
field or column names in the Zeek TSV logs.
You can make changes to the log streams either globally or
locally within each filter that you apply.
By default, the filters are set to log everything.
So, when we create a log stream,
it's given a default filter which is called default.
We could define the record and even if using the default filter,
we can use the &log attribute for each key value
pair that we want to include in it.
We have a lot of global events that are already defined
within Zeek to use for the logging framework,
like connection stats and file analysis.
We can call the event to trigger something when Zeek
is initialized or stops running.
We'll dive in and check some of these out in the coming demo.
The log streams, as I mentioned previously,
are the components that we can use to create logs or
modify the output to the built‑in ones.
Every stream needs to define a record type that tells us
all of the fields that we'll be logging.
We'll also need to have a log ID that defines the stream itself.
There are a few options and attributes that we can use for creating the
stream and actually writing the events to the logs,
like the log::create_stream and log::write functions.
In order for us to create a new stream,
we can use an existing module or create a new one.
To properly define and use the streams,
we have to define the events that we want logs generated for.
We can use built‑in events or define our own.
Customizing the events that we want to log involves a more in‑depth
discussion about the event handlers and how they work,
which we'll cover in the next module.
We can also disable various streams if we don't want to use them
anymore or if we needed to disable it for other reasons.
In the next clip,
we'll jump into the logging framework and walk through a
script that we can create and manipulate.