Now that the audit logs are set up to be produced in the Kafka cluster,
we can do a lot of interesting things with them.
We can, for example,
set up an alerting mechanism if someone creates a lot of
resources in a short amount of time, by effectively spamming our server.
To set this sort of mechanism up, we can use ksqlDB.
First, we need to give our data some structure using the CREATE STREAM statement.
Basically,
we are telling ksqlDB that the log events produced in the
audit‑logs topic have a DELIMITED structure.
All the fields, date and time; identifier,
represented by username and IP; resourceType; action; and
resourceName are delimited by a comma.
Once we've defined the data structure,
we can start defining the SPAM resource creation detection mechanism.
This query might look a bit intimidating,
but it will be pretty easy to understand once we go through it.
So, how is this mechanism going to work?
Well, for each time interval of 60 seconds,
we calculate the number of entries where a resource was created.
If the user creates 3 or more resources in that time interval of 60 seconds,
then an alert mentioning the username and the account will
be sent in the audit‑logs‑spam topic.
By pressing Enter,
we're launching a process in the background that
will start the analysis process.
So, how can we check if it's working?
Well, we can inspect if any alerts are produced in the audit‑logs‑spam topic.
Just like in the previous demo, we can easily do this using the PRINT statement.
No alert has been published so far,
but that's because no one has created more than 3
resources in a 60‑second time interval, so let's do just that.
First, I need to open a new terminal, because in order to create three resources,
we need to execute three HTTP requests to the audit logger using curl.
I'm going to copy and paste the first example request and execute it 3 times.
This should definitely trigger our alerting mechanism.
If we switch back to the ksqlDB CLI,
we see that something has already been published.
We have the key, which is our identifier represented by the key,
and a little bit of the IP.
The other two large numbers represents the time window timestamps,
meaning when it started and when it ended.
Finally, we also have the value that represents the number of resources created.