So if you've watched any of my videos, any of my courses,
you know I always talk about logging
and how important it is for any aspect of security.
If you're not recording your data,
how can you know what's happening in your environment?
How can you catch security issues?
So that's why we're going to talk about logging
here when it comes to designing a secure architecture as
well, because it is a huge part of it.
So we are going to take a look at a scenario to start off with
to give you an example of how logging can have
good practical implications.
So there's a man who creates a company.
This man is named Scott.
And he creates a company known as the Scott Michaels Paper
Company.
This company manufactures and sells paper products.
And you, as a security engineer, work there.
One day, a competitor appears that seems
to have very, very similar, suspiciously similar products
as the Scott Michaels Paper Company does.
This company happens to be based out of another country.
And the products they are producing,
they are selling for significantly lower prices.
They are definitely undercutting this original company.
So the organization suspects that somebody may
have broken into their network.
Someone from this other company that's just
popped up out of nowhere just-- and they
suspect this because of how similar those products are.
But unfortunately, the organization
does not have a proper logging setup.
They can't see any account activity that happened
more than 24 hours ago.
So any account log ins or modifications
that happened more than 24 hours ago are gone to them.
They don't have any logs from any of their network traffic
that's older than six hours.
So all they have is the last six hours worth of information
for their logs.
And that information, as far as what files were accessed,
who accessed them, when they accessed them,
from where they accessed them, those logs
aren't recorded anywhere as well.
So the company has no proof that any
of their intellectual property may have been stolen.
They suspect it.
They have very strong suspicions because of the similarity
of the products, but they have no actual proof
that any of their intellectual property was stolen.
So they keep trying to cut prices and cut prices.
But eventually, they can't make any more money.
They're losing customers to this cheaper competitor.
And the Scott Michaels Paper Company goes out of business.
So when we're talking about logging--
we'll come back to this scenario in a minute.
When we're talking about logging,
one of the most important things is what should you be logging?
The easiest way to answer that is
whatever data is critical to your organization.
There's hard to answer the question of specifically what
should you log without knowing more
information about the organization, what
the organization does, what kind of infrastructure they have.
However, there are some core things
you should always log, regardless of the network,
regardless of what's happening.
You should always be logging and monitoring
any activity related to your accounts, your groups,
your authentication and authorization systems.
All of that should be logged, and this includes any account
creations, any account deletions, and any account
modifications.
Any time a password is changed, that should be logged.
Any time the spelling of a username has changed,
that should be logged.
Any new accounts that are created
and any accounts that are deleted, or disabled,
or anything like that should always be logged.
And this also includes any time an account
is used to log on or off to any other system.
So you log when it happens and the source of that activity,
where did this log income from.
Do they log in from workstation A or server C?
Where did they log in from?
So in addition to that, you should
log as much network traffic, as is relevant and practical
for your organization.
At a minimum, though, you should always
be logging your firewall traffic,
your external perimeter firewall.
All of your inbound and outbound traffic
should be logged and recorded for analysis,
for forensics later down the road just in case
it needs to happen.
That's why you keep logs.
Just in case you need the information,
you can have proof about what happened.
And then, any internal network as needed.
This one's really going to depend on the organization,
on the complexity of the network, how
large the network is.
Because the larger your network is, the more you have to log,
the more resources it takes to perform that logging,
and the more storage you need to store those logs.
Also, any file access, any time any files
are accessed on any file servers,
or centralized storage, or anything
like that, especially when that access is protecting
any sort of sensitive or proprietary company
information.
By sensitive, I mean, anything confidential-- user
information, customer information,
financial information, anything that
is protected by any regulations or compliance requirements.
All of that should be logged.
Any company secrets, all of the access to those files
should always be logged.
Your email systems-- you should have comprehensive logging
for your email systems.
This can include being tied to any of the email filtering
appliances or systems you may have
in place for security purposes.
Were these messages identified as spam?
Were these malicious URLs in this message?
You know, how many messages are we receiving from this one IP
address?
And are they mostly classified as clean
or are they mostly classified as malicious information?
That is extremely helpful.
In addition to keeping a log of all
of your inbound and outbound mail--
and I don't necessarily mean the contents of the message.
At a minimum, you should log these source and destination,
IPs, and email addresses, header information,
subject information, date and time, those kind of things.
You don't necessarily want to or need
to log the message bodies because then you
can start getting into some privacy issues there,
but it's something you could log.
But it's not necessarily required.
It should be as part of your email filtering systems,
though.
So it really just depends.
You should also be logging any of your server event
logs, any servers you have set up, especially
that have any specific purpose to them.
So any specific application servers or anything
like that, make sure you're logging and monitoring
the logs for those applications.
So any specific logs that you're looking at,
that are for a specific purpose on that server.
But at a minimum, if you're talking about, say, a Windows
Server, you should be logging things like these system logs,
the system event logs, the Application Event logs,
the security event logs, and also the PowerShell event logs.
I have another course on security sensors and logging,
where I go a lot more in-depth on all of this.
So I definitely recommend you check that out
if you haven't already.
Any applications that you may have running on your network,
depending on how important they are to the business operation
and depending on what type of data they are handling.
Any application logs that are relevant to security
should be logged.
This includes so-and-so accessed this application at this time
from this machine.
That type of information should be
logged because it can be extremely
useful for any sort of incident response
or forensic investigation that you may need
to complete after the fact.
And then any public-facing interfaces
that you may have in your environment, anything
that is facing the internet that can be accessed externally.
This includes web servers.
Any web servers you have that are facing the internet,
that you're hosting in, hopefully, your DMZ,
you should be logging the activity
on those public interfaces.
What's being accessed, when it's being accessed,
from where it's being accessed, all of those standard things.
Any remote access gateways you have,
always should be logging-- who's accessing them from where,
for what duration, all kinds of information like that.
If you have any, say, web-based email portals--
this is similar to the web servers.
But usually, if you have any email portals that
are publicly accessible like that,
those are usually for employee use,
but still, you want to monitor and log the traffic on those
very carefully as well.
Now, ideally, your email portals would actually be behind some
remote access and not available to the internet like that .
There's other security implications
that come in there.
If you have any customer interfaces,
and by that, I mean, this can be anything like APIs
or other interfaces that may tie directly into any customer
systems, depending on what your organization is doing,
those should be logged and monitored as well.
Now, to make all of this logging much easier, and practical,
and much easier to monitor and manage,
this is where centralized logging comes in.
This is where you have a single or multiple destinations,
where all the logs in your environment
actually are sent to.
This typically also give you a single dashboard
for being able to view all of those logs
and perform whatever analysis you need to perform on those
logs.
So the way it works is all of your systems
that are generating logs, you have-- basically,
all IT systems will generate logs in some fashion.
Now, if you want to keep those logs more long-term
or have any kind of automated analysis on them,
then you need to have them sent to a centralized destination
or multiple destinations, depending
on what your centralized logging infrastructure looks like.
And again, I've got that other course
where I go a lot more in-depth on this topic.
This is going to give you a lot of benefits.
You have one place where you can look
for all of your critical logs that you need to.
Instead of having to log in to a thousand
different machines across servers, and workstations,
and network devices, and things like that,
have one dashboard you log into and you can
look at all of your logs there.
It also gives you the ability to start
introducing some automation into your logging, some analysis,
some automated analysis into those logs,
where, if you're a centralized logging platform
might look at your logs, analyze them, detect something
malicious, and automatically take
action on that potential malicious activity
in addition to sending you automated alerts
that something may be happening.
This is much better than having to manually review
the logs to try to detect anything suspicious.
This can also tie in logs from multiple different systems
to help out and give you better intelligence and better
analysis on these logs.
So instead of looking at your web server logs independently,
and your firewall logs independently,
and your authorization logs independently,
your centralized logging platform
will take all of that data and mesh it together
and perform one analysis based on the data
across multiple systems.
Another benefit that not everybody
thinks about with centralized logging
is it also makes it a lot harder for attackers
to cover up their tracks.
Attackers. a lot of times, will use a technique--
a tactic or technique of deleting event
logs on systems after they have finished what they
or whatever they are doing on there.
That's fine.
They can still delete logs on those local systems.
But if you're using centralized logging,
those logs have already been sent off to another platform.
The attackers can delete them from that local system
all they want.
The logs are still intact in a different location.
Like I said, they frequently delete
the logs to cover their tracks, and offloading those logs
helps protect that.
So when we're talking about analysis,
you've got a couple of different ways you can do analysis.
You can automate the analysis, and you can also
do manual analysis on your logs.
Automated analysis, again, is going
to provide you a lot of that's better alerting
and better intelligence based on certain events that
are happening and, even sometimes,
on certain behavior that may be inferred from those logs.
That's going to depend on the specific type
of centralized logging you're really using.
But again, for automated analysis, that is going to,
basically, require you to have some sort
of centralized logging.
But you also want to do manual analysis on your logs.
This gives you that human insight and that second set
of eyes onto your logs and lets you really deep dive and maybe
get a little different type of information
that your automated system may not specifically be looking at.
So it's really a good practice to do both automated analysis
with your centralized system and manual review of logs,
occasionally, as well.
So when we're talking about that automated analysis,
you also want to pay attention to how you set up your alerts.
And also, you want to make sure you're constantly
tuning those alerts and adjusting
them to try and prevent what's known as alert fatigue.
That's when you receive just so many alerts.
And for the most part, they're false positive.
They don't mean anything, but you've been receiving so many
of them that you become insensitive to them.
And it takes you longer, and longer,
and longer before you actually look at the alerts
because you're just receiving so many of them.
You get tired of looking at them,
and they no longer become important.
That's alert fatigue.
So proper tuning and adjustment of the alerts and what logs
are monitored can really help avoid that
and can help prevent you from missing
a potential malicious activity on your network.
So how long do you want to keep your logs?
Well, as long as you need to.
There's not really a good answer for that.
Now, some of that is going to be based on what
your organization needs.
Some of that's going to be based on any regulatory requirements
you have.
Maybe you have compliance requirements
you have to stick with that may dictate
how long you keep your logs.
Your organization may say, we want
to keep these user logs for a year.
Or we only care about keeping these web
server logs for 30 days.
How long you keep this depends on a lot of different criteria.
But one thing to keep in mind is the longer you keep the logs,
the more storage it does take.
Now, you can split up that storage
in the more expensive storage for active logs,
cheaper storage for archive logs, and things like that.
But what it boils down to is the more logs
you keep, the more storage you need.
And you can also vary the retention periods
across different systems.
So you can keep your Active Directory logs for a year,
your web server logs for six months, your internal network
traffic for 30 days, things like that.
You don't have to keep all of your logs
for all the same amount of time.
So we've talked about our log analysis.
We've talked about retention.
We've talked about what needs to be logged, centralized logging,
and how it's really beneficial.
Let's go back to the Scott Michaels Paper Company.
But this time, they have proper logging set up.
They have a centralized logging system in place.
They are logging and monitoring and have automated analysis
in place for things like account activity, for file access,
especially on those file servers that
store their proprietary data, their intellectual property,
those trade secrets for the company.
They're also monitoring and logging
their network traffic, inbound and outbound, at the firewall.
They're logging a lot of their email traffic--
incoming, outgoing email traffic,
including traffic on their email filtering systems.
They're monitoring their public web server.
They're logging that data and sending it
to their centralized logging platform.
In addition, they're monitoring their endpoints.
They've got their EDR and antivirus
software logs going to a centralized logging platform.
So they have a fairly comprehensive logging strategy
in place that they design through the entire process
of planning out that secure architecture.
So as they're going through their day
and through their regular activity,
they start seeing the logs indicate
they're getting multiple different phishing
emails all of a sudden.
But because they are alerted to that,
and they can see it easily, they just
block the IP address that's sending those messages.
They block the URLs that are in those malicious phishing
messages, so users can't click on them.
This can be an automated process, depending
on how the system is set up.
They also see multiple different port scan attempts
against their firewall from a couple of different IPs,
but not many.
The occurrences of this have definitely increased.
So they just blocked those IPs also, and the port scans stop.
They show multiple different attempts to brute force
different accounts.
Some of those accounts exist.
Some of them don't exist.
It looks like whatever the attacker is doing,
they're trying to just guess usernames and passwords,
and they're trying to brute force those.
But because of other different controls
they have in place, whether it's account lockout, periods,
or multifactor authentication, those brute force attempts
are all failing.
So they begin blocking IPs for that as well.
What it boils down to is because of all the logging
they have in place, they're able to identify
all of these potential impacts to their network
before they become an actual breach, and all of the attempts
fail.
As a result, there's no new suspicious company
that appears.
Nobody starts undercutting their prices
with suspiciously similar products.
So the business continues to thrive just like it always has.
There's no suspicious activity.
There's no company.
None of the activities in that first scenario happen.
So you can see, as an example, how
something as simple as logging and monitoring
your infrastructure can really help the organization
and why careful planning of your logging setup
is so important when you're doing this design phase
and you're planning out your secure architecture.