We talked about detection, and, of course,
the next logical step is going to be response.
So, thinking about response, once you've identified this,
you're in the oh crap moment, like, now what do we do?
‑Yeah, so once you've found a business email compromised,
there are absolutely a number of steps that you want to go through first.
And again, we're talking Office 365 here,
these steps still apply to any other email system that you might have access to.
So,
the first thing you're going to want to do is you're going to
want to set it up so that you force a password change across
all of the compromised users.
If you don't know who all the compromised users are,
then just do a forced password change across everybody.
I know people don't, or organizations don't like to do that,
but sometimes it's something that has to be done.
So, once you set that forced password change,
then the next thing you want to do is you want to kill
all active sessions in Office 365.
What that means is you go into the users,
there is a button in there where you can say revoke active session.
You want to click on that or write a PowerShell script to
go through and do it for everybody,
and what that does is that kills any active session that's there,
so if an attacker is still able to get into the network
because they have that session open, that's going to stop that,
that's going to remove that.
This is important to do because even if a user changes their password,
those active sessions could be valid for up to 6 hours,
depending on how the user has accessed it or how the attacker has accessed it.
‑Yeah, no kidding.
And likely with this dwell time they do have active sessions open,
so that's something that's nearly a requirement for this.
‑Yeah, absolutely.
They will, and so you absolutely have to do that.
And then the next thing you want to do is you want to find those inbox
rules that we were just talking about and remove them.
You want to prevent the attacker from getting any more emails coming in.
And the best way to do that is to remove those inbox rules.
You know,
another way that you could do this is just completely disable the
user's account and then put in blocks to prevent any emails from going
out from that user or coming into that user,
depending on how you want to set it up,
but you absolutely want to go in and remove those inbox rules.
‑So, as far as like response objectives,
our first things are like we've shown up,
we know what's happening, we need to reset accounts,
we need to disable sessions, or revoke sessions,
active sessions,
we need to lock down those inbox rules or remove them completely,
and maybe we start from scratch,
I haven't been too many places where inbox rules are absolutely required,
the user can go set them back up, it's okay,
and/or you need to figure out what was accessed.
Now, to do all of that, you kind of have to know what was the scope,
how many people were actually compromised,
where were they coming from, all of that stuff.
How do you find all those things?
‑Right.
And that goes back to the investigation,
doing the investigation that that we talked about.
You know, what we just mentioned,
the forcing the password change and killing the
active sessions and and all that, that is,
we're reaching containment,
that's to contain the incident and eradicate the attacker.
Basically,
stopping the bleeding or preventing the attacker from
maintaining access in the environment.
But you're right, you need to define those rules,
you need to figure out all those people who have been compromised.
And, we went in,
you showed your PowerShell script to pull the logs out and do that,
I showed how to get into the audit logs and do that,
but, you know, not everybody is going to have the time to do that manually,
you need to have some automated process.
And so, fortunately,
there are a couple of tools out there that will kind of
pull all these things down for you.
And so,
we can show how to use one of those tools to make the investigation at least a
little bit easier so you can go in and find that information.
‑Yeah, please do, I love open source tools,
and I think the more we show, the better.
So, getting rid of that scrolling through logs like we just said,
that's not really tenable for this kind of investigation.
And the other thing about automation is humans are less likely to
miss stuff if it's being automated by the computer.
Like if I'm like, oh, I'm looking for 188,
is what I think the IP address was from Vietnam,
if I'm looking for that, I may miss it as I scroll,
whereas that may not happen with the automated tool.
‑Absolutely.
So let me do this, let me share my application here.
‑Yeah, show me what you got.
‑All right, so, you're going to love this, it's a PowerShell script.
‑I love PowerShell.
‑There is a project out there called KITT‑0365‑Tool,
it's on GitHub, and it's got this really great PowerShell script.
So, all you need to do,
once you install all the prerequisites and all that
which are on the GitHub website, is you run it,
yes I do want to run this,
it's going to create a transcript of everything that it does,
so I'm just going to throw that on my desktop for the moment,
and then it's going to ask you to sign in.
You do need to sign in with somebody with admin privileges to Office 365.
So, I'm going to sign in to that, into our demo corp that we set up.
‑This already looks way cooler than my PowerShell script.
‑So it's going to do that,
and then it's going to start processing the information.
It's going to start downloading things,
it's going to start grabbing everything that it needs,
and you can see how it's doing that.
Obviously, you know, we have a very small domain here,
I think what, seven users,
I think when we looked at all events there was like maybe 500 events.
You can imagine that with a larger organization
that this could take a lot longer.
‑Yeah, it seems like it'd take quite some time.
And another thing for information timeout,
this is probably one of the first things you want to do,
right?
You don't want to lose any data?
‑Absolutely.
Depending on your license,
the audit logs may only be kept for 7 days or potentially up to 90 days,
again, it depends on your license.
So as soon as something like this happens,
you need to get in there and grab that information out.
‑It sounds to me like someone needs to redo the order of
volatility to include all these cloud services as well.
‑I agree, that absolutely needs to happen.
‑That's a nice, new NIST document that someone needs to get on,
other than me, right away.
‑So, the application has popped up this nice little GUI,
it's all GUI‑driven, even though it's PowerShell,
you don't have to worry about modifying anything like that.
I will say you will have to go into the PowerShell script
before you run it to modify a couple places,
but again, read the documentation, it'll tell you that.
But the first thing I'm going to do is I'm going to go in
and I'm going to tell it the date range.
I'm just going to go this week,
and then I'm going to click Search and it's going to ask,
well, what user do you want to look at?
We're like, we're going to cheat a little bit,
we know that Kim is the one who's been compromised,
so let's just choose Kim, and then I'll click OK.
And then, it's going to be doing a bunch of things,
right now it's pulling back all of her information.
So here on the left pane, we see the user information for Kim,
we see when she was last logged on, when her password was last changed,
and so on.
On the right‑hand side there's some additional information about her,
but we are interested in a couple of things.
We want to see her inbox rules and we want to see the logins.
So I'm just going to jump over the Logins first.
And here, unfortunately, it doesn't do the GoIP lookups,
but you know, it's PowerShell, it's open source,
we could add that in if we wanted to.
I expect you to have that done by tomorrow, Aaron.
‑Yeah, I'm doing it right now.
‑But we can at least see all the different IP addresses
that Kim's user ID logged on from.
Additionally, you remember, we were talking about the user agent,
you can actually see that user agent here,
all of the different user agents that she's logged in as.
She logged in from using Edge and on Firefox.
And we can, again, have that conversation with her to ask,
hey Kim, what browser do you use when you access your email?
Or do you only use like the Outlook Client because the Outlook
Client will have its own user agent as well.
So this absolutely is a lot easier than us downloading the logs,
grepping through them, pulling out what we need,
or even looking through the web interface because we might have to scroll
through page upon page upon page to see all of this.
This kind of gives it to us right here.
‑Certainly.
And if you look at this initially, you also get stuff like,
hey, if you've had this environment for quite some time,
you're going to have hundreds of logins from a user and then just a few
logins that are probably from the malicious actor,
right, it's not going to be equal numbers.
That's a bit of an indicator as well.
‑Right, absolutely.
Now, this is her logins.
Let's let's go over and look at her inbox rules.
And it automatically pulled down all of her inbox rules,
and you can see that it's actually a much nicer interface
than what the audit logs were showing us.
The audit logs,
we kind of had to go through the JSON that it was showing us in
order to kind of parse that out ourselves.
The PowerShell script here does it automatically for us.
So I'm going to move to Rule 2 first because this is the one that's
a little bit more interesting where it shows,
if the message, the body of the message contains the words wire,
wiring, fund, and so on, for the email to that,
oh in this case, it's Kimberly.logan@outlook.com instead of ProtonMail.
If you remember when we were looking at the audit logs,
we saw that there was a create inbox rule,
and then there was an update inbox rule.
‑Oh right, the set inbox, yeah.
‑Right, we never looked at the set inbox rule, which would update it.
And so, this is,
the attacker obviously went in and updated it from ProtonMail to Outlook.com.
‑That's awesome.
That's really cool attention grab to, again, go check the set inbox stuff too.
‑Yeah, exactly.
So this is showing us a lot of information.
One other thing that we can look at,
remember I said that if one user is compromised,
chances are another user is compromised as well.
Well, again, we know that these 188 addresses are the attacker's addresses,
so let me ‑‑‑And at this point, that's super valid because we've already,
we're in response now.
We've done detection, we know that this is bad,
we've gone and validated that in response mode.
It seems like cheating now just because there's not a lot of data,
but this is, you'd already be at this point.
‑Yeah, exactly, exactly.
We're just kind of,
you would normally probably do this at the beginning of your
investigation to just kind of pull all the information,
or in the middle of your investigation, just to kind of validate some things,
make sure you're not missing anything, and so on.
But, I'm going to copy this IP address, go to Logins by IP,
nope, there's something hidden over here.
There we go.
So I'm going to type in that IP over here and then I'm
going to click Search Using Fresh IP.
And in this case, it shows us that yes, from this IP address,
this .76 IP address, Kim was the only one who logged on.
‑Got it.
But if there were other users.
‑Right.
Well, there is a second IP address, the second IP address in this range.
So let's look at this one.
Let's see if this pulls us anything.
In this case, it kind of confirms for us that,
yes, Kim was the only one that, at least using these IP addresses,
that the attacker logged in under.
Which is good, it would mean that we are restricted to just this.
‑Though if we had seen that from multiple users,
this now helps us detect, okay,
what's the extent of this compromise of the users?
Now, I suppose, realistically, depending on the access of an individual user,
they don't really need more than one, depending on where they land,
but, you at least can figure out,
if you try to figure out like whose accounts need to be reset right now,
it's this list of accounts.
‑Yeah, absolutely correct.
One other thing that you can look at is it has this Message Trace tab.
So what we can do is we can start looking to see if that logging is in place,
what email addresses were receiving, sending and receiving emails.
So I need to enter in one user email address.
So I'm going to copy Kim's,
and we're just going to go in here and say I want to
see all the emails that Kim sent out.
‑Oh, great.
‑And here we can see it kind of shows us all of the emails,
the list of senders, the recipients, how many were delivered and failed.
You can see here that there was one email to our
ProtonMail account that went out,
and there were two emails to our malicious Outlook account.
So, because of that,
now we know that at least three emails went out to the attacker.
‑Absolutely.
And that starts to scope some of the information that was probably lost as well.
‑Yeah, absolutely.
And we can go in and we can download these message trace logs.
In fact, if we wanted to click View Message Trace Logs,
now we can actually go in and see what these emails were.
So if I wanted to see what email was sent out to this fake Kimberly Logan one,
now we can see at least the subject, whether or not it was delivered,
when it happened, and then we can go in,
hopefully into the user's sent items folder and see what that email was,
or at least start looking through the inbox to see what it is.
Unfortunately, we can't pull it up here,
but we can at least pivot off that information into their inbox.
‑Yeah, and as I'm looking at this too,
I see the status of failed ends up being highly associated
with those two malicious emails also,
certainly not an indicator that every failed message is bad,
but if you're already in this scenario where you have other stuff going on,
attackers make mistakes, and/or I believe in some of these cases it was,
you couldn't, like you said, it said invoice,
it wouldn't allow sending of invoices.
Like that was something that got caught by whether it was
ATP or one of the Defender products.
‑Right.
Now, the thing to keep in mind, and this is,
I think,
a great place for us to kind of pivot over to talking about the licenses because
these message trace logs are not always going to be there.
So, let's talk about the licenses just a little bit.
So, we are using what's called the E5 license within Office 365.
That is kind of not, that's not their very top tier, but it's pretty close.
It's like their second from the top.
I think they're top tier is like F5 is what it's called.
For the E5 license,
the most important thing for us is that they turn on a
lot of the audit logs by default.
So things like the mail items accessed are going to be logged by default,
or at least shown to the user by default.
Now, unfortunately, most organizations don't have E5 because it costs more.
Most organizations have E3 or below.
And Microsoft has a couple of levels below E3,
but most organizations have at least an E3 level.
And what that means is, if you go into those audit logs,
or you run the kit PowerShell tool that we just did,
you're not going to be able to necessarily see all the information.
And in my experience, you'll still see the inbox rules being created,
you'll see the signins and logons, but that's it.
Now,
in order to see all the other things like what the attacker actually accessed,
which mail items they accessed, you need to either have E5 or,
and again, this gets a little bit confusing,
if you have E3 you have to specifically go in and use
PowerShell to turn on that auditing for that,
for specific users.
It's there, Microsoft will record it,
but you don't get to see it unless you turn it on.
And unfortunately, you only get to see it when you turn it on going forward.
So Aaron, if you got compromised right now and we had E3,
and we didn't discover it until next week and we turned
your auditing on under E3 next week, we wouldn't see any of this data beforehand.
‑Right.
And we wouldn't catch those inbox rules,
we wouldn't catch any of that from there.
‑Exactly,
so this is definitely something that you, as administrators,
want to go in and see what is turned on.
I would highly recommend actually going in and users who are
going to be handling sensitive data like this,
that are going to be handling invoices or who are more
susceptible to these types of attacks,
to going in and turning on that mailbox auditing or
upgrading their license to E5 just in case.
‑I see what you're saying now, it's at an individual level.
‑Right?
For E3 it is at an individual level.
‑So you have to go turn it on for a very specific user,
and, of course, that takes up more log space,
which means it probably costs a little bit more on the storage end of it,
too.
And to be clear,
this isn't a sales pitch for buying a higher level of premium
service for Azure or any other cloud service,
but what I do think is an important note,
and you just brought it up now, was the sensitive data component.
Maybe this is where you start looking at,
if we are an organization that has the responsibility to manage sensitive data,
and again that's a word to a reasonable amount of security,
like the legal phrase within the regulatory bodies that
want to come look at this after the fact,
is are you protecting it within a reasonable amount?
And the reasonable amount could be like,
well, of course, should you manage sensitive data,
you probably should turn on the audit logging for
users who manage the sensitive data.
And that's, I just wanted to make that delineation for,
here's a good point to maybe look at that and maybe where
you should invest a little bit more money.
‑Right.
And you just, I think you hit it on the head right there.
It all comes down to money.
Obviously the attackers, they're motivated by money here,
but our protections as well are also motivated by money.
And, you know,
if I'm a security person who's trying to convince management or
whoever controls the budget that we need to do this,
the way that I would phrase it is,
well we can spend a couple extra bucks a month to have these
logs sitting there in case we need them,
or, in the future if we get compromised by this,
we cannot have these logs and get hit with regulatory
penalties because we can't say whether or not sensitive
information was exfiltrated or accessed.
And that right there, those fines aren't cheap.
We're not talking a $10 parking ticket here.
We're talking potentially thousands or hundreds of thousands of dollars
or more fines that could potentially happen here.
Would you rather pay that,
or would you rather pay a couple extra bucks a month just to have these logs?
‑Yeah, absolutely.
I think that kind of takes us in,
we're moving into a wrap up of where that puts some of this stuff that
we're talking about doing for detections and response,
and what level you need to go to to be able to protect yourself.
We can get into some of that risk management stuff here in this last bit.
‑Okay.
Sounds good.