[Autogenerated] so that's a pen test methodology and
the sort of people should use for it.
We got two sets of pent is where she three sets of pen test to do.
Now, we've got first one on external ___________ test annually.
And after that, significant changes.
Yeah,
on then an internal ___________ test and then a segmentation
___________ test if you're using segmentation to have your
your scope production effectively, Although,
although it's the same methodology,
so they will be done by the same person at the same time.
Eso no end.
Sure, right.
In the sense that no, they don't always have the same methodology.
But maybe they can be done by the same person of the same firm,
you know, in the same sort, of course of activity.
Right.
That could be the case.
Uh, so So, yeah, the external, The internal ___________ tests are are,
you know, annual or after significant change.
I find a lot of organizations really sort of skate on that significant change
because the pen test is often a fairly expensive endeavor.
Right?
And I don't wanna look cheaper than being compromised by criminal.
Yeah, yeah, that's true.
That's true.
Um, if only that argument worked as well as it seems like it should, Right.
Um but but a lot of organizations say, Well,
you know, that wasn't really a significant change,
even though we just like tor down from this one data center
and move everything over to this other day.
But do they know I have assurance functions that say,
you know, the massive I mean, ignoring PC IDE s s any sort of change like that.
I'd want to do my own pan test before the criminal do that.
You would think you would think.
Um, but yeah, the goal.
Right.
The external pen test.
What can the attacker get from an internet perspective?
Right.
What can they do?
Right?
And that should that should cover sort of network layer or
infrastructure layer and application layer right.
The language and standards as network layer and application layer right
and and read network layer as infrastructure layer.
Right, um, and application layer.
Right.
And And that that matters, right?
And the internal test.
Let's take on the external because I wanna ask a really naughty
question that people always ask about external pan test.
Should I white list the pan test from my ideas.
Right?
But what's the point?
Because, you know, I'm gonna run the argument that I hear.
But look, that's what I've got on the network.
So that's what a real criminal would see.
And therefore the pipes or the wife and everything will just stop the activity.
Well, okay.
My counter argument to that is the The pen test is a sort of focused,
limited time and effort kind of activity,
right?
You want to You want to pay the pen tester?
You know,
the reasonable sum for them to find what they're gonna
find in the reasonable timeframe, Right?
A to real attack.
Right on attacker who is motivated by financial
gain by harming your organization, uh,
can use techniques to operate more slowly or to operate more surreptitiously,
maybe take more time and effort, uh,
but can evade defenses like an intrusion detection
system or web application firewall.
Right.
So if he was a Q s, A c a pen test,
where they weren't white listed where the external pen test was just blocked,
by the way for by the ideas.
And you see a report that says that you're not accepting it?
No, I wouldn't accept that now.
I think it's a reasonable goal first to white,
list the tester and have the test to do their testing.
And then,
as maybe they're sort of ultimate step before they wrap testing to then unlike,
list them and make sure that your detective controls error,
actually working as advertised, right?
Okay.
Now go ahead and do some other testing just to see if my ideas picks it up.
That's a reasonable testing goal, right?
And maybe you should stick that in your pen.
Testing methodology.
Okay, that's a That's a really good TIFF, actually.
Now, let's talk about the internal pen test.
Sure.
Again, How does that happen?
What?
Walk me through what you're looking for because you're coming
outs and center at the end of the year, right?
You hope they've had an internal pen test?
What are you looking for in that?
I want to I want to make sure that the internal tester,
I mean, likewise with the extra faster.
I wanna make sure they understood that scope, right, right?
And that they tested all of the relevant portions of the
environment from an interior vantage right.
Similarly to the Vulnerability scan thing right?
What can you see from the inside now?
The sort of tactical way of going about it might
have a few different approaches, Right?
You might send your internal pen tester on site and they plug into some
spot and do the testing in person while they're there.
Right?
And and sometimes that makes the most sense.
And sometimes you know it's it's more travel costs than is really necessary.
But that's one way to do it.
Um,
you could conceivably have them connect over like a VPN from
wherever they happen to be into your environment.
Um uh,
tunneling pen testing traffic over VPN is often sort of problematic in terms
of performance or or other sort of connectivity reasons.
So most pen testers that I'm aware of really don't prefer that approach the
sort of best approach I've seen for for a sort of limiting,
uh, those kinds of problems and then not,
you know, sending your pen testers all all over the country.
Ises UI have the client, you know,
set up a physical or a virtual host right apprentice box that
phones back home to the tester and then allows the tester to
connect into the environments right?
And and run their utilities.
Right?
So they're they're testing again, the network sort of layer.
And then and then conceivably the application layer for
whatever applications are are visible there.
Yeah.
I mean, we I mean, you know, typically,
we spend API UNIX box and give the tests and remote access to like,
there's your box.
Do are practice has a stable of physical boxes of
somebody insists on physical box, and then has a standard,
like virtual build that they can put in a sftp site or whatever.
And have somebody download.
Okay, Um, so you look when you get the pen test,
you looking back at the methodology and checking that the
actual pen test to follow their methodology.
They said we're going to, right?
Where do you get failures in here?
So both internal and external pen test where the people fail,
whether people become non compliance.
So I find a lot of lousy pen tests where the tester either just wasn't
very good or didn't put a lot of effort into it.
You know, I found I found one of those sort of empty reports where,
uh,
like we couldn't find anything we couldn't get access to anything like
What do you mean you couldn't get access to anything?
Do you think that's because the climb said I'm gonna I'm gonna pay.
I'm gonna pay this right.
This only buys you a lousy pens.
Us That.
I think I think that is that is the maybe not expressly stated,
but fundamental reason for for a lot of things.
I think a lot of clients don't necessarily realize that a $7000 test is garbage.
You're better off giving that $7000 to a monastery and
having them pray for your network.
Um A And I think Ok, well,
I'm getting I'm getting something that's a lot less than of,
you know, $40,000 test by, you know, a more reputable firm.
Uh, right.
But the tester maybe doesn't have the time to do that much
effort because they have this tiny budget on.
So I think you need to be sort of careful about that.
Um, but if you're if you're sitting there is an assessor.
It's the end of the year.
You're doing your annual DSS assessment.
You see,
a lousy pen test with a lousy methodology is that failure e had a pan test.
It doesn't say I have a decent fantastic.
Just have a pan test.
He could say have a lousy pants.
Right?
Well, but does it find Doesn't find the vulnerabilities that,
frankly, I'm finding right by looking at it from the other perspective.
Um, and you know, the methodology sort of requirement helps me in this, right.
Did it test your entire scope?
Right.
I find pen test that, you know,
have have some meaningless boilerplate front matter kind of text and then
immediately launch into some grid of mostly useless findings that looked like
they came straight out of a vulnerability scanner.
Because because Because that's precisely what happened, right?
Like, you know, barely any text formatting or anything, right?
And there's no description of scope in the report.
And there's no description of methodology, right?
I mean, it's pretty implicit.
It's, you know, hit the start button on the tool.
Uh, go get tacos and then and then send the report to the client.
Right.
But want to to APIs that you going to?
Yes.
Do going to to quite well what I have done, right?
What I've done in cases where I found this,
particularly the last handful of years, uh,
is that I've been a little bit fortunate in that my assessment work.
Our assessment work comes in relatively shortly after the test has concluded,
as the months and months right.
And I look at the pen test report and say, There's no substance here.
There's no methodology, There's no scope.
I have no assurance that they've tested any of the
stuff that they're supposed to.
There's no description of any of the segmentation
testing like this isn't gonna work right,
And and one time to my clients credit.
And I feel sort of bad for the tester.
You know, we were on the phone.
Uh, they basically put us on hold, called the tester and,
like,
joined this poor guy to the conference call where I I then
proceeded to sort of roast him alive.
Uh, and you know, then, in that case,
that firm then went back and actually did some more testing,
Uh,
presume presumably brought us for the client and then
yielded some more useful results.
Right, Right.
But I think what What I'm hearing is the best advice is to get
your QoS a early to help you by your pen test,
I'd say Engage your que se for helping you select a pen test fender,
right?
You've seen good ones of balance.
You have a good view of the market, right?
Right on.
And I would say, You know,
it's not even so much like I know this firm always does good work,
and I know this from always does lousy work.
But let's talk to the tester, right?
Who's going to be staffed on this particular test?
Get a sense of that person's expertise,
Get a sense of this person sort of grasp of the client environment and in,
you know, does that make sense, right?
You know, does the does the scope of that engagement cover right?
And really,
there's There's some onus on the client to make sure that
what they're agreeing to is appropriate,
right?
I mean, even our Prentice practice says, you know, we're a pen test practice.
We're not a PC.
I pen test practice, and we're not the que se right.
The client says, I want you to pen test this.
I want you to do these things in the pen test.
We're going to do that.
And if that's insufficient for their PC IDE is that's compliance purposes.
Even if it's my colleagues over here in my same company that are
the que se is like this was in our agreement.
This is what we said we were gonna do.
So because I mean,
my reflection is that people don't know how to buy pan test very well.
Which is why you should actually hire someone to help you speck out,
too by pants, right, e frankly, again,
I would say if I'm doing your assessment in a couple of months,
call me.
I would just assume not charge you to help you get the right pen test vendor
right then have to have all of the extra time and grief and budget overrun in
the assessment to go deal with the lousy pen test.
Okay, now,
getting the test is just one bit because there's a requirement that you don't
have to fix things that are brought up in the pen test.
So there's a queue.
Say, when you do an assessment, you're looking for that fixing, right?
Absolutely.
And that's you're actually gonna go through.
And so show me a retest.
Show me a retest.
Yeah, right.
Uh, in some cases, right now,
pen test vendors that are that are sort of moderately clever about this.
Have some time limit on how long they will sit around waiting for you to
request your retest as as the sort of tested entity.
Right?
And if you come back six or eight months later and say,
Hey, I'm ready for a retest, they might say,
That's nice, right?
And that's fair because in many cases where I've been in the
situation where the _________ vendor or will bend,
the pen test vendor hasn't been sort of clever enough to cap that off.
Right now, six or eight months have passed and the environment has changed.