To be effective, red teams must have key information about their targets.
They need to know data about network services they can explore,
technical details of systems they could attack,
as well as organizational resources to which they
could apply social engineering.
This information feeds the planning and execution
activities during the engagement.
Having the right tools to recon the target leads to more
efficient and effective red team testing.
Hello and welcome to Reconnaissance was Shodan.
My name is Keith Watson, and I'm an information security professional.
I hope you've enjoyed my other courses on information
security and penetration testing at Pluralsight.
Here, we'll turn our focus to a key tool for red teams, Shodan.
Reconnaissance is the first step in building a solid plan
for red team testing against a target.
It consists of both information‑gathering activities and an
analysis of the information collected.
This process is often the cycle as more specific information may be
needed as the analysis of the gathered data progresses.
The team may take advantage of open‑source intelligence resources,
as well as more active technical approaches that probe a
target for data directly or passive methods,
which utilize information maintained or gathered by others.
As information is collected,
we can categorize it based on its type as either technical
information about the target's digital assets,
administrative information about the target's network domains,
assigned IP address ranges, autonomous system numbering,
and outsourced services,
physical information about the location and characteristics of its offices,
data centers, ancillary facilities, and operations,
and collocated digital assets, and human information regarding employees,
leaders, and the organizational structure.
With that information collected,
an analysis is needed to determine which data elements would be most valuable
in the planning and execution phases of the engagement.
The first consideration should be the goals of the red team exercise.
Identifying the successful outcomes of the engagement will
narrow the focus to relevant data elements.
During the analysis,
the rules of engagement must be considered as not all data elements
might be allowed in execution of the engagement.
From the analysis will come actionable targets on which
the team can direct their efforts.
The red team may plan to target specific business assets or try
to manipulate a set of business processes.
The outcome of the gathering and analysis activities then
feeds into a threat modeling phase.
Shodan brings a new dimension to reconnaissance for red team
operations by exposing detailed information on internet‑connected
systems through an easy‑to‑use interface.
John Matherly, a programmer with a background in data mining and web development,
created Shodan and launched it in 2009.
The project started as a curiosity about what could be learned about
devices connected to the internet and continues to be a leading source
of information about the Internet of Things,
including building control systems, industrial control systems,
desktops, printers, voice over IP, surveillance cameras,
and many other types of network‑enabled devices.
Prior to Shodan,
we had web‑focused search engines that gathered human consumable content.
Shodan crawls the internet and reveals details about the
various devices with public IP addresses.
Similar to a web search engine,
the Shodan data is easily searched and is very detailed.
Shodan crawls the internet on a 24/7 basis and
updates its database in real time.
It also has crawlers in at least eight countries distributed
around the world for a different view of systems and to avoid
countrywide blocking that may be in place.
As it crawls systems,
Shodan gathers banner data from specific services it understands.
It also records various bits of metadata.
With that information in its database,
users of Shodan can search that banner text and apply
various filters to refine the search results.
All the search capabilities are available to anyone
through the Shodan website at shodan.io.
Shodan also has a command line tool.
Unlike Google or Bing, Shodan searches for systems behind the World Wide Web.
It knows how to interact with specific types of
network services to capture banner data,
as well as metadata about the device, such as its operating system,
software versions, configuration data, host names and IP addresses,
and even its geographic location.
There's an API for integration with other red team tools,
and the data available in Shodan can be used in planning red team operations.
Shodan is a reconnaissance tool and sits squarely in the
recon space in the cyber kill chain.
The information gathered and analyzed during the reconnaissance phase
aids in progressing through other phases of the chain.
The MITRE ATT&CK framework is a knowledge base of
attacker tactics and techniques.
You can see the high‑level tactics listed here.
Red teams can use MITRE ATT&CK to replicate known attacker
methods when conducting an engagement.
Shodan can be a useful tool for red teams at the start of an
operation. It specifically provides information for the first
attack framework tactic of reconnaissance.
Specifically,
there are three relevant techniques, T1590, gather victim network information,
T1592, gather victim host information, and T1596, search open technical
databases. For T1590, the metadata collected and shown by Shodan corresponds
to the following subtechniques, .002 DNS and .005 IP addresses. For T1592, the
banner data collected and shown by Shodan corresponds to the following
subtechniques, .001 hardware,
.002 software, .003 firmware, and .004 client configurations.
For T1596, Shodan an is a database of sorts providing a plethora of
technical details about systems it has discovered.
These include the following subtechniques, .003 digital
certificates and .005 scan databases.
Let's take a quick look at the architecture used by Shodan.
We start with the users that want to understand
systems that exist on the internet.
Shodan,
realizing this need, created a system of search engine crawlers
that focus on the Internet of Things and collect banner data and
metadata about the systems found, which is recorded into a database.
Shodan created a web interface and API to make that data accessible and makes
it available to users and for integration with tools. I mentioned that Shodan
collects banner text from network services.
Here's an example of an HTTP response from a web server it might collect. In
the blue text, we see the HTTP response code. In the orange text are the
HTTP headers and, in black, their values. Here, at the bottom, is the most
interesting part of this information.
We can see the server header field.
This web server is powered by version 1.18.0 of the Nginx
server software coupled with version 6.0.8 of the Phusion
Passenger application server software.
As you use Shodan, you'll find some very interesting
information about the devices on the internet.
Just remember that Shodan is gathering this information in
order to protect systems on the internet.
Our use of this information is to further that goal. Viewing this
information is not illegal. Acting on the information without
permission can be. As with any engagement,
be sure to have permission of the system owners.