Using nmap for Continuous Vulnerability Monitoring

I don't have to tell you how important it is to make sure that your systems are not vulnerable to attacks. There are a number of complementary measures that you should implement. Here is an (incomplete) list:

• Keeping your software supply chain secure
• Protecting your systems from outside access through Firewalls and VPNs
• Making sure that the operation system is always up-to-date in terms of security patches
• Continuous scanning for vulnerabilities on the systems
• Continuous scanning for vulnerabilities from the outside

In this post, we want to focus on the last item on this list. Using a vulnerability scanning tools to monitor your systems externally.

It is important to not only perform the scan now and then but continuously: New CVEs are discovered all the time. Also, your system might change over time for any number of reasons (updates being applied, new software is installed, etc.)

A very powerful tool for vulnerability scanning is the venerable nmap. As an example, we will use nmap to monitor the SSH daemon on our servers for vulnerabilities.

Take the following command:

nmap -p 22 -sV -oX /tmp/nmap-output.xml --script vulners example.com

• -p 22: Only scan port 22 (SSH)
• -sV: Detect the version of services based on probing them
• -oX /tmp/nmap-output.xml: Output the result of the scan as an XML file
• -script vulners: Use the vulners script ((more info)[https://nmap.org/nsedoc/scripts/vulners.html]). This will use the detected version and check the API of https://vulners.com for CVEs.
• example.com: The server to check

An example output could look like this:

22/tcp  open  ssh      OpenSSH 8.9p1 Ubuntu 3ubuntu0.6 (Ubuntu Linux; protocol 2.0)
| vulners: 
|   cpe:/a:openbsd:openssh:8.9p1: 
|_      CVE-2023-51767  3.5 https://vulners.com/cve/CVE-2023-51767

nmap has found out that the version of the OpenSSH server is 8.9p1. For this version there is a CVE for this version.

You can of course do this as well for other services that are running, e.g. for web servers running on ports 80 and 443.

Our goal is to not only perform this scan once, but do it continuously. For this we will use Checkson. We will write a script that performs the scan using nmap and report a failure to Checkson if any CVEs are found. As an added bonus, we will create an attachment to the check run with an HTML report of the findings. First, the script. We will use Python for this and invoke nmap via the subprocess module:

from subprocess import check_output, call
import xml.etree.ElementTree as ET
import os
import sys


def main():
    cmd = "nmap -p 22 -sV -oX /tmp/nmap-output.xml --script vulners example.com"
    nmap_output = check_output(cmd, shell=True)
    print("nmap output was: ", nmap_output)


if __name__ == '__main__':
    main()

As you can see it performs the scan using nmap. It saves the output to an XML file: /tmp/nmap-output.xml

The output XML file looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<nmaprun scanner="nmap" args="nmap -p 22 -sV -oX /tmp/nmap-output.xml -&#45;script vulners example.com" start="1712122535" startstr="Wed Apr  3 07:35:35 2024" version="7.80" xmloutputversion="1.04">
  <scaninfo type="connect" protocol="tcp" numservices="1" services="22"/>
  <verbose level="0"/>
  <debugging level="0"/>
  <host starttime="1712122535" endtime="1712122536">
    <status state="up" reason="syn-ack" reason_ttl="0"/>
    <address addr="100.100.100.100" addrtype="ipv4"/>
    <hostnames>
      <hostname name="example.com" type="user"/>
    </hostnames>
    <ports>
      <port protocol="tcp" portid="22">
        <state state="open" reason="syn-ack" reason_ttl="0"/>
        <service name="ssh" product="OpenSSH" version="8.9p1 Ubuntu 3ubuntu0.6" extrainfo="Ubuntu Linux; protocol 2.0" ostype="Linux" method="probed" conf="10">
          <cpe>cpe:/a:openbsd:openssh:8.9p1</cpe>
          <cpe>cpe:/o:linux:linux_kernel</cpe>
        </service>
        <script id="vulners" output="&#xa;  cpe:/a:openbsd:openssh:8.9p1: &#xa;    &#x9;CVE-2023-51767&#x9;3.5&#x9;https://vulners.com/cve/CVE-2023-51767">
          <table key="cpe:/a:openbsd:openssh:8.9p1">
            <table>
              <elem key="id">CVE-2023-51767</elem>
              <elem key="is_exploit">false</elem>
              <elem key="cvss">3.5</elem>
              <elem key="type">cve</elem>
            </table>
          </table>
        </script>
      </port>
    </ports>
    <times srtt="33172" rttvar="25279" to="134288"/>
  </host>
  <runstats>
    <finished time="1712122536" timestr="Wed Apr  3 07:35:36 2024" elapsed="0.95" summary="Nmap done at Wed Apr  3 07:35:36 2024; 1 IP address (1 host up) scanned in 0.95 seconds" exit="success"/>
    <hosts up="1" down="0" total="1"/>
  </runstats>
</nmaprun>

Let's extend the script to parse the CVEs. While we are at it, we will add the possibility to ignore CVEs that we have analyzed, but that we consider harmless, e.g. because there is no way to practically exploit them. We will use an env variable called $IGNORED_CVES.

from subprocess import check_output, call
import xml.etree.ElementTree as ET
import os
import sys


def main():
    cves = []
    ignored_cves = os.environ.get('IGNORED_CVES', '').split(',')

    cmd = "nmap -p 22 -sV -oX /tmp/nmap-output.xml --script vulners example.com"
    nmap_output = check_output(cmd, shell=True)
    print("nmap output was: ", nmap_output)

    xml = ET.parse('/tmp/nmap-output.xml')
    root = xml.getroot()
    xpath = './/host/ports/port'
    for port in root.findall(xpath):
        table = port.find('script').find('table')
        for vulnurability in table.findall('table'):
            vulnurability_id = vulnurability.find('.//elem[@key="id"]').text
            cve_score = vulnurability.find('.//elem[@key="cvss"]').text
            print(vulnurability_id, cve_score)
            if vulnurability_id not in ignored_cves:
                cves.append(vulnurability_id)

    print("Non ignored CVEs: ", cves)
    if len(cves) > 0:
        print("At last one relevant CVE found, exiting with status 1")
        sys.exit(1)

    print("No relevant CVEs found, exiting with status 0")
    sys.exit(0)


if __name__ == '__main__':
    main()

This will exit with exit code 1 if there are non-ignored CVEs. An exit code that is not 0 will tell Checkson that the check should be CRITICAL and notifications should be sent out.

We can execute the script and ignore certain CVEs with this:

export IGNORED_CVES="CVE-2023-51385,CVE-2023-51384"
python3 check.py

Let's add one final thing: A report of the findings in HTML format. For nmap XML output this can be achieved with XSLT for turning XML into HTML. There is a command line tool for that: xsltproc. We will simply invoke that from Python. The file is placed in the $CHECKSON_DIR/attachments directory, so it will be made available by Checkson as an attachment. This attachment will be accessible via the Checkson web app. The complete listing of the Python script is this:

from subprocess import check_output, call
import xml.etree.ElementTree as ET
import os
import sys


def main():
    cves = []
    ignored_cves = os.environ.get('IGNORED_CVES', '').split(',')

    cmd = "nmap -p 22 -sV -oX /tmp/nmap-output.xml --script vulners example.com"
    nmap_output = check_output(cmd, shell=True)
    print("nmap output was: ", nmap_output)

    create_html_report()

    xml = ET.parse('/tmp/nmap-output.xml')
    root = xml.getroot()
    xpath = './/host/ports/port'
    for port in root.findall(xpath):
        table = port.find('script').find('table')
        for vulnurability in table.findall('table'):
            vulnurability_id = vulnurability.find('.//elem[@key="id"]').text
            cve_score = vulnurability.find('.//elem[@key="cvss"]').text
            print(vulnurability_id, cve_score)
            if vulnurability_id not in ignored_cves:
                cves.append(vulnurability_id)

    print("Non ignored CVEs: ", cves)
    if len(cves) > 0:
        print("At last one relevant CVE found, exiting with status 1")
        sys.exit(1)

    print("No relevant CVEs found, exiting with status 0")
    sys.exit(0)


def create_html_report():
    checkson_dir = os.environ.get('CHECKSON_DIR', '/tmp')
    if not os.path.exists(checkson_dir):
        os.makedirs(checkson_dir)
    call(f'xsltproc /tmp/nmap-output.xml -o {checkson_dir}/attachments/nmap-output.html', shell=True)


if __name__ == '__main__':
    main()

In order to be able to deploy this check to Checkson, the only step left to do is to wrap it in a Docker container. The following Dockerfile installs the required pieces of software nmap and xsltproc:

FROM alpine:3
RUN apk add --update nmap nmap-scripts libxslt bash python3
ADD main.py /check.py
CMD [ "python", "/check.py" ]

You can build this Docker image locally and test it:

docker build . -t myuser/sshd-scan:1.0
docker run --rm -it -e IGNORED_CVES="CVE-2023-51385" myuser/sshd-scan:1.0
echo "Exit code was: $?"

Everything is ready for the check to be deployed to Checkson. Please refer to the guide on how to to it using the CLI or to the guide on how to do it via the web UI whichever you prefer. It only takes 5 minutes.

After the check is deployed, you will receive an E-Mail or Slack message when new CVEs for the SSH daemon on your server of servers are discovered.

This was an example of using nmap for vulnurability monitoring. Of course, we only touched the surface. There is a whole category of scripts for nmap for all kinds of vulnerabilities.