Automate ZAP With Docker

In the previous posts, you learned how to use ZAP with the Desktop client and via the command line with ZAP CLI. This post, you will learn how to use the Docker images which are provided by OWASP. This will even make it easier to automate ZAP, especially in a CI/CD pipeline.

1. Introduction

It is strongly advised to read the two previous posts about ZAP before starting with this one. You will need some files which were created in the previous posts. If you already have experience with ZAP, you can continue reading and use the files from the GitHub repository from directory zap2docker. The generated reports will also be available in this repository. This way, you will be able to compare your results.

In the previous posts, you were shown how to use the ZAP Desktop client and how to use ZAP CLI in order to automate the penetration test. However, OWASP also provides some Docker images which can be used for an automated scan.

You will again use WebGoat as vulnerable web application. If you followed the previous posts, it is better to start from scratch again and remove the Docker container you created.

$ docker rm goatandwolf

Since the application under test is running in a Docker container and ZAP will also run in a Docker container, it is necessary to create a Docker network. Otherwise it will not be possible to access WebGoat from within the ZAP Docker container.

$ docker network create zapnet

Next, create the WebGoat container within the just created network zapnet.

$ docker run --name goatandwolf -p 8080:8080 -p 9090:9090 -d --net zapnet webgoat/goatandwolf

Navigate to the WebGoat URL and create the user mydeveloperplanet with password password. This user will be used for authentication during the scan.

2. ZAP Docker Full Scan

The ZAP Docker image provides several scan possibilities. One of them is a Baseline Scan which will scan your application passively. The active scan, however, will give you better results and this can be accomplished with the Full Scan.

You will need the IP address of WebGoat within the zapnet network. This can be achieved with the following command. In the example below, 172.22.0.2 is the IP address where WebGoat can be accessed.

$ docker inspect goatandwolf | grep IPAddress
            "SecondaryIPAddresses": null,
            "IPAddress": "",
                    "IPAddress": "172.22.0.2",

First, you will scan the application without any user information. The complete list op options can be found here, below the used options are explained:

• --net: in order to add ZAP to the network together with WebGoat
• -v: this will map your current directory to the Docker image work directory
• -I: do not return failure on warning
• -j: run the AJAX spider in addition to the classic one
• -m 10: the number of minutes to spider for (just a safeguard, the spider takes less time than 10 minutes)
• -T 60: limit the total scan to 60 minutes
• -t: the URL to scan for
• -r: the name of the report for the results

$ docker run --net zapnet -v $(pwd):/zap/wrk/:rw \
  -t owasp/zap2docker-stable zap-full-scan.py -I -j -m 10 -T 60 \
  -t http://172.22.0.2:8080/WebGoat \
  -r 20210417-zap-full-scan-without-user.html

Just as you noticed when running the scan with ZAP CLI in the previous post, this scan will give you less results than expected. The spider does some work, but not enough and since you did not provide any user credentials, a large part of the application is not scanned.

In order to provide the user credentials, you can provide the context Webgoat.context you created last time. The only thing you need to do, is to replace localhost with the IP address in the entire file. Move the context file to the current directory in order that it will be accessible in the ZAP work directory inside the Docker container. You add the following two extra options to the command:

• -n: The context file
• -U: The user to use

$ docker run --net zapnet -v $(pwd):/zap/wrk/:rw \
  -t owasp/zap2docker-stable zap-full-scan.py -I -j -m 10 -T 60 \
  -t http://172.22.0.2:8080/WebGoat \
  -r 20210417-zap-full-scan.html \
  -n Webgoat.context \
  -U mydeveloperplanet

Running this command, results in the following error. It states that the URL is not in the context, but it is. Even if this would work, it is doubtful whether the spider would have found all of the URLs of the application. You have noticed in the previous posts that a manual exploration of the website together with a spider gave much more URLs to scan.

20432 [ZAP-ProxyThread-11] WARN  org.zaproxy.zap.extension.api.API - Bad request to API endpoint [/JSON/ajaxSpider/action/scanAsUser/] from [127.0.0.1]:
org.zaproxy.zap.extension.api.ApiException: url_not_in_context
    at org.zaproxy.zap.extension.spiderAjax.AjaxSpiderAPI.startScan(AjaxSpiderAPI.java:337) ~[?:?]
    at org.zaproxy.zap.extension.spiderAjax.AjaxSpiderAPI.handleApiAction(AjaxSpiderAPI.java:202) ~[?:?]
    at org.zaproxy.zap.extension.api.API.handleApiRequest(API.java:507) [zap-2.10.0.jar:2.10.0]
    at org.parosproxy.paros.core.proxy.ProxyThread.processHttp(ProxyThread.java:497) [zap-2.10.0.jar:2.10.0]
    at org.parosproxy.paros.core.proxy.ProxyThread.run(ProxyThread.java:333) [zap-2.10.0.jar:2.10.0]
    at java.lang.Thread.run(Thread.java:834) [?:?]

3. ICTU ZAP Docker Full Scan

ICTU, a Dutch IT organisation of the government has extended the ZAP Docker images with a webhook for authentication. It would be interesting to find out whether this way you can scan the application including authentication. Notice that the Docker image is now taken from the ICTU DockerHub page. Two extra options are added compared to the full scan without user authentication:

• --hook: the link to the Python script which will take care of the authentication
• -z: some extra parameters needed for the authentication

$ docker run --rm -v $(pwd):/zap/wrk/:rw --net zapnet -t ictu/zap2docker-weekly zap-full-scan.py -I -j -m 10 -T 60 \
  -t http://172.22.0.2:8080/WebGoat \
  --hook=/zap/auth_hook.py \
  -r 20210417-ictuzap-full-scan.html \
  -z "auth.loginurl=http://172.22.0.2:8080/WebGoat/login \
      auth.username="mydeveloperplanet" \
      auth.password="password" \
      auth.auto=1"

This seems to do its work. However, less results are found compared to the ZAP CLI scan. Most likely due to the spider again.

4. ZAP CLI With Docker

The good news is that ZAP CLI is also shipped in the ZAP Docker image. Good results were achieved with ZAP CLI, so let’s see whether this also applies when you run it from within the ZAP Docker container. You run the Docker container again with a volume mapping to your current directory and with option -i in order to start the container in interactive mode. This will allow you to execute commands inside the Docker container.

$ docker run --name zap-cli --net zapnet -v $(pwd):/zap/wrk/:rw -i owasp/zap2docker-stable

As a test, you can verify whether WebGoat is accessible from within the ZAP Docker container with a wget.

$ wget http://172.22.0.2:8080/WebGoat
--2021-04-18 12:02:03--  http://172.22.0.2:8080/WebGoat
Connecting to 172.22.0.2:8080... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://172.22.0.2:8080/WebGoat/ [following]
--2021-04-18 12:02:03--  http://172.22.0.2:8080/WebGoat/
Reusing existing connection to 172.22.0.2:8080.
HTTP request sent, awaiting response... 302 Found
Location: http://172.22.0.2:8080/WebGoat/login [following]
--2021-04-18 12:02:03--  http://172.22.0.2:8080/WebGoat/login
Reusing existing connection to 172.22.0.2:8080.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘WebGoat’

     0K .                                                      94.3M=0s

2021-04-18 12:02:03 (94.3 MB/s) - ‘WebGoat’ saved [1877]

You will follow the exact same steps as in the previous post. The only difference is that you will execute the commands from within the Docker container. First thing to do is to start ZAP. For simplicity, you will disable the API key. Remember that the API key was necessary to access the ZAP API. You can retrieve the API key if you want via the webswing ZAP UI.

$ zap-cli --log-path wrk/ start --start-options '-config api.disablekey=true'
[INFO]            Starting ZAP daemon

Import the context. Remember that you changed localhost in the context file to the IP address where WebGoat can be accessed.

$ zap-cli -v context import /zap/wrk/Webgoat.context
[INFO]            Imported context from /zap/wrk/Webgoat.context

In the previous post, you exported the manually explored URLs in a file webgoat-exported-urls.txt. Copy this file to your current directory and find/replace localhost with the WebGoat IP address.

Also, copy the open-urls.sh script to your current directory and change the path to the text file.

#!/bin/bash
input="wrk/webgoat-exported-urls.txt"
while IFS= read -r line
do
  zap-cli open-url "$line"
done < "$input"

Execute the script, this will take approximately 10 minutes.

$ ./wrk/open-urls.sh
[INFO]            Accessing URL http://172.22.0.2:8080
[INFO]            Accessing URL http://172.22.0.2:8080/WebGoat
[INFO]            Accessing URL http://172.22.0.2:8080/WebGoat/
[INFO]            Accessing URL http://172.22.0.2:8080/WebGoat/AuthBypass.lesson.lesson
...
[INFO]            Accessing URL http://172.22.0.2:8080/sitemap.xml

Start the classic spider.

$ zap-cli -v spider -c Webgoat -u mydeveloperplanet "http://172.22.0.2:8080/WebGoat"
[INFO]            Running spider...
[DEBUG]           Spidering target http://172.22.0.2:8080/WebGoat...
[DEBUG]           Running spider in context 1 as user 2230
[DEBUG]           Started spider with ID 0...
[DEBUG]           Spider progress %: 0
[DEBUG]           Spider #0 completed

Start the active scan, this will take approximately 15 minutes.

$ zap-cli -v active-scan --recursive -c Webgoat -u mydeveloperplanet "http://172.22.0.2:8080/WebGoat"
[INFO]            Running an active scan...
[DEBUG]           Scanning target http://172.22.0.2:8080/WebGoat...
[DEBUG]           Scanning in context 1 as user 2230
[DEBUG]           Started scan with ID 0...
...
[DEBUG]           Scan progress %: 98
[DEBUG]           Scan #0 completed

Generate the report.

$ zap-cli -v report -o /zap/wrk/report-zap-full-scan.html -f html
[DEBUG]           Generating HTML report
[INFO]            Report saved to "/zap/wrk/report-zap-full-scan.html"

As you can see, this gives you similar results as in the previous post.

Save the session for next use.

$ zap-cli -v session save /zap/wrk/webgoat-20210418-active-scan.session
[DEBUG]           Saving the session to "/zap/wrk/webgoat-20210418-active-scan.session"

Shutdown ZAP.

$ zap-cli -v shutdown
[INFO]            Shutting down ZAP daemon
[DEBUG]           Shutting down ZAP.
[DEBUG]           ZAP shutdown successfully.

Finally, type exit to exit the interactive shell and shutdown the Webgoat Docker container.

$ docker stop goatandwolf

5. Conclusion

It is great that OWASP provides Docker images with ZAP pre-installed. This simplifies installation and makes it easier to integrate it into your CI/CD pipeline. The default scans which are provided did not give good enough results. Luckily, ZAP CLI is also provided and this did the job. Also note that ZAP CLI will be replaced in the near future with the Automation Framework.