DEV Community

Cover image for Automated Pen Testing With ZAP CLI
mydeveloperplanet
mydeveloperplanet

Posted on • Originally published at mydeveloperplanet.com

Automated Pen Testing With ZAP CLI

#testing #security #devops #tutorial

In the previous post, you learnt how to execute an automated penetration test by means of Zed Attack Proxy (ZAP). This time, you will learn how to execute the test via a Command Line Interface (CLI) which will make it possible to add the test to your CI/CD pipeline.

1. Introduction

In the previous post, the different steps were explained how to execute an automated penetration test. The application under test being used was WebGoat, a vulnerable application developed by OWASP in order to learn security vulnerabilities. This application will be used in this post also. The steps to be executed for a penetration test with ZAP are:

  • Setup the context and session (especially login credentials when a part of the application is behind a login);
  • Manually explore all parts of the application;
  • Spider the application;
  • Execute the automated scan;
  • Inspect the results.

Beware that an automated scan will not find all vulnerabilities! It is always necessary to execute a manual penetration test. The automated scan will however give you a good indication about the state of security of your application.

The files being used in this post are available at GitHub in directory zap-cli.

2. ZAP CLI

ZAP contains an API for controlling ZAP. The ZAP CLI tool is a tool which wraps the API in order that commands can be executed via the command line. In this section, you basically will perform the same or similar actions as in the previous post, except that you will not use the ZAP Desktop this time. A complete list of the commands of ZAP CLI can be found at the GitHub website.

2.1 Installation and Preparation

As a prerequisite, you must have ZAP already installed of course. Next to ZAP, you need to install ZAP CLI, which is quite simple. Just execute the following command:

$ pip install --upgrade zapcli
Enter fullscreen mode Exit fullscreen mode

Before you continue, it is very important to set the following environment variables:

export ZAP_PORT=8090
export ZAP_PATH=/usr/local/bin/zap.sh
export ZAP_API_KEY=<your API key>
Enter fullscreen mode Exit fullscreen mode

The ZAP_API_KEY can be found in ZAP Desktop. Therefore, start ZAP Desktop and choose Tools – Options… in the menu. In the API section, the API key is shown and needs to be used for the environment variable (but do not yet set the environment variable until it is mentioned to do so in the next section). You also have the possibility to disable the usage of the key, but this is not advised.

Alt Text

First, close the ZAP Desktop tool before you continue. Let’s see what happens when you try to start ZAP:

$ zap-cli start
[INFO]            Starting ZAP daemon
Traceback (most recent call last):
  File "/home/<user>/.local/bin/zap-cli", line 8, in <module>
    sys.exit(cli())
  File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/<user>/.local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/<user>/.local/lib/python3.8/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/home/<user>/.local/lib/python3.8/site-packages/zapcli/cli.py", line 58, in start_zap_daemon
    zap_helper.start(options=start_options)
  File "/home/<user>/.local/lib/python3.8/site-packages/zapcli/zap_helper.py", line 88, in start
    with open(log_path, 'w+') as log_file:
PermissionError: [Errno 13] Permission denied: '/usr/local/bin/zap.log'
Enter fullscreen mode Exit fullscreen mode

It seems that you do not have sufficient rights to write to the zap.log. There are several solutions for this, but the easiest one is to set the log path to a directory where you have sufficient permissions. Navigate to this directory and execute the following command:

$ zap-cli --log-path . start
[INFO]            Starting ZAP daemon
Enter fullscreen mode Exit fullscreen mode

This seems to have started the ZAP daemon. Let’s verify this:

$ zap-cli status
[INFO]            ZAP is running
Enter fullscreen mode Exit fullscreen mode

Allright, ZAP is running. Let’s try to shutdown the ZAP daemon:

$ zap-cli shutdown
[INFO]            Shutting down ZAP daemon
...
    raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8090): Max retries exceeded with url: http://zap/JSON/core/action/shutdown/?apikey= (Caused by ProxyError('Cannot connect to proxy.', RemoteDisconnected('Remote end closed connection without response')))
Enter fullscreen mode Exit fullscreen mode

Why is this not working? The reason for the exception is quite vague because it mentions ‘Cannot connect to proxy’ and ‘Remote end closed connection without response’. But why? The answer is shown in the preceding URL http://zap/JSON/core/action/shutdown/?apikey=. As you can see, the parameter apikey is empty. There are numerous questions on the internet about this exception, but very few will give you the answer. The solution is to set the ZAP_API_KEY environment variable.

$ export ZAP_API_KEY=<your API key>
$ zap-cli . shutdown
[INFO]            Shutting down ZAP daemon
Enter fullscreen mode Exit fullscreen mode

And now it is possible to shut down the ZAP daemon. It appears that you do not need the API key for every API call, but when you set the environment variables, you will not need to worry about this.

2.2 Context Preparation

Remember from the previous post, that it was important to set user credentials in the Context and the manual exploration of the website. In this post, you will reuse this information in order to feed it to ZAP CLI.

Open the previously stored session via File – Open Session… and export the context via File – Export Context… Select the context, the path where you want to save it and click the Save button.

Alt Text

The manually explored URLs of the application are also present in the Session. You will need those URLs for ZAP CLI. Right-click in the Sites section on http://localhost:8080/ and choose Export Selected URLs to File… and name the file webgoat-exported-urls.txt. You will need this file in the next paragraph.

Close ZAP Desktop.

2.3 Scan With ZAP CLI

At this moment, you have done all the necessary preparation in order to get started to scan your application with ZAP CLI. Identical steps as during the ZAP Desktop scan will be performed, but this time via the ZAP CLI and you will make use of the Context and the exported URLs which have been created in the previous post.

Before you continue, ensure that the WebGoat application is running. Otherwise, it is a good moment to start the application now.

$ docker start goatandwolf
Enter fullscreen mode Exit fullscreen mode

Start the ZAP daemon.

$ zap-cli --log-path . start
[INFO]            Starting ZAP daemon
Enter fullscreen mode Exit fullscreen mode

Import the context. Ensure to include the complete path to the context file.

$ zap-cli -v context import /<path>/Webgoat.context 
[INFO]            Imported context from /<path>/Webgoat.context
Enter fullscreen mode Exit fullscreen mode

When you list the contexts, you will notice that the list seems to be empty, but it isn’t. It is probably a bug that the name is not correctly displayed.

$ zap-cli context list
[INFO]            Available contexts: []
Enter fullscreen mode Exit fullscreen mode

Next step is to explore the website, you will use the exported URLs file you created in the previous paragraph. What you will do, is to open each URL which is present in this file with the zap-cli open-url command. Create a bash script open-urls.sh for this with the following contents.

#!/bin/bash
input="webgoat-exported-urls.txt"
while IFS= read -r line
do
  zap-cli open-url "$line"
done < "$input"
Enter fullscreen mode Exit fullscreen mode

Give the bash script executable permissions.

$ chmod +x open-urls.sh
Enter fullscreen mode Exit fullscreen mode

Execute the bash script, this can take some time dependent on how many URLs you have in the text file.

$ ./open-urls.sh
[INFO]            Accessing URL http://localhost:8080
[INFO]            Accessing URL http://localhost:8080/WebGoat
...
Enter fullscreen mode Exit fullscreen mode

Run the spider. Notice that the context is provided and also the user mydeveloperplanet which will allow you to login to the application.

$ zap-cli -v spider -c Webgoat -u mydeveloperplanet "http://localhost:8080/WebGoat"
[INFO]            Running spider...
[DEBUG]           Spidering target http://localhost:8080/WebGoat...
[DEBUG]           Running spider in context 1 as user 2230
[DEBUG]           Started spider with ID 0...
[DEBUG]           Spider progress %: 0
[DEBUG]           Spider #0 completed
Enter fullscreen mode Exit fullscreen mode

Do not run the AJAX spider. It is good to note that with the argument --help, you can view the command arguments available for this command.

$ zap-cli ajax-spider --help
Usage: zap-cli ajax-spider [OPTIONS] URL

  Run the AJAX Spider against a URL.

Options:
  --help  Show this message and exit.
Enter fullscreen mode Exit fullscreen mode

As you can see, only the URL is available as a parameter. This is strange because in the Desktop version it is possible to provide a context and a user. It also seems that it negatively influences the end results (less alerts were reported). Therefore, do not run the AJAX spider.

Start the active scan. This will take some time (about 10 to 15 minutes).

$ zap-cli -v active-scan --recursive -c Webgoat -u mydeveloperplanet "http://localhost:8080/WebGoat"
[INFO]            Running an active scan...
[DEBUG]           Scanning target http://localhost:8080/WebGoat...
[DEBUG]           Scanning in context 1 as user 2230
[DEBUG]           Started scan with ID 0...
[DEBUG]           Scan progress %: 0
[DEBUG]           Scan progress %: 1
[DEBUG]           Scan progress %: 3
...
[DEBUG]           Scan progress %: 97
[DEBUG]           Scan progress %: 99
[DEBUG]           Scan #0 completed
Enter fullscreen mode Exit fullscreen mode

Generate the report.

$ zap-cli -v report -o /<path>/report-zap-cli-first-scan.html -f html
[DEBUG]           Generating HTML report
[INFO]            Report saved to "/<path>/report-zap-cli-first-scan.html"
Enter fullscreen mode Exit fullscreen mode

Save the session. This will allow you to open it in ZAP Desktop for example.

$ zap-cli -v session save /<path>/webgoat-20210410-active-scan.session
[DEBUG]           Saving the session to "/<path>/webgoat-20210410-active-scan.session"
Enter fullscreen mode Exit fullscreen mode

And shutdown the ZAP daemon.

$ zap-cli -v shutdown
[INFO]            Shutting down ZAP daemon
[DEBUG]           Shutting down ZAP.
[DEBUG]           ZAP shutdown successfully.
Enter fullscreen mode Exit fullscreen mode

3. Conclusion

ZAP CLI gives you the opportunity to execute an automated penetration test via the command line. This will make it also possible to include it into your CI/CD pipeline. Beware that it will give you only an indication about the security of your application and it does not replace a complete penetration test. Also note that the results are not identical to the results from the previous post. But when you execute the same steps as in this post starting from a clean session in ZAP Desktop, the results will be quite similar. Reason for the difference is probably the execution of the AJAX spider or the manually exploration of the website from within ZAP Desktop.

Oldest comments (0)

Read next

janoskocs profile image

Setting up a React project using Vite + TypeScript + Vitest

János K. -

jerezm profile image

A Modular Approach to Integrating React-Router v6

Martin Jerez -

lukeecart profile image

Planet scale DB alternative

Luke Cartwright -

ehienabs profile image

Deploying Amazon Managed Service for Apache Kafka (Amazon MSK) with CloudFormation

Ehi Enabs -