DEV Community: Jit - Minimum Viable Security for Developers

Playing Around with AWS-Vault for Fun & Profit

Ohav Almog — Mon, 12 Jun 2023 19:49:52 +0000

Introduction

AWS-Vault is an excellent open-source tool by 99Designs that enables developers to store AWS credentials in their machine keystore securely. After using it for a while at Jit, I decided to dig deeper into how it works and learned a lot along the way. In this article, I will summarize and simplify the information I learned to help others with their AWS-Vault adoption and lower the barrier to usage.

I will start with a basic explanation of AWS access keys, the Session Token Service, and its helpful API calls. Then I will show how aws-vault uses it to provide a secure way to access AWS resources, reducing the risk of exposing AWS credentials. In addition, I will represent a typical AWS account pattern, and again we'll see how aws-vault works perfectly with it.

AWS Access Keys

AWS makes it possible to create a set of access keys for a specific user (those are called aws_access_key_id and aws_secret_access_key). Together with these keys, it's possible to authenticate to AWS and perform actions on behalf of the user.

When using Python, for example, it's possible to use the boto3 library, which is the AWS Software Development Kit for Python, to authenticate to AWS:

import boto3

session = boto3.Session(
    region_name='us-east-1',
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
)
session.client('s3').list_buckets()

With the AWS CLI, a configuration file can be used to store the access keys, which can then be used in the future to authenticate to AWS as follows:

> cat ~/.aws/config

[default]
region=us-east-1
aws_access_key_id=***
aws_secret_access_key=***

> aws s3 ls

With such a configuration file, long-lived credentials are stored in plain text on the user's machine. That's a security risk, as anyone with access to the device or the file can steal the credentials to perform actions on behalf of the user.

That is why developers and teams have moved on to more secure authentication practices, such as using session tokens.

Session Tokens

Sessions are a more secure approach, allowing the user to create valid temporary credentials for a specific period. Thus, if stolen, the keys might already be expired or will only limit the time the attacker can leverage them.

Below is a code example for how to generate temporary credentials using the AWS CLI:

> aws sts get-session-token
{
    "Credentials": {
        "AccessKeyId": "***",
        "SecretAccessKey": "***",
        "SessionToken": "***",
        ...
        }
}

But hang on a minute, as this example uses the long-lived credentials to authenticate to STS and get the temporary credentials. So, how could this possibly solve the problem of having unencrypted long-lived credentials on a user's machine?

That is where aws-vault comes in. But just before diving into aws-vault, let's first look at popular STS API calls.

AWS STS

STS, which stands for AWS Security Token Service, is a web service that enables you to request temporary, limited-privilege credentials for AWS IAM users or IAM roles. All temporary credentials consist of an access key ID, a secret access key, and a session token. These three credentials are then used to sign requests to any AWS service.

GetSessionToken

GetSessionToken is an API call that fetches temporary credentials for an IAM user. The simplest example is when a user uses its long-lived credentials to request temporary credentials (for any reason), where a widespread use case for this API call is when a user wants to authenticate to AWS using MFA. If MFA is used, the received credentials can be used to access APIs requiring MFA authentication.

AssumeRole

Another method used to get temporary credentials, this time for an IAM role, is AssumeRole. Like GetSessionToken, it returns an access key, a secret access key, and a session token. It's possible to authenticate with an MFA device when assuming a role, and moreover, it's possible to protect an IAM role, requiring it to be assumed with MFA.

GetSessionToken and AssumeRole Comparison

	GetSessionToken	AssumeRole
Used to receive credentials for	IAM User	IAM Role
Can be called with only long-lived credentials	YES	NO
Can be called with MFA	YES	YES
The received credentials can be used to issue a call to `GetSessionToken`	NO	NO
The received credentials can be used to issue a call to `AssumeRole`	YES	YES
Maximum session duration	36 hours	12 hours
Cross-account access	NO	YES

AWS Vault

Now let's see where AWS Vault comes in to add a layer of security, preventing the storage of long-lived credentials in regular system files.

Copied from the project’s README:

“AWS Vault stores IAM credentials in your operating system's secure keystore and then generates temporary credentials from those to expose to your shell and applications.”

What this means for the end user is that instead of storing the credentials as clear text in a configuration file, they are stored in the OS's secure keystore. That is a safer approach, as the credentials are encrypted and can only be decrypted by the user who created them.

Not only does AWS-Vault store the credentials in the OS's secure keystore, but it also generates temporary credentials from those to expose to the shell and applications. That means that even if the credentials are stolen from the environment or an application, they are only valid for a specific period.

How Does It Work?

First, aws-vault stores the long-lived credentials in the OS's secure keystore. To do it, the aws-vault add command should be used:

# A configured profile can already exist or not. If not, aws-vault will create the profile in `~/.aws/config`.

> aws-vault add some-developer
Enter Access Key Id: ***
Enter Secret Access Key: ***

For instance, in macOS, the credentials are stored encrypted in the Keychain Access app and can be viewed under 'Custom Keychains'.

Then, when requesting to use that profile, aws-vault will generate temporary credentials from the long-lived credentials using GetSessionToken. These temporary credentials are then exposed to the shell and applications using environment variables.

> aws-vault exec some-developer -- env | grep AWS
AWS_ACCESS_KEY_ID=***
AWS_SECRET_ACCESS_KEY=***
AWS_SESSION_TOKEN=***

AWS-Vault also stores the session credentials in the same keychain to be used until they expire.

Popular AWS Account Pattern

A typical user management pattern and recommended security practice is having a single AWS management account with only IAM users grouped into IAM groups (like developers and admins). Then, in addition to that account, there can be multiple AWS accounts (tagged dev, staging, and prod) with IAM roles that are assumed by the users in the developers and admins groups.

This pattern has many advantages, and the biggest of them all is resource isolation. Having each environment resource in a different AWS account makes it very unlikely that resources from a dev account will interact, interfere, or be included in the same list of resources in the production environment. In addition, in terms of security, this separation makes it harder to access prod resources or data (which are usually the most valuable) from a development or a staging account.

Not only that but separating accounts also helps by viewing and monitoring cost and usage. With that, setting specific alerts and budgets for each environment is very easy.

A developer in such an organization will probably have an AWS configuration file that looks like this:

[default]
region = us-east-1
mfa_serial = arn:aws:iam::1000:mfa/some-developer

[profile some-developer]
Credential_process = aws-vault export --format=json some-developer

[profile dev]
role_arn = arn:aws:iam::2000:role/Admin
source_profile = some-developer

[profile staging]
role_arn = arn:aws:iam::3000:role/ReadOnly
source_profile = some-developer

[profile prod]
role_arn = arn:aws:iam::4000:role/ReadOnly
source_profile = some-developer

This is where the beauty of aws-vault, together with this pattern, comes in. At first glance, it's possible to think that AWS-Vault would ask for a session token for each environment (as it's in a different profile and AWS account). But, if you remember the previous section, GetSessionToken requests credentials for an IAM user, not for an IAM role. And in this case, the developer only has one user, which exists in the management account.

Thus, the developer will store only a single pair of long-lived keys of the management account in the OS’ secure keystore, as can be seen in the following output:

> aws-vault list

Profile                  Credentials              Sessions                      
=======                  ===========              ========                      
default                  -                        -                             
some-developer           some-developer           sts.GetSessionToken:8h29m31s  
dev                      -                        -                             
staging                  -                        -                             
prod                     -                        -

AWS-Vault identifies the source_profile and role_arn attributes in the config file and understands it needs to assume each environment role with the same session from the management account's developer profile.

By leveraging the source_profile property in the ~/.aws/config file, aws-vault will use the credentials from the source_profile to assume the role in the role_arn property. That allows aws-vault to ask for MFA only once and then use the temporary credentials to assume the roles in the other accounts.

Usually, these roles will have a condition requiring MFA to be assumed. So, why isn't the MFA being asked for each time the developer switches between environments? That's thanks to the fact that a session token that was created using MFA can be used to assume a role that requires MFA, and aws-vault is smart enough to know that both roles use the same MFA device.

With this pattern and configuration, developers can easily switch between different environments without a hassle:

> aws-vault exec dev -- aws lambda invoke --function-name some-function --payload '{}'
> aws-vault exec staging -- aws lambda list-functions

Wrap Up

AWS-Vault is a powerful open source security tool to help add a layer of much-needed essential security for AWS users and developers. It is also built on top of the classic AWS config file, helping to avoid extra configuration and pains of usage.

I hope the examples provided will help you ramp up aws-vault more quickly and unleash the security capabilities this tool offers AWS users.

AssumeRoleWithWebIdentity WHAT?! Solving the Github to AWS OIDC InvalidIdentityToken Failure Loop

Ariel beck — Wed, 18 Jan 2023 10:07:16 +0000

How many of us have encountered all kinds of CrashLoopBackoff or other random error messages, and start to go down the Stack Overflow rabbit hole, only to hit a wall?

‍In our case this occurred with the AssumeRoleWithWebIdentity that started throwing the InvalidIdentityToken error when running pipelines with an OIDC provider for AWS. We went through a whole process of researching and ultimately fixing the issue for good, and decided to give a quick runthrough in a single post of how you can do this too.

‍‍

Figure 1: Error received from Terraform that flagged the issue.
‍
So let’s take a step back and provide some context to the issue at hand, when you’re likely to encounter it, and ways to overcome it.

‍## Web Applications and AWS Authentication through the OIDC Plugin
When companies work with AWS with third-party resources, they need to create a “trust relationship” between AWS and the service to ensure the resource has the necessary permissions to access the AWS account. To do so, the OpenID Connect (OIDC) plugin, a simple identity layer on top of the OAuth 2.0 protocol, is a commonly chosen method for doing so.

‍For us, the service at hand was Github, where the OIDC authentication was configured in Github, to provide our Github repo the required trust relationship to access our AWS account with the specified permissions through temporary credentials, in order to create and deploy AWS resources through our CI/CD process. We basically had a primary environment variable configured ("AWS_WEB_IDENTITY_TOKEN_FILE"), which then tells various tools such as boto3, the AWS CLI or Terraform (based on the relevant pipeline) to perform the AssumeRoleWithWebIdentity and get the designated temporary credentials for the role to perform AWS operations.

Random Failures with Esoteric Error Messages

The AssumeRoleWithWebIdentity error manifests itself mostly around parallel access attempts, and how the various AWS interfaces are able to authenticate, as well as run and deploy services. We started encountering this issue when running our pipelines for deployment, and attempting to authenticate our Github account to AWS via the OIDC plugin. This is a well-known (and widely discussed) limitation for authentication to AWS for web application providers. In our case it was Github, but this is true for pretty much any web application integration.

‍
At first this error would randomly fail builds, with the InvalidIdentityToken error, which would only sometimes succeed on reruns. At first we ignored it, and just assumed it was the regular old run of the mill technology failures at scale. But then this started to happen more frequently, and added sufficient friction to our engineering velocity and delivery, that we had to uncover what was happening.

‍So what was happening under the hood?
‍
Our design consisted of a method to retrieve the token from AWS, where we then used the token in order to try to connect to AWS. Where this fails is with the threshold that AWS enables for multiple parallel authentication requests, along with the very low Github timeout for response. The tool chain we were using essentially tried to access this token simultaneously many times, and this caused the authentication access to fail upon throttling of the resources.

Fixing the InvalidIdentityToken error with the AssumeRoleWithWebIdentity Method

Overcoming this issue was a concerted effort, which began with even understanding what triggers this issue to begin with and how to get to the root cause, and then solving the issue ––particularly since when authenticating to AWS there is usually a canonical order, for example, first the environment variables, to ensure the environment runs as it should with access to the relevant keys and private information.

‍We found that the current method we were using of a temporary token from the provider for direct authentication access, was clunky. There is no way to use this method without creating a system of retries, which is never robust enough when it’s out of your control, and not all of our tools supported, either.

Therefore, we understood a more stable method was required, we would need a way to control the token calls for authentication. By doing so, we would be able to ensure retries until success, so that failures due to inherent limitations can be avoided or at the very least retried until successful.

First and foremost, let’s talk about our tool chain. Not all the tools we were using (boto3, Terraform), have the logic or capability for retries, and this where we were encountering these failures. With multiple pipelines running simultaneously, all using the temporary access token method for authentication, we quickly reached the threshold of authentication and access.

In order to enable parallel access we realized we were missing a critical step in the process to make this possible. The actual recommended order to make this possible would be to retry the AssumeRoleWithWebIdentity until success, and then set the environment variables upon successful access (based on the docs these include : AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY + AWS_SESSION_TOKEN). Another equally critical piece to make this all work was to provide a longer validity window for our token, in our case we set the expiration to 1 hour, and the last and most important part, by performing the retry ourselves.

Once we understood the process we needed to make parallel access work, we started to do some research, and happened upon a Github Action that basically does this entire process end-to-end for us, all the way through configuring the environment variables under the hood for the temporary credentials. The added bonus is that this Github Action prevents Terraform from assuming the role, and simply uses the credentials that were set (that are now valid for 1 hour). Another advantage of using this Github Action is that it also addressed a known exponential backoff bug that was fixed in January.

By setting these environment variables that are valid for up to an hour, this adds stability to the system, as the tokens are now AWS tokens and not unstable OIDC tokens, basically bypassing the need to assume a role directly for authentication vs. the provider.

‍Similar logic can basically be applied to any application that uses the OIDC plugin, that encounters such issues, even those that we couldn’t leverage the excellent gem of a Github Action for. We borrowed from this, and wrote our own code that replicates similar logic for non-Github applications to achieve similar stability. See the code example below:

‍

export AWS_ROLE_ARN=arn:aws:iam::1234567890:role/RoleToAssume
export AWS_WEB_IDENTITY_TOKEN_FILE=/tmp/awscreds
export AWS_DEFAULT_REGION=<region>
export DEFAULT_PARALLEL_JOBS=4

OUTPUT_TOKEN_REQUEST=$(curl -H "Authorization: bearer $ACTIONS_ID_TOKEN_REQUEST_TOKEN" "$ACTIONS_ID_TOKEN_REQUEST_URL")
echo "$OUTPUT_TOKEN_REQUEST"  | jq -r '.value' > /tmp/awscreds

RET=1
MAX_RETRIES=5
COUNTER=1
WAIT_FACTOR=2
RUN_ID=$(uuidgen)


until [ ${RET} -eq 0 ]; do

    # 5 retries are enough, then fail.
    if [ $COUNTER -gt $MAX_RETRIES ]; then
      echo "$RUN_ID - Maximum retries of $MAX_RETRIES reached. Returning error."
      exit 1
    fi

    # Try to perform the assume role with web identity
    OUTPUT_ASSUME_ROLE=$(aws sts assume-role-with-web-identity --duration-seconds 3600 --role-session-name my_role_name --role-arn $AWS_ROLE_ARN --web-identity-token $(cat /tmp/awscreds) --region $AWS_DEFAULT_REGION)
    RET=$?
    echo "$RUN_ID - attempt: $COUNTER, assume rule returned code: $RET"

    if [ $RET -ne 0 ]; then
      echo "$RUN_ID - attempt: $COUNTER - Error happened in assume role, error code -  $RET, error msg: $OUTPUT_ASSUME_ROLE. retrying..."

      WAIT_FACTOR=$((WAIT_FACTOR*COUNTER))
      sleep $WAIT_FACTOR
    else
      access_key_id="$(echo "$OUTPUT_ASSUME_ROLE" | jq -r '.Credentials.AccessKeyId')"
      # Set the AWS environment variables to be used.
      export AWS_ACCESS_KEY_ID=$access_key_id
      secret_access_key="$(echo "$OUTPUT_ASSUME_ROLE" | jq -r '.Credentials.SecretAccessKey')"
      export AWS_SECRET_ACCESS_KEY=$secret_access_key
      session_token="$(echo "$OUTPUT_ASSUME_ROLE" | jq -r '.Credentials.SessionToken')"
      export AWS_SESSION_TOKEN=$session_token
    fi
    COUNTER=$((COUNTER+1))
done

# Perform any calls to AWS now - the 3 environment variables will take precedence over AWS_WEB_IDENTITY_TOKEN_FILE

To wrap up, what we quickly understood is that even with a small feature change from single to multi-region, that suddenly required parallel access, a process that worked perfectly fine for sequential access, wasn’t the best fit when we started to grow. This naive design broke down at scale and required us to unpack the complex logic happening under the hood.

When we encountered this issue we were searching for a tutorial just like this one to help us resolve the issue quickly, so we hope you found this useful and can save some of the time and effort it took us to resolve the issue

NPM Audit: 5 Ways to Use it to Protect Your Code

Avichay Attlan — Wed, 04 Jan 2023 12:53:51 +0000

You might already know Node and the accompanying JS package manager - NPM. NPM is the most extensive package manager in the world, with over one million packages available. Since the packages and dependency trees are updated frequently, vulnerabilities from old package versions may find their way into your project.

‍If you have not touched your project in a while and find that you have far more vulnerabilities than expected, you’ll need a more comprehensive tool for dealing with your entire node module folder in one fell swoop. Node’s tool for the job is the NPM Audit. In this article, we’ll dive deeper into the various options in NPM Audit and how you can utilize them to protect your code.

NPM Advisory Database

The npm install and npm audit commands check for vulnerabilities against known security risks reported in the public npm registry. As of late 2021, this vulnerability database has been hosted on GitHub, called the GitHub Advisory Database. The same vulnerability database powers GitHub’s Dependabot tool alerting developers to known vulnerabilities in their code base hosted on GitHub.

The npm audit command now includes a URL with each proposed vulnerability fix linking to the GitHub Advisory Database’s specific vulnerability report. If you’re interested, GitHub also provides an API for browsing the Advisory Database for vulnerabilities based on severity or a particular package name. You can also offer suggestions for fixing vulnerabilities or editing a specific vulnerability description to clarify it.

Here’s an example of a vulnerability description on GitHub:

How to run NPM Audit

The basic command you need to run to get Node’s suggestion on fixing your vulnerabilities is npm audit. Firstly, ensure you have installed the latest Node and NPM versions and open your project.

You should navigate to the project’s folder where your package.json file is saved. Once you have your project open, open a new terminal where you can type any of the mentioned CLI commands and where you’ll see the results.

The npm audit command will give you a list of the vulnerabilities found and more information about them.

At the end of the report, you’ll see the number of vulnerabilities that NPM Audit can fix automatically and the vulnerabilities that require a manual review.

If the report is particularly long, you can use npm audit --json to get the report in a JSON format. To pipe the report to a file, use the > (pipe) sign along with the path and filename you wish to generate: npm audit --json > report.json.

You can also use a package called npm-audit-html to generate the same report in an HTML format (npm audit --json | npm-audit-html --output report.html). JSON and HTML formats are more straightforward to view than a simple data dump to the terminal.

‍Jit orchestrates security for Node.js stacks. Your developers can quickly and seamlessly integrate npm-audit into their code security layer to help run dependency checks within a centralized CI workflow. From Code, pipeline, and infrastructure to runtime, Jit provides security-plan-as-code (SaC) and orchestrates all security tools at every stage of the software development lifecycle.

5 Ways to Use NPM Audit to Protect Your Code

1. Generate security audit reports frequently

Since the NPM package ecosystem is updated often, you may already have vulnerabilities in your code due to old package versions you may not even be aware of. Most of us don’t run a full npm install frequently in the regular course of events.

You should run the npm audit report regularly, as this will help you ensure you don’t have hidden dependency vulnerabilities in your project. What you consider a 'regular basis' depends on your project's size and complexity.

About once a month may be enough for a large and complex project. If the project is stable and doesn't have a lot of updates, then even once a quarter may suffice. The frequency is up to you, but you can keep those reports as testimony that you are on top of the project’s vulnerability issue.

2. Review the report and security advisory

An NPM Audit report contains the following data in the following structure:

Severity - Description

Package (title) - relevant info

Patched In (title) - relevant info

Dependency Of (title) - relevant info

Path (title) - relevant info

More Info (title) - URL link to GitHub Advisory DB

For example:

Here’s a brief description of what each piece of information means:

Severity - The severity of the vulnerability based on its potential impact. The severity is divided into

Critical - Address immediately
High - Address as quickly as possible
Moderate - Address when time allows
Low - Address at your discretion

Description - A short description of what might happen if you don’t address the vulnerability. For example - ‘Vulnerable to DoS attack.’

Package - The name of the package where the vulnerability was found.

Patched in - Assuming the vulnerability was addressed in a later version of the package, this part will say which version contains the patched version. For example >=2.0.1 (in all versions starting at 2.0.1 and higher)

Dependency of - Which other package (or packages) uses this particular package? You might see a lot of packages in the report you have no recollection of ever installing or using. That’s because each of the packages you use comes with its dependencies, and sometimes those dependencies have dependencies of their own in a very long chain called the software supply chain. Telling you which package is using this problematic package is vital since removing the original package eliminates the dependency problem without updating anything.

Path - The path to the package folder in the node modules containing the vulnerability.

More info - A link to the security report in the GitHub Advisory Database.

3. Verify the registry signatures of downloaded packages

Packages published to the public npm registry are signed to make it possible to detect if the package content has been tampered with. The signing happens automatically once the developer uploads the package to NPM.

Should a proxy server, a mirror, or a similar attack affect the users of a particular package, the signature found on the local package will not match the expected signature saved in the NPM registry for that package.

‍By adding the tag Signatures to the npm audit command (npm audit signatures), you’ll get a report explicitly checking each of your packages’ signatures. This tag only works starting at npm v8.15.0, so make sure you have the latest NPM version if you want to use this option. Note that the output might vary from version to version.

Here’s an example of the final report you might receive:

For each package, you’ll receive a list of its keyid that has to match one of the public signing keys and the actual signature based on that key. For example:

To check if the keyid matches one of the public keys, you can go to registry-host.tld/-/npm/v1/keys and compare the keys provided there. Make sure you compare based on the same key format. In this case, for example, you’ll need to check the SHA256 key.

Pay close attention to any packages with problematic signatures - it might indicate that your version has been tampered with.

‍Since there could be thousands of packages, you can pipe the results to a JSON file and only check packages where there is an indication that the signatures don't match. If you’re worried, an easy fix would be to re-install a problematic package and check again. You can contact NPM directly and report the problem if you still get the same issue.

4. Check for meta-vulnerabilities and remediations

A "meta-vulnerability" is a vulnerable dependency since the package depends on a vulnerable version of a different package. So, package ‘A’ might not have any vulnerability in itself. Still, it will be displayed as containing a vulnerability since it depends on package ‘B,’, which has a known vulnerability.

‍Once meta-vulnerabilities for a given package are calculated, they are cached in the ~/.npm folder and only re-evaluated if the advisory range changes or a new version of the package is published (in which case, the latest version is checked for meta-vulnerable status as well).

Suppose the chain of meta-vulnerabilities extends to the root project, and it’s impossible to update without changing its dependency ranges. In that case, the npm audit fix will require the --force option to apply the remediation.

Remediations may not require changes to the dependency ranges. In this case, all vulnerable packages will be updated to a version that does not have an advisory or meta-vulnerability posted against it without any need for developer interference.

5. Fixing vulnerabilities automatically

Assuming you have more than one or two vulnerabilities, this report can quickly get tedious. If you trust Node’s suggestions, you can run an npm audit fix and fix whatever can be fixed automatically. Since the npm audit fix is essentially running npm install under the hood, be prepared to wait a short while for it to be completed.

Once the fix command has concluded, you’ll get a summary of the completed changes:

This report will tell you if there are any potential breaking changes you should review. You can also rerun the command with --force to make the update and deal with the broken code later.

6. Fixing vulnerabilities manually

Based on the audit report you have received, you have two options for dealing with changes that cannot be fixed automatically with an npm audit:

Forcing the audit fix to make the necessary changes, even if they might break your code.
Go over each suggested fix and problem and decide how to deal with those yourself.
In the example we showed earlier, there are potentially 18 packages that require manual review and two packages that involve breaking changes.

Once you run the npm audit fix command, rerunning the report will give you only the problematic packages. Here, you need to look at the suggested remediation (assuming there is one) and see what else might be affected if you apply that fix.

In some of these cases, it might be easier to replace a package with a different one with similar usability rather than tracking a vulnerability down the dependency tree. That’s assuming the vulnerability is tagged as high or critical severity.

Starting Auditing Now

None of us want to face disgruntled users after our entire project fell apart due to a problem that wasn’t fixed on time. That’s why you need to audit your dependency vulnerabilities often.

Since most projects involve a team of developers, it’s much more efficient to audit your code after each PR. This is where CI/CD integrated tools for dependency audit come in handy.

How to Automate OWASP ZAP

Simon Bennetts — Wed, 14 Sep 2022 14:52:47 +0000

Introducing ZAP

OWASP ZAP is the world’s most popular web app scanner that now sees over 4 Million “Check for Updates” calls per month (up from 1 million just earlier this year).

It is free, open source, and used by people with a wide range of security experience, ranging from newcomers right up to experienced security professionals to get a better understanding of web application security posture. The way OWASP ZAP works is by attacking your web apps in a similar way to a malicious hacker, where it attacks your apps when they are running and shows you what attackers will be able to find when they attack your app.

The ZAP Desktop

ZAP was built for flexibility and adoption, and therefore it is possible to run it in a diversity of ways aligned with how you like to work:

From the command line
As a desktop application
As a background daemon

In this blog post I’ll show you how to automate ZAP from the command line, and to set it up by using the ZAP desktop application. The ZAP desktop app allows you to see exactly what ZAP does, and to tune ZAP to handle your app as effectively as possible.

ZAP requires Java 8+ to run locally, but you can also run ZAP in Docker and access it via a browser using Webswing. We won’t dive into this setup here but you are welcome to reference the docs to get started.

Just note, if you use the docker option then you will need to make sure you start docker with the option to map a local drive so that you can access the file you are going to generate after you stop the docker image. On *nix systems the docker option is -v $(pwd):/zap/wrk/:rw - on Windows you will need to replace $(pwd) with the full path of a suitable directory.

The Automation Framework

The Automation Framework (AF) allows you to control ZAP with one yaml file. There are other ways to automate ZAP, but the AF is the recommended approach for most users.
You can create the yaml file in a text editor but it is also possible to create it using the ZAP desktop, as that allows you to test the plan as you go, that is the option we will use here.

Exploring your App Using the Spiders

The first thing to do is to explore your app.
Apps designed for humans are typically best explored by humans, but that’s not a good option for automation and eventually scale.
However we will start by doing a bit of manual exploration just to make sure we can connect to the app.

In the ZAP desktop click on the “Manual Explore” button.
In the next form fill in the URL of your target app, uncheck the “Enable HUD” box and click on “Launch Browser”.

A browser should be launched and display your target app - for the sake of this example we will be using Firefox, but Chrome should work as well, if that is the browser you have installed.

Now look at the Sites tree - you should see at least one URL from your target app, probably many more. In this case I’m using OWASP Juice Shop as my target.

Now we can see how well each of the 2 spiders handles our app.

Right click on the top node of your app in the Sites tree and select “Attack -> Spider…”

Then click “Start Scan"

This spider should run very quickly, and it will tell you how many URLs are found and how many were added to the Sites tree:

If your target app is a more traditional app, then this should be sufficient. However, if it is a more modern web app which makes heavy use of JavaScript then you will need to use the AJAX spider.

Right click on the top node of your app in the Sites tree and select “Attack -> AJAX Spider…”

Then click “Start Scan"

This spider will take longer as it launches browsers in order to click on UI elements. It will also tell you how many URLs it found:

Exploring your App by Importing API Definitions

If your app just defines an API then you will not be able to explore it either manually or using either of the ZAP spiders.

If you have an API definition then you can import that via the “Import” menu item. ZAP supports importing the following definitions:

Open API
GraphQL
WSDL
HAR

Defining a Context

You do not need to define a context if you are just using ZAP manually, but you do need to define one when using the Automation Framework. Luckily this is easy to do - just right click the top node of your app in the Sites tree and select “Include in Context -> New Context“

and optionally give it a meaningful name:

Creating a Plan in the Desktop

We’re now ready to create the plan.

First, you need to find the “Automation” tab. ZAP has lots of tabs so all but the most essential ones are hidden by default. Click on the green plus tab in the bottom panel and select “Automation”:

In the new Automation panel click the “New Plan…” button:

Select the context you defined above and then one of the following Profiles:

Baseline - if you just want to passively scan your app and not attack it
GraphQL - if you have a GraphQL API definition
OpenAPI - if you have an OpenAPI / Swagger API definition
SOAP - if you have a SOAP definition
Full Scan - if you want to attack your app (and do not have an API definition)

A summary of your plan will now be displayed:

A plan can be much more flexible than these profiles, but they are the best option to use when you are getting started.

The Baseline and Full scans will include both the spider and the spiderAjax - you can remove one of them if you find it is not necessary. Do not remove both as then the plan will not explore your app at all!

If you want to import an API definition in addition to spidering then add the relevant job.

You can edit a plan in the ZAP desktop:

Double clicking on any job will bring up a dialog which will allow you to configure it
The “Add job…” button will allow you to add a job to the existing plan
The “Remove Job…” button will remove the selected job
The “Move Job Up” button will move the selected job up one place
The “Move Job Down” button will move the selected job down one place

Passive Scanning

ZAP will passively scan every request initiated by ZAP or proxied through it.
The profiles will all add two related jobs:

passiveScan-config - this allows you to fine tune the passive scanner
passiveScan-wait - the waits until all of the requests have been passively scanned, it should always be run after any jobs which explore your app

Active Scanning

The activeScan job runs the active scanner - this performs the actual attacks.

You should always include this job unless you only want to passively scan your app.

Double clicking on the job will allow you to fine tune the active scan rules.

Generating a Report

The report job not surprisingly has the task of generating a report - there are a variety of templates with different options for you to choose from. See https://www.zaproxy.org/docs/desktop/addons/report-generation/templates/ for the latest list with examples of each.

Authentication

Authentication is hard, and definitely out of scope for this blog post (but we will do our best to write a deep dive on this in the future). However, ZAP can handle pretty much any authentication mechanisms - more details on the ZAP website. And the good news is that you can test authentication handling in the ZAP desktop where you can see exactly what is going on, and when you create a plan using that context all of the authentication configuration will be imported into the plan.

Testing Your Plan Locally

You can run the authentication plan you have created in ZAP using the “Run Plan…” button.
You will then see the status of the jobs as the plan runs:

If any of the jobs fail then you can investigate them in the ZAP desktop.

However jobs may appear to succeed while still not doing what you expect.
Thats is why the AF supports job outcome tests - these tests can be added to any job and can do things like:

Check any of the ZAP statistics
Check is specific alerts are present of absent
Check if specific URLs have been found and optionally check their content

Statistics tests are added by default to the spider jobs, but you can add more tests to any of the jobs using the “Add Test…” button (there’s also a “Remove Test…” button) and don’t forget that you can double click on any of the tests in the plan in order to edit them.

Running in CI/CD

You can run AF tests from the command line using a command like:

zap.sh -cmd -autorun plan.yaml

You can also run them in docker. If you are using the stable image then we recommend updating ZAP using a separate command:

docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-stable bash -c "zap.sh -cmd -addonupdate; zap.sh -cmd -autorun /zap/wrk/plan.yaml"

The $(pwd):/zap/wrk/:rw part of the command maps your local CWD to the directory /zap/wrk in the docker container.

If you use any scripts in your plan then you will need to make sure they are in or below your local CWD and change your plan so that they are referenced via the location they appear in the docker container (i.e. under /zap/wrk).

OWASP ZAP + Jit Integration

While ZAP is an extremely powerful tool, it is very much a “point solution” and does not provide any scan scheduling, history or an online interface. Jit is a DevSecOps orchestration platform which integrates ZAP and other security tools. By using Jit you can have your ZAP findings available in an aggregated dashboard with the rest of your security tooling, and receive greater context for the overall security posture of your application.

What is Minimum Viable Security (MVS) and how does it improve the life of developers?

David Melamed — Tue, 05 Jul 2022 13:33:27 +0000

Last year, Google shook up the cybersecurity and software development community by launching the Minimum Viable Security Product (MSVP).

Developed in collaboration with Salesforce, Slack, Okta and others, MSVP's goal is to create baseline security standardization for third-party software developers, ensuring companies in the supply chain can rely on a minimum level of security practices and standards when building their products.

For fast-paced startups building software products in the B2B or B2C or even the B2D industry (like ourselves at Jit), MSVP is great, but it represents a significant development that needs consideration.

What even is the concept of Minimum Viable Security EXACTLY? And how can developers successfully learn how to follow and comply with these new high level requirements?

It’s no news that with the need to deliver software products quickly and continuously, the tech world has seen a shift in operations towards DevOps, DevSecOps and ‘Shift Left Everything’ approaches. These practices were created to support short, iterative and continuous cycles, and also to avoid running quality and security tests as an afterthought and delay the release.

But the reality isn’t running as smoothly as the theory.

Due to multiple issues, many professionals in the industry are starting to think about taking Shift Left practices a step further through 'Born Left.' This happens today mainly in software testing that is entirely owned by the engineering team as a native function, rather than by a siloed QA or Ops teams.

“Born Left” means that the engineering organization takes full ownership of the testing as part of the processes, known as Continuous Integration (CI), and operations through Continuous Deployment (CD).

But what about security?

The natural progression of this strategy puts security next in line, with Continuous Security (CS) becoming an emerging standard.

The problem with shift-left security (or anything else)

The problem with making developers responsible for more and more areas of the software cycle is the potential for overwhelming the team due to added tasks out of their domain expertise, frustrating them and causing them to be delayed with their main coding tasks. Quality, operations, security – the requirements quickly add up, and these domains often require expert knowledge, and this is particularly true when it comes to security. The cybersecurity landscape is constantly evolving with a range of ever evolving new threats to consider and the proliferation of new shift-left security tools designed to combat them.

Herein lies the problem: how can software based companies achieve dev-native security while maintaining development velocity?

That's where the Minimum Viable Security (MVS) approach comes into play.

What is a Minimum Viable Security and how does it relate to software development?‍

We are all familiar with, and many of us follow the concept Minimum Viable Product (MVP)—a product is initially built with the minimum set of features needed to test market fit and ensure the business strategy without first expending all the resources. Then, the product continues to be optimized with an MVP mindset, adding minimum viable new features or capabilities every cycle. In fact, many of the most popular software products brought by brands we all know and respect are built that way.

During the software development, this is done iteratively, focusing on delivering a minimum baseline value with every single version.

The MVP approach to the product is analogous to the MVS approach to security.

The MVP concept, each release is a standalone viable one.
‍
For developers to be willing to take over security responsibilities and fully own them, the process must work like any other aspect they are familiar with: starting small/lean, improving in a continuous and agile manner, automating as much as possible along the way, and running security 'as code.'

Let’s take this a step further.

Minimum Viable Security in Details ‍

Starting with a Minimum Viable Plan

While different security checklists that can be found all over the web are available to engineering leaders, such as Google's newly developed MVSP mentioned above, they are hardly helpful if you want to come up with a minimum security plan that is operational.

It is crucial to always keep in mind the distinction between a high level security checklist and a product-tailored actionable plan.

A Minimum Viable Security Plan is not a checklist, but a detailed, actionable, step by step plan, that includes all of the processes and needed tools, but most importantly, defines the minimum amount of steps developers should take to make a product secure enough for a specific purpose - just in time.

For instance, consider a security baseline that is based on a checklist and it codifies the most up-to-date knowledge and strategies designed to deal with specific threats to a company's tech stack. It needs to be as simple as possible, still to cover the entire product boundary, be continuously updated, and follow GitOps principles (with customizable code).

You can’t expect developers to master such a task without properly equipping them.

The first obstacle developers face is knowledge. They need to be in the know of the security threat landscape, in addition to the relevant tooling (and there are many of them). They must keep updated, codify the plan and keep the codified plans evergreen.

Unlike a checklist, an MVS plan should easily codify this knowledge and create the initial capability to continuously and automatically update the product’s security.

As mentioned above, the plan must also constantly evolve and include additional plans at each stage to support the constantly maturing product. A serverless plan, for instance, isn’t a SOC2 compliance plan, and isn’t an OWASP Top 10 plan and so forth.

A codified plan is a necessity for developers that are lacking security domain expertise

Images below taken from the Jit platform: a couple of different MVS (minimum viable security) plans available to automatically activate:

Born left MVS? Security as Code

Identifying and selecting the optimal tools (open source or not) that are required as part of implementing the plan is a resource intensive, tedious task that takes a lot of time and effort. Integrating OSS tools into relevant stacks, testing them, and plugging them in to run automatically via CI/CD in a security-as-code format is another heavy task that is also a key part of the born left, dev-owned security mindset.

On top of that, if you wish to be effective, tool selection and integration must be continuously updated due to the nature of cyber threats and security vulnerabilities. That means it should therefore be fully automated, and properly orchestrated, both as part of the development environment and as part of the pipeline - following the concept of MVS as code.

If you expect developers to initiate the above on their own, a common problem is overstretching an already busy team.

Adding new responsibilities in fields where developers aren’t experts take a toll, resulting in 'Shift Left fatigue’ (as seen in many discussions). That’s making the case for a Born-Left approach even more compelling, given that the born-left approach is an alternative, offering the relevant tooling to actually do the heavy lifting including the automation and orchestration.

To summarize, automating and implementing product security plans as code and following GitOps principles in familiar development environments significantly reduces the Shift Left burden. Image below: The inventory of security actions that are included in specific MVS plans; some are shared across plans:

Maintaining Velocity and Avoiding Developer Burnout

To meet MVSP requirements while maintaining development velocity and not burning out your developers, adopting an MVS mindset and taking an automated approach to product development is essential.

This includes automation of:

Continuously updated and constantly evolving MVS plans
MVS plans-as-code, with security tests generated by multiple tools.
Integration and orchestration of multiple security controls, in a unified and consolidated interface, as part of the dev environment and pipelines.

There are many things to consider when it comes to MVS requirements. The tech industry has already united to formalize some guiding principles and define standardization practices that match the evolving threat landscape, the next step is the implementation as code.

Feel free to get started here >> www.jit.com

Bootstrapping a Secure AWS as-Code Environment - Your MVS Checklist

David Melamed — Tue, 22 Mar 2022 17:32:58 +0000

Infrastructure as Code (IaC) has changed the way we manage our cloud operations, by making it infinitely easier and quicker to roll out infrastructure on demand––with a single config file.

In this article, we’ll delve into both the benefits and security challenges introduced to the underlying stack that comes with adopting an AWS anything-as-code model. We’ll also introduce the minimum viable security (MVS) approach that delivers baseline security controls for any stack.

Embracing an Everything-as-Code Model

In line with the principles of IaC, organizations are increasingly adopting as-code frameworks for different components of a tech stack, including security, policy, compliance, configuration, and operations. AWS supports various as-code frameworks, including their own CloudFormation, Terraform, Pulumi, and have even recently rolled out their next-gen IaC in the form of AWS CDK. By providing an API to provision and manage resources, it’s now possible to spin up a complex cloud architecture by defining simple, code-based templates.

With environment-as-code pipelines, organizations can then manage and extend their deployment environment across multiple regions and accounts through a single workflow, leveraging the same code.

Adopting Minimum Viable Security for Baseline Security Controls

While IaC frameworks come with the benefits of automation across the entire stack for more rapid delivery and tighter controls, securing each environment comes with its own unique set of challenges. On top of this, with all of the noise and panic being constantly generated around security and exploits, it is hard for organizations that are just starting up to understand the minimum critical controls, and what should be out of scope. The end result being that emerging companies striving to launch the first version of their product, have little understanding of the baseline security they actually need to implement to get ramped up.

To solve this, the MVS approach offers a vendor-neutral security baseline that reduces the complexity and overhead when deploying infrastructure, and specifically cloud (native) environments. Similar to Agile methods, MVS focuses on a minimal shortlist of critical security controls that adds some initial security to the launched product to tackle the most common threats. This approach helps organizations establish a sufficient security posture while integrating seamlessly into existing automation tooling and pipelines used for configuring today’s complex cloud-based environments.

In order to demonstrate this in practice, we’ll show how this actually applies and works when securing AWS instances (as the most popular and most widely adopted cloud), through the automated MVS approach.

Best Practices for How to Secure AWS Environments as Code

At Jit, we have identified a few layers upon which we focus our security controls for AWS environments that provide the baseline security required that can be expressed as code to automate the bootstrapping of your AWS environments without compromising velocity.

These include:

Account Structure
Identity and Access Management
User Creation and Secret Management
Hierarchies, Governance, and Policies
Access Controls

Below we’ll dive into each individually and how we can automate these eventually within your existing IaC and automated pipelines.

Build a Secure AWS Account Structure

Many of the practices we will list below are tried and true, and applied at Jit for our own security controls. First off, we split our AWS accounts into three primary organizational units (OUs): users, a sandbox, and workloads.

The users OU let us host a dedicated account to set up all users.

The sandbox unit is for developing or testing new code changes. This OU can also host accounts for experimenting with as-code templates and CI/CD pipelines.

Workload units include staging/production environments and contain various accounts that run external-facing services.
As cloud workloads grow, DevOps teams are inclined to set up multiple accounts for rapid innovation and flexible controls. The use of multiple AWS accounts helps DevOps teams achieve isolation and independence by providing natural boundaries for billing, security, and access to resources.

While building AWS accounts, the below practices are recommended:

Use organizational units (OUs) to group accounts into logical and hierarchical structures. Accounts should be organized based on similar and related functions instead of an organization’s reporting hierarchy. Although AWS supports OU depth of up to five levels, it is best to maintain the lowest possible structure depth to avoid complexity.
Maintain a master account created for managing all organizational units and related billing for cost control and ease of maintenance.

Assign limited cloud resources, data, or workloads to an organization’s management or master account for maximum security since the organization’s service control policies (SCPs) do not apply to the management account.
Isolate production and non-production workload environments from each other. AWS workloads are typically contained in accounts, where each account can have more than one workload. Production accounts should have either one or a few closely related workloads. By separating workload environments, administrators can secure production from unauthorized access.

Follow Identity and Access Management Practices

For managing access to and permissions for AWS resources, Identity and Access Management (IAM) offers a first line of defense by streamlining the creation of users, roles, and groups. When provisioning an AWS environment through automation, organizations should leverage existing modules to manage IAM users, roles, and permissions.

Administering robust security through IAM typically relies on a set of common practices that we also apply internally at Jit:

Make sure IAM policies for a user, group, or role grant only the permissions needed to accomplish a given task––this approach is also dubbed “least privilege” and there is plenty of excellent material about it. Permissions should initially only contain the least number of privileges required; these can later be increased if necessary.
Create separate roles for different tasks for each IAM user.
Use session tokens as temporary credentials for authorization. You should additionally configure a session token to have a short lifetime to prevent misuse in the event of a compromise.
Do not use a root user’s access key to perform regular activities or any programmatic task, as the root access key grants full access to all AWS services for any resource. You should also rotate the access key of the root user regularly to prevent misuse.
If account users are allowed to select their own password, make sure there is a strong baseline password policy and the requirement to periodically change it.
Implement multi-factor authentication (MFA) for additional security. MFA adds an additional layer of authentication on top of the user credentials and will continue to secure a resource in the event that credentials are compromised.

Automate User Creation with Encrypted Secrets

To cut down the risks associated with manual efforts, it is strongly recommended that organizations embrace automation for user creation. This ensures that all stages of the process flow, including account creation, configuration, and assignment to an OU, require minimal manual intervention.
Automation also assists with streamlining user experience by integrating with onboarding and offboarding user workflows. The mechanism provides a fine balance between agility and control by permitting automated configuration and validation of IAM policies across multiple environments (dev, staging, or production).

Figure 1: A typical user creation process flow (Source: Amazon)

Apart from user creation, you should also automate identity federation and secret provisioning to ensure a comprehensive user creation cycle. A typical workflow resembles the above process flow, along with leveraging tools such as Keybase for the automatic encryption of credentials and keypairs, supported by IaC frameworks like Terraform. ]

Create Hierarchical Structure & Policies with AWS Organizations

AWS Organizations helps you implement granular controls to structure accounts in a manageable way. The service offers enhanced flexibility and hierarchical structure to AWS resources based on organizational units (OUs). For any AWS organization, it is recommended to start with a basic OU structure with core OUs such as infrastructure and security.
You should also create a policy inheritance framework that allows maximum access to OUs at the foundation level and then gradually limits access with each layer of the OU. This layering of policies can further continue to the account and instance levels.

Organizations should also apply service control policies (SCPs) on the OU rather than individual accounts. SCPs offer a multi-layered approach to access management, as they offer a redundant security check that takes precedence over IAM policies.

As a best practice, it is recommended to use trusted access for authorizing services across your organization. This mechanism helps to grant permissions to only designated services without affecting the overall permissions of users or roles. As workloads grow, you can include other organizational units based on common themes, such as: policy staging, suspended accounts, individual users, deployments, and transitional accounts.

Secure Remote Access to the AWS Console

Securing remote access to the AWS console is one of the easiest yet crucial parts of maintaining security in an AWS as-code environment. A minimal approach here can be achieved by leveraging the AWS Management Console and AWS Directory Service to enforce IAM policies on account switching. Once logged in, based on the user’s role (read-only or read-write access), this approach allows individual users to switch accounts from within the console.

Additionally, you can also enforce MFA through a trust policy between the user’s account and the target account to ensure only users with MFA enabled can access the target account.

Enforce Secure Access of AWS APIs

Since the majority of API endpoints are public-facing, it is extremely crucial to secure them. It is always recommended to limit unauthenticated API routes by enforcing a robust authentication and authorization mechanism for accessing the APIs. Apart from leveraging various AWS built-in mechanisms to safeguard both public and private API endpoints, you should also adopt minimal security controls such as enabling MFA to use the AWS CLI or using AWS Vault to secure keypairs.

Apart from this, there are several approaches to achieve controlled access to APIs. These include:

IAM-based role and policy permissions
Lambda authorizers
Client-side SSL certificates
Robust web application firewall (WAF) rules
Throttling targets
JWT authorizers
Creating resource-based policies to allow access from specific IPs or VPCs
API keys

AWS Security - the TL;DR

The as-code model for various computing components allows you to automatically, consistently, and predictably spin up deployment environments using manifest files. While the everything-as-code approach simplifies the deployment and management of resources on AWS, security can’t be ignored as part of this process and should also benefit from the guardrails automation can provide.

This article delved into the MVS approach and how it can be applied as code. In the next article of this series, we will give concrete code examples of how to bootstrap a secure AWS environment using Terraform in practice.

5 Open Source Security Tools All Developers Should Know About

David Melamed — Wed, 26 Jan 2022 08:33:27 +0000

With product security becoming an increasingly important aspect of software development, “shift left” is gaining wide acceptance as a best practice to ensure security is baked into development early. More and more traditional (read: incumbent) security companies are releasing shift-left products and capabilities, and the practice is becoming almost de facto for engineering teams.

However, the industry has begun to realize that simply “shifting left” is hardly enough for a continuous delivery world. High velocity, progressive development teams are embracing a new and trending “born left” security approach, where security aspects - like more and more product-related aspects - are addressed starting from the first line of code. This means product security isn’t just delivered by the developer team, but is rather owned by them.

Understanding this shift, comes with the realization that already burdened developers are faced with additional responsibilities that are continuously dropped in their lap. This has led the industry to hunt for ways to provide solutions and tools that help developers manage this growing workload, including security, and maintain velocity.

We acknowledge that currently the open source “shift left” tooling doesn’t solve the overhead put on the developers (due to the noise these create, and the burden of learning security in general and the ropes of each open source tool). That is on our shoulders to solve.

But still, not all open source tools are created equal, and there are quite a few open source security tools that are not-only developer-friendly, but provide much-needed security controls early in the development cycle. That’s why we’ve compiled a list of 5 security tools that we believe all developers should know about, and consider adopting if they do not have such a control currently in place. This post will provide a quick overview of what makes a tool developer-friendly. We will then introduce a tool per security domain and its coverage, do a quick dive into how they work, and why you should consider adopting it into your toolchain.

What makes a tool developer-friendly?

Let’s start by defining what makes tools ‘dev-friendly’ in the first place.

To me, a dev-friendly tool sets out to make developers (and dev leaders) lives easier by either simplifying tasks or speeding up processes.

Open source

One of the greatest benefits of open source tools is that they are free to use (of course check the license first!), so there is no need for budget approval, and you can try a tool out locally (you should probably make sure to verify before using it on any company resources - more on that later), without having to commit to it. Instead of lengthy selection processes, you can simply try it out and see how you like it. In addition, and this is particularly critical for security tools, as the name implies, they provide you access to the entire codebase, so that you don't have any surprises regarding what actions the tool is performing when running it in your environment.

Runs locally first...

Running code locally from your terminal allows software developers to launch and test code with one simple command. The ability to run a tool locally ensures that you can get immediate feedback and easily tweak the configuration. When launched from a container, you don't even have to bother with possible environment issues related to compilation.

... and then in the CI/CD pipeline

Tools that can be integrated into the CI/CD pipeline have higher value. Once I have used a tool locally and found it to be useful, then my next move would likely be to run it continuously as part of my development lifecycle - and not only on my local machine, consuming my local resources. Of course, once a tool and process is part of the pipeline, the benefits are also extended across the entire dev team and codebase. So, starting locally is an advantage, but then being able to easily integrate the new tool into existing environments and processes is an advantage as well.
‍

Part of the developer work environment

Developers should not be wasting time switching between development tools and security tools. All the tools on this list either run in the CI/CD pipeline (e.g. Github Actions) or as a plugin into the IDE. Context switching has been proven to adversely affect flow and productivity. The less context switching, the greater the development velocity, and happiness.

Great Documentation

I believe this requires little explanation...if a user doesn’t know how to use your tool practically then, you’ve gained very little by releasing it.

Readily available documentation made for dev professionals can make or break a smooth user experience. With great “how-to” documentation, ramp-up time is much shorter.

The better the documentation, the smoother the learning curve, and the easier to troubleshoot, making the tool significantly easier to adopt.‍

Configurable output formats

If you can receive a tool’s output in multiple formats, it then becomes possible to ingest this output by another tool through an API or other form of integration, allowing you to manipulate and analyze results in other tools. If results are only readable by humans, what you can then do with those results is limited and requires human effort - i.e. time that you simply don’t have.

So without further ado...

5 Open Source Security Tools We Love - And You Should Too

Based on the 5 criteria above, I’ve collected five security tools that are dev friendly, and I’ve enjoyed using them as a security engineer.

The list aims to cover various domains of code analysis tools that should be a part of minimal requirements for security applied to development processes:

Static application security testing (SAST)
Dynamic application security testing (DAST)
Hard-coded Secrets detection
Infrastructure as Code analysis (IaC)

Pycharm Python Security Scanner

Pycharm Python Security Scanner is a security scanner for Python code wrapped as a Pycharm plugin, checking for vulnerabilities while also suggesting fixes. Alongside acting as a comprehensive security scanner, it also offers some additional extensions that can run dependency check analyses as well, which are quite useful.

What makes it unique is that beyond being a plugin, it is also available as a CI/CD workflow for GitHub Actions in the Github Marketplace.
‍

Semgrep

Semgrep is a highly-configurable SAST tool that looks for recurring patterns in the syntax tree. It can either run locally using Docker or be integrated into the CI/CD pipeline with Github Actions.

Results are delivered as JSON files, allowing you to pipe the results into other tools, like jq in order to manipulate them.
‍

gitLeaks

Gitleaks is a great project used to quickly detect hard-coded secrets based on a configuration file containing hundreds of built-in regex expressions tailored to find API keys of popular SaaS. It can run locally using Docker and or be integrated into the CI/CD pipeline with Github Actions. Results are delivered in various formats.

The rules can be easily extended to match your internal patterns and homegrown tools as well.‍

ZAP

OWASP's Zed Attack Proxy (ZAP) is another open source tool, used for dynamic scanning (DAST) built by the OWASP team (the same folks who gave us the Top 10 Security Vulnerabilities). It can run locally using Docker and provides a Github workflow to run in the CI/CD pipeline.

The common output for this tool is an HTML report, but you can also receive the output in JSON with an addon.

KICS

KICS is used to perform code static analysis of infrastructure, and includes about 1,400 rules supporting various platforms like Terraform, CloudFormation, Ansible or Helm Charts. It can run locally using Docker and can be integrated into the CI/CD pipeline with Github Actions.

High Velocity Development and Security

Development teams are being tasked with end-to-end responsibility and ownership of their products - whether it’s production readiness, performance or security, while all along there’s the pressure to ship code to production with high velocity.

This growing challenge is what set us out on a mission at Jit, to ease this growing burden on developers making the ownership of product security much simpler - from the planning, through open source orchestration and more, based on an MVS approach (Minimum Viable Security). Basically this manifesto says start small, and constantly iterate, you don’t need to build a fortress on Day 1, but have baseline security controls, and grow from there.

As I mentioned above, while dev-friendly security tools offer great benefits, the growing responsibility assigned to developers requires a shift in today’s approach - one that requires a minimum viable mindset and automated orchestration, so that devs will be able to own product security without compromising velocity.