DEV Community: Alex Savage

API provider contract testing for all with Portman, OpenAPI and Postman

Alex Savage — Wed, 10 May 2023 15:08:00 +0000

Introduction

APIs are everywhere and power almost everything we use today.
OpenAPI has revolutionized the way APIs are designed, documented, deployed... Its kind of a big thing.

Making a great OpenAPI definition is helped with amazing tools such as Stoplight Spectral which can help you ensure that your definition is not only valid against the OpenAPI specification but also your own API standards through custom linting rules.

How do you ensure that what you create and deploy matches your definition? Do you write all your tests by hand? Do you probe with Postman and hope for the best?

Maybe you do today but this blog will help you try something new... Welcome to the world of contract testing.

By the end of this blog you will have completed your first contract test (against my mocked API) and you should be free to use this knowledge to find gaps in your or any API out there subject to permission!

Getting started

Rather than spending too long explaining what contract testers are, where they came from etc... Lets use one! (It's free so what is stopping you?)

We will be using apideck Portman. You are welcome to follow their readme.MD and come back later or follow the instructions here.

Installing Portman

Install Portman

$ npm install -g @apideck/portman

Initialize your CLI

portman --init

Creating a contract test to run in Postman

Make a new directory and download this example portman configuration file and store it there
https://api.oneadvanced.com/portman/configurations/portman-config.adv.json

Download this openAPI definition from SwaggerHub (the unresolved YAML works fine)
Link to SwaggerHub: https://app.swaggerhub.com/apis/AdvancedComputerSoft/Contract-Breaker/1.0.0
Direct link to YAML: https://api.swaggerhub.com/apis/AdvancedComputerSoft/Contract-Breaker/1.0.0

Open the directory that has both files and run the following:

portman -c portman-config.adv.json -l AdvancedComputerSoft-Contract-Breaker-1.0.0-swagger.yaml

Portman will use the configuration file to port the openAPI definition to a Postman collection you can now import into Postman and start running tests!

Run the test

Import the Postman collection file from the tmp/converted folder that Portman created as part of the process.

Open the List Countries request and press send!

Congratulations! You ran a test and you got some failed tests. (Hopefully)
Don't worry, that was supposed to happen.

What happened?

Lets go back and look at the collection.
At first glance, it looks like any other collection. Check the tests tab and you should see something new.

This is where Portman has done its magic and created contract tests for you to run against the API. The OpenAPI definition was used to generate these tests.

You can see Postman will now test to ensure:

The status code matches 200
The response is application/json
The response has a body which is an object
The response body validates against the schema from the OpenAPI
Bonus - The response is received within 2000ms (This is not taken from the OpenAPI definition but from the Portman configuration

All of this and you did not need to write anything. No human error, no missed tests. All of it was auto-generated.

Interpreting the responses

Postman has reported that 3 out of 5 tests have failed:

1) [GET]::/countries - Response status code is 200 | AssertionError: expected 201 to equal 200

This error was raised by the status code check as the API incorrectly returned a 201 response code instead of a 200 which it was expecting

2) [GET]::/countries - Schema is valid | AssertionError: expected data to satisfy schema but found following errors: data.data[0] should have required property 'countryId', data.data[0].population should be integer, data.data[1] should NOT have additional properties, data.data[1].countryId should be string, data.data[1].population should be integer, data.pagination should NOT have additional properties, data.pagination.offset should be integer

This error was contains all the json schema validation issues. There are missing required properties, wrong types and additionProperties (these are caused by misspelled properties or those that were not documented)

What you should find is this isn't an API that you would want to release! Contract testing lets you find where your deployed API (or your one that you are developing) doesn't match the OpenAPI definition. Catch things now before you release!

More tests

Portman has lots of other functionality to explore.

I would recommend to first look at overrides to try manipulating the collection. I started by making tests that removed the security and tried to call the API. You can stipulate which response from the OpenAPI you wish to receive. I was expecting a 401 to be returned. The Portman documentation has details of all of the functionality on offer.

Summary

APIs need to match their documentation. Whether or not you start with Design in OpenAPI or start with code, contract testing can ensure the 2 are in sync so your consumers don't get surprises and raise issues.

JWT.io signature validation

Alex Savage — Wed, 09 Feb 2022 15:59:51 +0000

Have you ever wondered how jwt.io gives you the Signature Verified badge when you paste a new JWT?

Note: This only works for asymmetric signing algorithms such as RS256 signed JWTs.

The JWT was signed using a private key which is safely inside the issuer but there is a public key available so that any recipient of the token can validate if it is valid or not.

Introduction

This blog is focused on using a familiar and popular tool as a worked example for how automatic signature verification of JWTs can happen. It is not about recommending you to or not to use tools like this. I do however want to pause for a second to remind you that jwt.io is an online tool. It does state that it runs in the client, but as ever you should never automatically trust online tools and always consider the risk / threat they pose.

As a minimum, I would suggest never putting production tokens into sites like this (or anywhere you don't control) and recommend you find a locally runnable alternative.

Jamie Tanna who is a member of my favorite API community (APIs you wont hate), wrote a fantastic blog actively discouraging the use of tools like this: Why I Actively Discourage Online Tooling like jwt.io. This might be a strange link in a blog that explains how this tool works but it is a popular tool that many take for granted and has some interesting functionality to share.

Hopefully the introduction has not put you off (that was not the intention). If not, lets explain how it works.

How does it work?

Inside the decoded JWT payload is the iss property (Issuer of the token). This will be a URI that can be used to find the well-known endpoint and from there the jwks_uri which is the location where the public key(s) can be retrieved.

Important note: JWTs are untrusted input so you should not directly trust what is located inside! This means if you are relying on the issuer to retrieve the public key, make sure you are validating the issuer before calling it to retrieve a public key. It might not be your issuer! Similarly the token could tell you where the public keys are via the jku parameter to send your verify function to retrieve keys an attacker controls.

Below is an example of an access token that is a JWT in encoded and decoded form using https://jwt.io

What is a well-known endpoint?

Thanks to RFC8414 - OAuth 2.0 Authorization Server Metadata, authorization servers provide a well-known endpoint that returns information about how to interact with the service at:

http://service-url.com/.well-known/openid-configuration

It returns information such as the OAuth token and authorization endpoints, which OAuth grants are available, scopes and many more. In order to retrieve the public keys we need to validate our token, we need the jwks_uri



// Example response from a well-known endpoint provided by Auth0.com
// Some items are removed to reduce the overall height of this code block
{
  "issuer": "https://dev-spu08zrz.us.auth0.com/",
  "authorization_endpoint": "https://dev-spu08zrz.us.auth0.com/authorize",
  "token_endpoint": "https://dev-spu08zrz.us.auth0.com/oauth/token",
  "jwks_uri": "https://dev-spu08zrz.us.auth0.com/.well-known/jwks.json"
  "scopes_supported":
    [
      "openid",
      "profile",
      "offline_access",
      "name",
      "email"
    ],
  "code_challenge_methods_supported": ["S256", "plain"],
  "response_modes_supported": ["query", "fragment", "form_post"],
  "subject_types_supported": ["public"],
  "id_token_signing_alg_values_supported": ["HS256", "RS256"],
  "token_endpoint_auth_methods_supported":
    ["client_secret_basic", "client_secret_post"],
}

jwks_uri

The jwks_uri returns the JSON Web Key Set (List of JSON web keys). There is often more than one to allow for key rotation. Each key in the set contains the public key that can be used to verify the signature of a the JWT.

Auth0 has a detailed write up on JSON Web Keys here

Here is an example response from a jwks_uri. You can see there are 2 keys currently available. It contains the required information for a token verifier to locate the correct public key that matches the kid (key id) from the token header and pass it to the verifying function.



// Key length has been shorted to make this code block more readable.
{
  "keys": [
    {
      "alg": "RS256", // This is the algorithm that was used
      "kty": "RSA", // This is the key type: RSA being asymmetric
      "use": "sig", // This is the usage: sig = signing, enc = encrypting
      "n": "xi9SZLDzHULsG_ab9zBO2....", //This is the modulo of the public key
      "e": "AQAB", // This is the exponent of the public key (used with the modulo)
      "kid": "uMSsz8nx8OEuXhEbnXcoH", // This is the Key Id needed to match the header
      "x5t": "k1uZJUy2G7i-acZ36Yg....", // This is the thumbprint of the x509 cert
      "x5c": [
        "MIIDDTCCAfWgAwIBAgIJEd...." // This is the x509 certificate chain
      ]
    },
    {
      "alg": "RS256",
      "kty": "RSA",
      "use": "sig",
      "n": "yBMPFvevpfesWmtdmt8V....",
      "e": "AQAB",
      "kid": "oHf3mAsLQ2vaXgxMRP0qh",
      "x5t": "jN_H055zOjrXkQSdiunn....",
      "x5c": [
        "MIIDDTCCAfWgAwIBAgIJej6...."
      ]
    }
  ]
}

Verifying JWTs

Using the kid and alg in the JWT header (JOSE Header from RFC7515 - JSON Web Signature (JWS)) and the public keys from the authorization servers jwks_uri, we can verify the signature.

To verify the signature, jwt.io uses the JWK’s: key id, e and n values which represent the public key.



{
    "alg": "RS256",
    "kty": "RSA",
    "use": "sig",
    "n": "xi9SZLDzHULsG_ab9zBO2b1L...",
    "e": "AQAB",
    "kid": "uMSsz8nx8OEuXhEbnXcoH",
    "x5t": "k1uZJUy2G7i-acZ36YgCw6...",
    "x5c": [
        "MIIDDTCCAfWgAwIBAgIJEd8o..."
    ]
}

If you want to manually verify signatures with jwt.io, you will need to paste in a JWK (as above) into the box in the bottom right (normally automatically completed) and it will verify the signature.

What about PEMs?

Some verify functions such as the popular Auth0 node-jsonwebtoken library requires the public key in PEM format. There are many libraries available to do this conversation.

Here is an example of a JWK converted into PEM format: (Its not a real one)

Summary

If you are using JWTs today you have probably come across JWT.io, but perhaps have never given a second thought to how the validation works. I hadn't, but found it interesting and felt the need to share with others in case they did too.

Perhaps you are using them today but had not looked into the detail of the various components you are or could be using.

I hope you found this helpful to gain some more knowledge but as always, be careful and keep your applications safe. They are many ways to open up security holes by misconfiguration or out of date libraries. Do your research and lots of testing.

Safely Handling JWTs

Alex Savage — Wed, 02 Feb 2022 13:58:54 +0000

Introduction

This article is focused on helping people already using JWTs today.
I did not want this article to try and persuade you to or not to use JWTs, there are plenty of those already. What I struggled to find was a summary for existing users. So if you are using them, let's help make sure you are handling them safely.

TLDR

What they are

JWTs - JSON Web Tokens pronounced as “JOT”.
User input so treat them as untrusted.
Start with eyJ (base64d JSON object).
Have 3 parts: header.payload.signature separated by dots.
Able to be signed by a variety of algorithms including none (no signature at all).
Hard to get right! Even some of the best get it wrong: Auth0's 2020 validation bypass.
Come in different types: An example being an access token.

The problems they can solve

Exist to allow claims to be transferred between two parties
Could be used to support solutions secured by OAuth2.0 and OpenID, to allow 3rd party applications to call the API on behalf of the user without the 3rd party having the users credentials (depending on the implementation). Meaning the user may not have an active session with the 1st party application but the API can still be called.

What you need to do

Choose one algorithm and stick to ONLY ONE such as RS256. Turn off the rest including the option for none.
Use an identity provider to create JWTs.
Decode != verified you need to do both before acting on the information.
Don't write your own decode or verify functions, use a library like https://github.com/auth0/node-jsonwebtoken.
If it doesn't decode, reject it
If it doesn't verify reject it
Verify functions often have a lot of options. Use them!
- Algorithm
- Issuer
- Audience
- Issued at isn't in the future
- Not before isn't in the future
- Bonus: expiry clock tolerance (max 30 seconds)
Check the scopes
Check the claims
Bonus: If you can apply a type to the token. Check it's an access/ bearer token.
If you are rotating signing certificates which you should, check if you have the public key matching the kid in the header before trying to validate.
- Check the issuer is trusted before attempting to retrieve new certificates. If it's not, reject it.
- Ideally your services would already have the latest / current key through another means after the rotation.
- Do not recklessly trust the issuer field and retrieve new public keys from it's JWKS_Uri!
If you're using a symmetric algorithm like HS256, make sure the secret is changed to something unique and long enough. (At least as long as the resulting token)

TLDR not enough?

What is a JWT?

RFC7519 - JSON Web Token (JWT) defines a JWT (Often pronounced “jot”) as:

JSON Web Token (JWT) is a compact, URL-safe means of representing claims to be transferred between two parties. The claims in a JWT are encoded as a JSON object that is used as the payload of a JSON Web Signature (JWS) structure or as the plaintext of a JSON Web Encryption (JWE) structure, enabling the claims to be digitally signed or integrity protected with a Message Authentication Code (MAC) and/or encrypted.

They have 3 parts separated by dots (note the colours in the figure below).
Each are base64 encoded separately.
They are instantly recognizable as they start with eyJ… (the first 2 parts are JSON objects when when base64 encoded become eyJ).

You may have seen them in cookies or Authorization headers with the word “bearer” in front of them.

Below is an example of a access token that is a JWT in encoded and decoded form using https://jwt.io

The 3 sections are as follows:

The first section shown in red is the Header (JOSE Header) which tells the receiving party how it's signed and which key has been used (if appropriate).
The second section shown in purple is the Payload with some standard properties based on the RFC but can also include custom properties/ claims.
The third section shown in in blue is the Signature for consumers to use to validate that the JWT was issued by the trusted authority.

The access token can be likened to a passport that states some claims and is certified by a trusted authority. Like a passport, they can be tampered with so we need to ensure we properly validate and verify the contents using the tools available and protect ourselves from the numerous vulnerabilities that have been discovered and treat all input as untrusted user input.

Signing algorithms

Your authorization server will let you choose which algorithm you want your tokens signed with.

Choose one, use one and only one if you can. The verify function will need to be told which you support and will reject anything else. You could even check before you verify and if you receive a token with anything else, reject it.

Algorithm names are usually split into 2 parts:

Signature algorithm: In our example, RS means RSASSA-PKCS1-v1_5 (typically)
Hashing algorithm In our example, 256 means SHA-256
Together it's known as “RS256”, a very popular JWT signing algorithm. It has been the default for Auth0 since around 2019.

Note: The latest signing algorithm “EdDSA” doesn't follow the same naming setup as the others (as does the “none” algorithm).

RFC7518 defines a list of 13 options (One being “none” where the JWT has no signature at all!)

In addition to the RFC, there are others available which you can find on the iana registry: JSON Object Signing and Encryption (JOSE)

What should I use?

RS256 and HS256 are the two most popular signing algorithms and will have the best support from libraries you might use. Two others to note are ES256 and PS256 which are mandated by OpenBanking so are gaining popularity. Another new algorithm to take note of is EdDSA which is recommended by cryptographic experts but today lacks the universal library support offered by some of the older options.

RS256 - Asymmetric (Private Key to sign and Public key to verify) using RSASSA-PKCS1-v1_5 RFC 3447 - Public-Key Cryptography Standards (PKCS) #1: RSA Cryptography Specifications Version 2.1) with a SHA-256 hashing

HS256 - Symmetric (shared secret used by signer and verifier) using SHA-256 hashing

Both are as cryptographically safe as each other but RS* type algorithms have the advantage of easier implementation and maintenance as you do not have to do secrets management in your consuming service. This has the added benefit of a stronger trust level as no shared secret is ever exposed. Not exposed = reduces chance of it leaking and tokens from rogue actors being received.

The other possible negative of HS256 or any symmetric algorithms is that because you have to distribute the secret to any verifying services, it has a much higher chance of being leaked (it’s a private key that should not normally ever leave the authorization server or a certificate store). This means the trust level is reduced as you cannot say 100% that the token was signed by the authorization server you are expecting.

A final note on symmetric algorithms is that the secret used to create the signature needs to be unique, sufficiently large and complex to prevent it being brute forced. There have also been instances where people have forgotten to change the default secret included with the package or example code they used. These have been compiled into helpful word-lists such as Wallarm's JWT Secrets list. This allows attackers to quickly check if your JWT is signed with something from an example rather than a traditional brute force.

Decode

As JWTs are base64 encoded, they need to be decoded before the information inside can be reviewed and the JWT verified. Important node: Decode != Verify

The library you are using may have a decode function included as well.

Before running the verify function you may choose to validate some of the tokens claims such as the issuer if you need to support more than a few. Most verify functions will take in a list of issuers but you may have too many to pass. It may be better to verify the issuer yourself and then pass the issuer in as an additional option for the verify function.

If you are doing RS256 you will want to check if the key Id that is present in the header matches the key Id you have. If it doesn't then the verify function will fail. There may have been a genuine key rotation meaning new tokens have a new key you don't have (or it's a bad token). Retrieve the keys from the JWKS URI and check if the key matches, if multiple keys are present but neither matches, reject the token. If one does match, optionally save it, then send it to the verify function.

Do not act on the information in the token until it's verified.

Verification

Verification ensures that the token is signed correctly and that it's currently valid for use.

Where verification often fails due to missing or incorrect configuration:

Algorithms are not defined. As above there should only be one algorithm in use which you can pass to the verify function.
Issuers are not checked. You know who can issue tokens for your service. Ensure that you are passing that issuer into the function as it will be able to check if that was the claim versus your expectation. You may have already checked this as well once it was decoded.
Audience isn't checked. You may have multiple APIs. Ensure that the tokens received are for the API that is being called.
Scopes are not checked. You may have scopes providing coarse grained authorization on the API. Check that the scope(s) for the resource that is being requested are present.

You can read what Auth0 has to say about validation here:

Rotate your keys

You should rotate your keys to prevent brute force attackers from cracking the key or secret used to sign the tokens. For asymmetrically signed tokens, this should be fairly trivial as you will likely have implemented a key retrieval mechanism. If you receive a new key, you would need to trigger that.

For symmetrically signed tokens this is much harder as you need to provide the secret to all services via a secure back channel. Note that some identity providers use symmetrically signed tokens for refresh tokens as they are the sole consumer of them so already have access to the secret.

Be careful to provide a long enough overlap depending on the lifespan of your tokens. You wouldn't want to reject a legitimate token that has not expired yet.

Okta's fantastic write up on key rotation

Summary

Read the TLDR
Choose a single algorithm
Decode and validate
Validate issuers against a known list or a rule set.
Don't act until it's validated
Test it in dev and prod

DEV Community: Alex Savage

API provider contract testing for all with Portman, OpenAPI and Postman

Introduction

Getting started

Installing Portman

Creating a contract test to run in Postman

Run the test

What happened?

Interpreting the responses

More tests

Summary

JWT.io signature validation

Introduction

How does it work?

What is a well-known endpoint?

jwks_uri

Verifying JWTs

What about PEMs?

Summary

Safely Handling JWTs

Introduction

TLDR

What they are

The problems they can solve

What you need to do

TLDR not enough?

What is a JWT?

Signing algorithms

What should I use?

Decode

Verification

Rotate your keys

Summary

Research links