DEV Community: Thomas Step

How to Create a Custom Error Class in Javascript

Thomas Step — Thu, 27 Jan 2022 09:00:09 +0000

This is going to be short and sweet.

class MyError extends Error {
  constructor(message) {
    super(message);

    this.name = 'MyError';
  }
}

module.exports = {
  MyError,
};

You can throw the error like so.

const { MyError } = require('./errors');

try {
  throw new MyError('This is my error that threw.');
} catch (err) {
  if (err instanceof MyError) {
    console.error('Instance of MyError found.');
  }

  console.error(err);
}

How I Implement Hexagonal Architecture in AWS Lambda

Thomas Step — Thu, 27 Jan 2022 09:00:09 +0000

At re:Invent 2021, I learned about hexagonal architecture. I am a curious person so I started looking further into it. Like most architecture concepts, I could not find many examples of hexagonal architecture implementations. I ran back into my own article about hexagonal architecture or some copy and pasted content farm's article which took some buzzwords from Alistair Cockburn's original post and made it their own.

Fast forward to this week and I stumbled across a video on YouTube titled Hexagonal Architecture by Example. This is what I was originally looking for, and it spurred me on to finish out my understanding of this architecture. The video is helpful to watch through, but the GitHub repo they made available was my fast track to a better understanding. Having the good and bad versions of the same code right next to each other was a key for me. They also used slightly different names than the code base demoed in the re:Invent talk that I attended.

While I had never heard of domains, ports, and adapters, I did know about the handlers and repositories used in this newly found repo, which brings me to my first point of understanding: the naming is mostly arbitrary, the use and context boundaries are what matter. Both of these codebases did the same thing, but with slightly different naming. The idea was that a request could come into an adapter, that adapter would then communicate with business logic (domain) through a port. If the business logic needed to initiate communication with anything outside of its boundaries, then it would also communicate through a port to another adapter.

While I was able to start reading and understanding the code in both repos and understanding where the context boundaries lied, I was still slightly confused about why we needed ports at all. Both repos' examples showed ports as simply a passthrough, so why not just have an adapter call the business logic and the business logic call adapters? The answer was something I had seen over and over again in the software engineering world.

SOLID is a commonly used acronym which gives us principles for OOP. The D stands for dependency inversion priciple, which means that code should rely on abstractions not code that implements the functionality. In the context of hexagonal architecture, ports are the abstraction and adapters and business logic is the code that implements the functionality. A port is commonly implemented using an interface, but it is a conceptually a contract between an adapter and business logic for requests and responses similar to an API model.

With my newfound understanding of hexagonal architecture, I looked back at some of my past projects and wanted to see how far off I was from implementing the architecture. It turns out that I was not far off. We can take a peek at a smaller Lambda real quick to get an idea of what I used to have, then I will refactor my code to fit a hexagonal architecture and highlight the changes I make along the way. For a slight bit of context, this code is a Lambda handler for a calendar API that I wrote. This particular endpoint creates an event for a calendar under the path /calendars/{calendarId}/events where {calendarId} is replaced by the user as the ID for a previously created calendar that they would like to modify.

/**
 * File: index.js
 */

const events = require('/opt/database/events');
const { getDateRange } = require('/opt/getDateRange');
const { logger } = require('/opt/logger');

exports.handler = async function (event, context, callback) {
  try {
    const calendarId = event.pathParameters.calendarId;
    const body = JSON.parse(event.body);

    // Create Dates
    let startTime;
    let endTime;
    try {
      startTime = new Date(body.startTime);
      endTime = new Date(body.endTime);
    } catch (err) {
      logger.error('startTime and endTime are not ISO 8601 format.');
      const errorPayload = {
        statusCode: 400,
        body: JSON.stringify({
          errorMessage: 'startTime and endTime must be ISO 8601 timestamps e.g. 1995-12-13T03:24:00Z.',
        }),
      };
      return errorPayload;
    }

    // Check start comes after end
    if (!(startTime < endTime)) {
      logger.error('startTime is not earlier than endTime.');
      const errorPayload = {
        statusCode: 400,
        body: JSON.stringify({
          errorMessage: 'startTime must come before endTime.',
        }),
      };
      return errorPayload;
    }

    const dates = getDateRange(startTime, endTime);
    const eventId = await events.create(calendarId, dates, body);

    const data = {
      statusCode: 200,
      body: JSON.stringify({
        id: eventId,
      }),
    };
    return data;
  } catch (uncaughtError) {
    logger.error(uncaughtError);
    throw uncaughtError;
  }
}

/**
 * File: /opt/database/events.js
 */

const { documentClient } = require('./databaseSession');
const { generateToken } = require('../generateToken');

async function create(calendarId, dates, event) {
  const eventId = generateToken();
  const batchWritePayload = {};
  // Construct batchWritePayload
  await documentClient.batchWrite(batchWritePayload);

  return eventId;
}

module.exports = {
  create,
};

Luckily enough, I already had my database functionality completely separate from my business logic due to a past experience. However, the database calls are the code implementing functionality, so my business logic should be communicating with it through an abstraction (port) not directly. My Lambda's business logic also knows about my data model and that I am using DynamoDB because the getDateRange call would only be used for a NoSQL database. (This is an implementation detail, and I do not believe that it is necessary for understanding the refactor. The main point is that my business logic knows about database-specific implementation details.)

My incoming adapter would be what transforms the Lambda-function-specific call into the parameters needed for my business logic to run and then communicate through a port to my business logic. Currently, there is no incoming adapter or port.

First, let's tackle the incoming adapter and port.

/**
 * File: index.js
 */

const { InputError } = require('/opt/errors');
const {
  GOOD_STATUS_CODE,
  BAD_INPUT_STATUS_CODE,
  SERVER_ERROR_STATUS_CODE,
} = require('/opt/config');
const { logger } = require('/opt/logger');

const { port } = require('./port');

exports.handler = async function (event, context, callback) {
  try {
    const calendarId = event.pathParameters.calendarId;
    const body = JSON.parse(event.body);

    const eventId = await port(calendarId, body);
    const data = {
      statusCode: GOOD_STATUS_CODE,
      body: JSON.stringify({
        id: eventId,
      }),
    };
    return data;
  } catch (err) {
    logger.error(err);
    let statusCode = SERVER_ERROR_STATUS_CODE;
    let message = 'Internal server error';

    if (err instanceof InputError) {
      statusCode = BAD_INPUT_STATUS_CODE;
      message = err.message;
    }

    const errorPayload = {
      statusCode,
      body: JSON.stringify({
        message,
      }),
    };

    return errorPayload;
  }
}

/**
 * File: port.js
 */

const { InputError } = require('/opt/errors');

const { logic } = require('./logic');

exports.port = async function (calendarId, payload) {
  // Validate input format
  try {
    startTime = new Date(payload.startTime);
    endTime = new Date(payload.endTime);
  } catch (err) {
    throw new InputError('startTime and endTime must be ISO 8601 timestamps e.g. 1995-12-13T03:24:00Z.');
  }

  // Construct event based on contract
  const event = {
    title: payload.title,
    startTime: payload.startTime,
    endTime: payload.endTime,
  };

  const eventId = await logic(calendarId, event);
  return eventId;
}

While going about this refactor I decided to leave the Lambda configuration completely intact and only move code, so the incoming adapter will remain as the index.handler. Since this is the incoming adapter all it needs to do is parse out the required parameters for the port, call the port, and then return to the client. That is all it does. The other additions used to clean up this code were the InputError and *_STATUS_CODE constants. I wanted to be able to throw and handle custom errors as well as get rid of "magic numbers."

The port could have been an easy pass-through, but I also wanted to add a bit of validation before throwing the parameters at my business logic. The Date creation validation was present in the old handler. Constructing the event based on my contract with the logic function might be a little overkill since I also have a model set up for this endpoint in my API Gateway but I am going for reusability and proper abstraction. Completely validating and constructing the event for logic ensures that whatever is calling my port abstraction does not matter. This port should take care of any details thus becoming a proper abstraction.

The next chunk of code I looked at was the business logic itself.

/**
 * File: logic.js
 */

const { InputError } = require('/opt/errors');
const { createEvent } = require('/opt/ports');
const { logger } = require('/opt/logger');

/**
 * Business logic
 * @param {string} calendarId
 * @param {Object} event
 * @param {string} event.title
 * @param {string} event.startTime ISO-8601 format
 * @param {string} event.endTime ISO-8601 format
 * @returns {string}
 */

exports.logic = async function (calendarId, event) {
  // Check that start comes before end
  const startTime = new Date(event.startTime);
  const endTime = new Date(event.endTime);
  if (!(startTime < endTime)) {
    throw new InputError('startTime must come before endTime.');
  }

  const eventId = await createEvent(calendarId, event);

  return eventId;
}

Here I could move the Date creation validation out since logic now knows that it will be called with a properly formatted event.startTime and event.endTime. If I were using Typescript, I would have created an interface. Using Javascript means that I need to hold myself a little more accountable and enforce the contract in code on my own. logic only deals with value-specific validation since it can know the contract has been met, which we see with the start and end time comparisons. It also no longer deals with the database implementation details, but rather passes the relevant information on and lets the database adapter deal with constructing the data model.

Lastly, I refactored my outgoing/database port and adapter.

/**
 * File: /opt/ports.js
 */

const events = require('/opt/database/events');
const { getDateRange } = require('/opt/getDateRange');

async function createEvent(calendarId, event) {
  const dates = getDateRange(startTime, endTime);
  const eventId = await events.create(calendarId, dates, event);
  return eventId;
}

module.exports = {
  createEvent,
};

/**
 * File: /opt/database/events.js
 */

const { documentClient } = require('./databaseSession');
const { generateToken } = require('../generateToken');

async function create(calendarId, dates, event) {
  const eventId = generateToken();
  const batchWritePayload = {};
  // Construct batchWritePayload
  await documentClient.batchWrite(batchWritePayload);

  return eventId;
}

module.exports = {
  create,
};

Notice anything? The code in /opt/database/events.js did not have the change. As I said, I luckily had the database functionality properly abstracted, the problem was that my business logic called the database functionality instead of an abstraction (the database port) to the functionality. This made the refactor for my database port and adapter simple. All I needed to do was move the getDateRange call outside of my business logic into my port and have my port call the database adapter.

Well, there we have it. I have been wanting to implement a hexagonal architecture, especially in Lambda functions, since I attended that talk at re:Invent a few months ago. After finding another few key resources on implementation, I am now much more confident that I understand the details and boundaries. As I have said in plenty of my other posts, I use kaizen in many aspects of my life. This is only my first pass at implementing a hexagonal architecture so take it with a grain of salt. I have implemented everything to the best of my knowledge but I am sure I will learn new and better ways in the future. For now, I hope this has helped someone else jump to implementation understanding quicker. Happy coding.

How To Write AWS CDK Tests

Thomas Step — Sat, 22 Jan 2022 09:00:09 +0000

Writing tests is a necessary nuisance. I would much rather spend my time writing functionality than writing code to verify what I wrote but mistakes happen and backward compatibility needs to be preserved. After I have finished writing tests I always feel much better about a codebase. For some reason, writing that code is simply not as enjoyable. That is one of the reasons why I dragged my feet for so long on writing tests for crow-api.

Another big reason I took so long to write tests is that testing infrastructure is a fairly new thing. How can we test a VM's configuration that was spun up by a different team in a data center that is homebrewed? Those scripts would also need to be tailored and probably not worth the effort. It would most likely be easier to write E2E or integration tests after code has been deployed onto the servers. I did not expect to find many resources online about testing CDK Constructs and Stacks simply because I figured it was new enough.

My assumption of a lack of documentation and examples was not too far off, but like the inspiration for many of my posts, I would like to contribute something back that I found missing.

Whenever we start a new CDK app, the tool automatically creates a test/ folder, installs jest, and gives us a little bit of boilerplate code. AWS obviously wants us to write tests. I did find a serious lack of official documentation around the tools the CDK has to write tests. The aws-cdk-lib.assertions module is about all I could find (someone please point me in the correct direction if I overlooked something). Even though that documentation ended up being about all I needed, it was still discouraging to not find much else.

The boilerplate code given to us with a fresh CDK app looks like the following.

const app = new cdk.App();
  // WHEN
const stack = new ScheduleApi.ScheduleApiStack(app, 'MyTestStack');
  // THEN
const template = Template.fromStack(stack);

template.hasResourceProperties('AWS::SQS::Queue', {
  VisibilityTimeout: 300
});

The first line should look familiar (const app = new cdk.App();) because it is the same as initializing an app whenever we want to deploy something. The stack creation is the same as well, const stack = new ScheduleApi.ScheduleApiStack(app, 'MyTestStack');. Once we get to const template = Template.fromStack(stack); things start diverging. What I say from here on out is based on my best knowledge at the time of writing. It is possible that I am not following best practices, but I can not find anything about best practices.

It seems to me like the best way to test CDK code is to synthesize the code into CloudFormation stacks then run assertions against the huge string that is the template. This is what the boilerplate code that the CDK generates shows and the aws-cdk-lib.assertions module shows no other way of testing anything. This means that the props given to the stacks used in tests should be identical to the props given to the stacks being deployed to correctly test configuration.

The Template created from running Template.fromStack() can then be queried for resources, mappings, and outputs using the Template class's methods. The methods starting with has will throw errors if the corresponding resource in the template is not found, and the methods starting with find will return the resources themselves as well as their logical IDs.

I am going to show some examples from the tests that I wrote for crow-api. (These tests might change but the exact commit's file I am referencing is here.)

One of the first and most straightforward tests I wrote looks like the following.

template.hasResourceProperties('AWS::ApiGateway::RestApi', {
  Name: 'testing-crow-api',
});

This call is simply asserting that the template contains a RestApi resource with the Name property set to testing-crow-api. Notice that the Name property is referencing the naming from the CloudFormation template not the prop from the CDK code (restApiName).

The next tests that I wrote started getting more complicated. I wanted to start testing that the API Gateway Resources were pointing towards the correct parents. With CDK this is simple, but there is more going on under the covers to make the CloudFormation work. A resource's logical ID is referenced in the CloudFormation template, but with CDK code, we do not interface with logical IDs. The question then turns into a matter of teasing the logical ID out of the CDK stack or the Template. For this first example, I was able to grab the ID from the CDK stack.

function getLogicalId(stack: cdk.Stack, resource: cdk.IResource) {
  return stack.getLogicalId(resource.node.findChild('Resource') as cdk.CfnElement);
}

const restApiLogicalId = getLogicalId(stack, stack.api.gateway);

template.hasResourceProperties('AWS::ApiGateway::Resource', {
  ParentId: {
    'Fn::GetAtt': [
      restApiLogicalId,
      'RootResourceId',
    ],
  },
  PathPart: 'v1',
  RestApiId: {
    Ref: restApiLogicalId,
  },
});

The next example become slightly more complicated. I first needed to use Template's findResources using properties unique to a specific resource, then grab the logical ID from the result of the findResources call, and finally use the logical ID in a hasResourceProperties call.

function logicalIdFromResource(resource: any) {
  try {
    const resKeys = Object.keys(resource);
    if (resKeys.length !== 1) {
      throw new Error('Resource is not unique.');
    }
    const [logicalId] = resKeys;
    return logicalId;
  } catch (err) {
    console.log(resource);
    throw err;
  }
}

const authorsPath = template.findResources('AWS::ApiGateway::Resource', {
  Properties: {
    PathPart: path,
  },
});
const v1AuthorsGetLambda = template.findResources('AWS::Lambda::Function', {
  Properties: {
    TracingConfig: {
      Mode: 'Active',
    },
  },
});

const authorsLogicalId = logicalIdFromResource(authorsPath);
const v1AuthorsGetLambdaLogicalId = logicalIdFromResource(v1AuthorsGetLambda);

template.hasResourceProperties('AWS::ApiGateway::Method', {
  HttpMethod: 'GET',
  ResourceId: {
    Ref: authorsLogicalId,
  },
  RestApiId: {
    Ref: restApiLogicalId,
  },
  Integration: {
    Uri: {
      'Fn::Join': [
        '',
        [
          'arn:',
          { Ref: 'AWS::Partition' },
          ':apigateway:',
          { Ref: 'AWS::Region' },
          ':lambda:path/2015-03-31/functions/',
          {
            'Fn::GetAtt': [
              v1AuthorsGetLambdaLogicalId,
              'Arn',
            ],
          },
          '/invocations',
        ],
      ],
    },
  },
});

There are some changes in the example code compared to the permalink but the idea is the same.

While the functions expose to use might not be the most comprehensive compared to what we might want to do, I was at least able to figure out some way of testing what I wanted to. I hope that my thoughts and examples have helped someone along their way. These examples are what the CDK authors intended to the best of my knowledge, but if I learn something different later, I will either update this post or make a follow-up post. For now, happy coding!

How To Use The DynamoDB Document Client

Thomas Step — Thu, 20 Jan 2022 09:00:09 +0000

In my Guide To Building With Serverless AWS chapter about DynamoDB, I mention using the Document Client to communicate with DynamoDB instead of some other alternatives like the lower-level client. In this post I would like to discuss and give code examples for some of the specific DynamoDB APIs. When I started using the Document Client I had a little bit of trouble translating the lower level client and APIs calls into Document Client calls but after seeing some examples and reading how I translate the payloads, I hope that all of that will be much easier for you. As a side note, what I will be showing in this post uses the AWS Javascript SDK v3. While the lower versions (and potentially future versions) will most likely look similar, this code is specifically for version 3.

Transforming a payload to include data types for DynamoDB calls can be tedious. For the sake of my own understanding, I have used the lower level SDK client to manually code the DynamoDB types, and having done that, I much prefer the Document Client for ease of use. The Document Client uses native Javascript types to conclude the data type for DynamoDB and then transforms the payload for us.

The Document Client in the SDK is a thin layer on top of the lower-level/normal client that does the transformation. It is available in the @aws-sdk/lib-dynamodb package but the @aws-sdk/client-dynamodb is still required.

Before I start showing the code examples I want to make note of a couple of assumptions that the code will be making.

First is that the DynamoDB table is created and the name of the table is available through an environment variable called TABLE_NAME. In the code examples, you will see TableName: process.env.TABLE_NAME.

The code also assumes that the table is configured to use a partition key with the name id and a sort key with the name secondaryId. This will be seen in examples looking like the following.

Item: {
  id: 'myId',
  secondaryId: 'mySecondaryId',
},

Both the partition and sort key names can be configured to whatever you would like. The id and secondaryId keys in the Item object would then need to be changed accordingly. I suggest not hardcoding either of these values like I am in the examples. For the partition key (id), I suggest using something like a generated UUID or a value unique to that item's data. The sort key (secondaryId) value will be more dependent on your data model, but some suggestions could be using a configured value pulled from a module, an environment variable, or a value specific to that item's data.

If you do not know much about data modeling, I highly suggest learning more and determining how to model your data before using DynamoDB. I lightly discuss modeling in this post but there are other great resources out there from Rick Houlihan (find his re:Invent talks on YouTube) and Alex DeBrie.

Remember that these are simply examples and you can change your code however you would like. I simply want to show what the calls could look like.

Table of Contents:

Creating the Document Client
PutItem
BatchWriteItem
GetItem
Query
UpdateItem
DeleteItem

Creating the Document Client

Creating the Document Client first involves creating a normal client and then initializing the Document Client with the normal client. I like to use a single module and then export the client for use in subsequent modules.

const { DynamoDBClient } = require('@aws-sdk/client-dynamodb');
const { DynamoDBDocument } = require('@aws-sdk/lib-dynamodb');

const client = new DynamoDBClient({
  region: process.env.AWS_REGION,
});

const documentClient = DynamoDBDocument.from(client);

module.exports = {
  documentClient,
};

`PutItem`

AWS Documentation

Creating a new item in DynamoDB involves either a PutItem or BatchWriteItem API call. An Item is passed in and the partition and sort key need to be present. Any additional values in the Item will be written as attributes.

const myItem = {
  hello: 'world',
  value: 3,
};

await documentClient.put({
  TableName: process.env.TABLE_NAME,
  Item: {
    id: 'myUniqueId',
    secondaryId: 'secondaryId',
    ...myItem,
  },
});

`BatchWriteItem`

AWS Documentation

This API was slightly more confusing to me. The top level of the payload contains a RequestItems key. The values for RequestItems are key-value pairs where the keys are table names and the values are arrays of requests being sent to each table. The requests in the arrays are also key-value pairs where the keys are either PutRequest or DeleteRequest and the values are either Item (similar to PutItem) or Key (similar to DeleteItem), respectively.

const myItems = [
  {
    hello: 'world',
    value: 3,
  },
  {
    hello: 'mars',
    value: 5,
  },
  {
    hello: 'saturn',
    value: 0,
    rings: true,
  },
];

const requests = [];
myItems.forEach((myItem) => {
  requests.push({
    PutRequest: {
      Item: {
        // Keep in mind that each partition and sort key will need
        // to be unique for each item
        id: 'myUniqueId',
        secondaryId: 'secondaryId',
        ...myItem,
      },
    },
    // Similar payload for DeleteRequest
    // DeleteRequest: {
    //   Key: {
    //     id: 'myUniqueId',
    //     secondaryId: 'secondaryId',
    //   },
    // },
  });
});
const batchWritePayload = {
  RequestItems: {
    [process.env.TABLE_NAME]: requests,
  },
};
await documentClient.batchWrite(batchWritePayload);

`GetItem`

AWS Documentation

Retrieving an item is fairly simple. We pass in the partition and sort key and get back a response containing some metadata and the Item.

const res = await documentClient.get({
  TableName: process.env.TABLE_NAME,
  Key: {
    id: 'myUniqueId',
    secondaryId: 'secondaryId',
  },
});
const item = res.Item;

`Query`

AWS Documentation

There are some additional fields in query that might seem confusing at first. KeyConditionExpression for a query needs to define the value of the partition key and can optionally specify a comparison with a value for the sort key. In my example, the sort key is simply compared with equals (=) to the value (secondaryId) but I could have just as easily used any of the supported key condition expressions.

The next confusing parts are the ExpressionAttributeNames and ExpressionAttributeValues. Each is a way to dynamically reference a value in the KeyConditionExpression without needing to do string substitution. With the Document Client, this is straightforward, but for some context, without the Document Client, we would have needed to add data types to the values in ExpressionAttributeValues. The strings to be referenced by ExpressionAttributeNames should be prefixed with a hash sign (#) and ExpressionAttributeValues should be prefixed with a colon (:). The prefix characters are not a convention, they are explicitly stated in the documentation.

After results have been gathered from the table, we can further refine the results by making DynamoDB filter what has been queried using the FilterExpression. This is a good place to go over results with a fine-toothed comb, but just know that a query consumes read requests units based on the results returned by the KeyConditionExpression, not only the results returned after further refinement using the FilterExpression. Again, this is a data modeling problem. The FilterExpression uses the same syntax as the KeyConditionExpression but can be written for any attribute in the Item.

Lastly, the ProjectionExpression is a comma-separated list of attributes that should be retrieved by DynamoDB and returned as a result of the query. If we were querying the items from the BatchWriteItem(#batchwriteitem) example, we would on retrieve hello and value but not rings.

We get back a response containing some metadata and the Items we match the KeyConditionExpression and FilterExpression.

const res = await documentClient.query({
  TableName: process.env.TABLE_NAME,
  KeyConditionExpression: 'id = :partitionKey AND secondaryId = :sortKey',
  ExpressionAttributeNames: {
    '#valueName': 'value',
  },
  ExpressionAttributeValues: {
    ':partitionKey': 'myUniqueId',
    ':sortKey': 'secondaryId',
    ':minValue': 3,
  },
  FilterExpression: '#valueName >= :minValue',
  ProjectionExpression: 'hello, value',
});
const items = res.Items;

`UpdateItem`

AWS Documentation

Updating an item is fairly simple. We pass in the partition and sort key as the Key's value and any updates to attributes as key-value pairs at the top level of the payload.

const updates = {
  hello: 'moon',
  value: 300,
};

await documentClient.update({
  TableName: process.env.TABLE_NAME,
  Key: {
    id: 'myUniqueId',
    secondaryId: 'secondaryId',
  },
  ...updates,
});

`DeleteItem`

AWS Documentation

Deleting an item in DynamoDB involves either a DeleteItem or BatchWriteItem API call. Deleting is as simple as retrieving, we pass in the item's partition and sort key and DynamoDB handles the rest.

await documentClient.delete({
  TableName: process.env.TABLE_NAME,
  Key: {
    id: 'myUniqueId',
    secondaryId: 'secondaryId',
  },
});

API Gateway Models

Thomas Step — Wed, 19 Jan 2022 09:00:09 +0000

As almost anyone in software knows, iteration is key. Kaizen has proved itself over and over again, and I like to apply the same concept to my projects. My earlier projects started fairly rudimentary and simple but have grown more complex with larger scopes over time. One of the areas in particular that I have set out to consciously improve is my infrastructure. For some reason, AWS and infrastructure have continued to be fun and of interest to me over the years.

I have written many APIs and configured many API Gateways to go with them. What once started as CloudFormation templates evolved into CDK code and my CDK Construct which makes iteration even easier. The latest improvement that I have taken to is using API Gateway models to define my endpoints' schemas. Using models is something that I have wanted to work on for a while now, but I had not taken the time to properly learn what they are or how they can be configured up until now.

Models are a way to define the accepted body using JSON and a syntax called JSON schema. That is about as complicated as they are. Once you have a schema in mind, the syntax is also fairly straightforward. I will show an example of a JSON schema that I wrote for a CDK application and then explain the different pieces.

{
  schema: apigateway.JsonSchemaVersion.DRAFT4,
  title: 'mySchema',
  type: apigateway.JsonSchemaType.OBJECT,
  required: ['startTime', 'endTime'],
  properties: {
    startTime: {
      type: apigateway.JsonSchemaType.STRING,
      format: 'date-time',
    },
    endTime: {
      type: apigateway.JsonSchemaType.STRING,
      format: 'date-time',
    },
    title: { type: apigateway.JsonSchemaType.STRING },
  },
  additionalProperties: false,
}

To start, we need to define the JSON Schema version we are using. At the time of my writing this, draft 4 is the current most up-to-date version. Then comes the title. The other keys are starting to define the schema. Since my endpoint was a POST that accepted a JSON object, I needed to define my root property type as apigateway.JsonSchemaType.OBJECT. Since my root type is an object, I can then use the properties key to define what the object's body looks like. The "type" for each property in my parent object is also a JSON Schema, so this is a sort of recursive syntax.

My schema accepts three and only three potential properties: startTime, endTime, and title. The additionalProperties: false piece tells API Gateway to return a 400 error if the caller supplies keys outside of what I have defined. The required: ['startTime', 'endTime'] piece also tells API Gateway to return a 400 error if the caller does not supply either of those required properties. For the time being, the additionalProperities and required properties are the only behavioral properties I needed.

Once the root object's properties are configured, we can move on to each of the children. The type and format for startTime and endTime are defined so that a small amount of pre-validation can occur. This eases the burden on my code for light validation so I can focus solely on deeper validation. This brings me to the main reason I wanted to start using models: API Gateway validates payloads for me and rejects those that do not conform without me spending a cent. In a similar fashion to authorizers, AWS will front the bill of validating that a call is "up to snuff" before spinning up a Lambda and charging me for it.

Once we have a JSON Schema written up, configuring it becomes a matter of applying the JSON to the correct property on the API Gateway Method and configuring a Request Validator to validate the body. I wish I would have started using models earlier because they save my time writing code to validate payloads before executing business logic on them. I plan on going back to some of my older projects and APIs to refactor in using models. I hope this convinced you to start using models, and if you have any further questions, including implementation-specific questions, feel free to reach out to me.

AWS CDK Example For API Gateway And SNS Integration

Thomas Step — Mon, 17 Jan 2022 09:00:09 +0000

TL;DR: The CDK code I used to directly integrate API Gateway and SNS is here.

While working on one of my more recent projects, I came across the need to create a completely asynchronous API path in API Gateway. I have used SNS in the past for asynchronous communication but in prior scenarios, I published messages to SNS using the SDK in a Lambda function. This time around there was no execution needed before calling SNS which made a good use case for an API Gateway integration directly with SNS.

Integrating API Gateway with other AWS services is possible, but I have read articles online claiming that the integration is not as easy as it could or should be. It seemed as if there were difficulties because the API Gateway integration more or less needs to be hardcoded with the proper API calls to the AWS service it is being integrated with. As someone who has configured this type of integration, I can confidently say that those articles were correct. I repeatedly needed to reference AWS documentation. The trickiest parts were needing a separate IAM Role (something I have not had to worry about since moving to the AWS CDK), the path and action properties, getting the requestParameters and requestTemplates to match up and work, and making the requestTemplates with Velocity work correctly because I wanted to pass in JSON.

The AWS CDK takes care of most IAM provisioning needed for resources to correctly operate. If actions are not configured out of the box, there are normally easy calls to make to get resources set up to complete their tasks. For some reason, API Gateway integrations do not come with that support out of the box. This problem was trivial to fix though since it only involved creating a Role resource and granting it the correct permissions. Either way, I think that this could be one area that the CDK could improve upon in the future.

const apiGatewayRole = new iam.Role(this, 'integration-role', {
  assumedBy: new iam.ServicePrincipal('apigateway.amazonaws.com')
});
topic.grantPublish(apiGatewayRole);

Configuring the correct path and action prop was the next big hurdle that I had to cross. According to the CDK docs for AwsIntegrationProps, the action is "the AWS action to perform in the integration" and the path is "the path to use for path-base APIs." None of that information helped me out nor did it tell me that one of the two props is necessary for correct integration. I started with action: 'Publish. Nothing. So I started looking around the internet for answers. If it had not been for Alex DeBrie and his post about API Gateway service proxies I probably would have taken a week to figure this out through trial and error. If you read through that post, you will notice that there is no mention of path or action used as a configuration item, but he does include Action=Publish in the request template and the integration's URI includes sns:path//. From those two bits, I pieced together something that works. Keep in mind that I am not sure what I did is correct, all I know is that it works. When API Gateway constructs the URL for SNS the double slash (//) is in the path, which looks ugly and is probably an indication that something is not optimally configured. For now, this will work, but I plan on revisiting it in the future.

Next on the list of hurdles was correctly configuring the requestParameters and requestTemplates. However, this was less a lack of documentation and more so a lack of me reading said documentation. The requestParameters are input to the integration, and in my case, I needed to set the Content-Type header to application/x-www-form-urlencoded. For some reason, it did not click with me until too late what requestParameters were for and that they should not be the same as the application/json that I used as the key in requestTemplates. Figuring out requestTemplates was easier for me. It is a mapping between a Content-Type header and a template used to transform the request into a payload for the backend. I think what tripped me up is that I thought the value used in requestParameters needed to be the same as the key used in requestTemplates, which is not the case.

Even though I said that figuring out the requestTemplates was slightly easier for me, figuring out the Velocity template syntax was difficult. I knew that I should be able to pass in JSON to SNS's Publish action, but since I had previously only used SNS from the SDK, I did not know how to translate what the SDK did into Velocity. My API had IDs in its path that I wanted to include in the payload to SNS. The biggest realization for myself was that I needed to manually write out and escape the JSON where I had previously thought that Velocity would do that for me. After I figured out that I needed to hardcode the JSON and inject the parameters where they were the values, I was off to the races.

Message=$util.urlEncode('{"someParam":"')$input.params('paramName')$util.urlEncode('"}')

Despite all the trouble that integrating API Gateway directly with SNS gave me, I am glad I went through it. This is a valuable integration and configuration to have set up. I have already implemented it more than once. I am sure that I will use it more in the future and iterate on top of what I already know to make the configuration run even smoother.

I Wrote a Guide to Serverless AWS

Thomas Step — Fri, 17 Sep 2021 00:00:00 +0000

EDIT: The guide has been released!

To newcomers, AWS might seem like a big confusing jumble of buzzwords and weird service names. When I started my journey into learning AWS and cloud engineering, I thought the same thing. I started my journey as a software engineer with some networking and traditional operations experience, but the cloud and its constituents were still very much foreign to me. Fast forward two years and I have learned more than I ever thought I would about AWS. I have worked with and weeded through a lot of the AWS services, and I have somewhat strong opinions about how to build workloads and which services are worth my time.

My personal preference (as is probably apparent by my previous AWS blog posts) is to use serverless technologies because they are quick to develop on and the pricing model suits new applications and startups well. The guide that I wrote is focused on the core serverless technologies that I have personally used to create applications. My goal throughout writing this guide was to provide a framework of AWS services to focus on for newcomers, the fundamental concepts of those services, and the integrations between those services that can create fully-fledged, production-ready serverless applications. This is what I wish existed when I first started learning AWS.

The last month and a half of my free time have been dedicated to writing down and organizing this information, and I feel much better getting it out of my head and into the world. I will host a free version on my website with each chapter being a new blog post, and I will offer a PDF version with a bit of additional content through GumRoad. The blog version will be out first with the full eBook version out soon thereafter. Once it is out feel free to send me feedback about what you did and did not like or if there are any areas you would have liked me to touch on more. I look forward to hearing from you!

Destructuring Objects in Javascript

Thomas Step — Thu, 12 Aug 2021 00:00:00 +0000

I first learned about destructuring objects a few months into learning about Javascript. I might have been late to the party but I am glad I joined. Destructuring is one of my favorite features of the language because it looks and feels so clean and succinct. The official definition is unpacking values from array or properties from objects into distinct variables. Today, I only want to go over a few examples with objects, but maybe I will make another post with examples for destructuring arrays.

The first and easiest to understand example is simple value extraction. Given the key names of a certain object, we can pull out the values of those keys for operation.

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
  numberValue: 42,
};

const {
  hello,
  other,
  numberValue,
} = initialObject;

console.log(hello); // 'world'
console.log(other); // 'iamavalue'
console.log(numberValue); // 42

Seems simple enough. The syntax also allows us to give those new consts different names instead of the default names corresponding to the keys. Providing new names might look a bit weird at first but starts to become second nature.

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
  numberValue: 42,
};

const {
  hello: planet,
  other: whatAmI,
  numberValue: randomNumber,
} = initialObject;

console.log(planet); // 'world'
console.log(whatAmI); // 'iamavalue'
console.log(randomNumber); // 42

The key’s name is on the left of the colon (:) and the new variable’s name is on the right. Destructuring this way sometimes makes more sense if the key names are generic or if multiple objects with the same keys are being destructured since the const variable names would otherwise conflict. What if our assumptions about the object’s contents are false? At the top level, the value is returned as undefined, but there is a way to provide default values.

let conditionalObject = {};

if (Math.random() > 0.5) {
  conditionalObject = {
    hello: 'world',
    other: 'iamavalue',
    numberValue: 42,
  };
}

const {
  hello,
  other = 'defaultOtherString',
  numberValue: randomNumber = 3,
} = conditionalObject;

console.log(hello); // 'world' || undefined
console.log(other); // 'iamavalue' || 'defaultOtherString'
console.log(randomNumber); // 42 || 3

Let’s say you have an object that is conditionally populated with properties; attempting to destructure those properties could be a hit or a miss. The example above shows what happens if we do not provide a default value (variable is undefined), how to provide a default value (other = 'defaultOtherString'), and how to provide a default value to a renamed property (numberValue: randomNumber = 3). The syntax for providing a default value looks the same as providing a default value to a function, so we add an equals sign (=) and the default value to the right side. The syntax for providing a default value to a renamed variable name needs to follow a certain order, but the order is intuitive. First, we provide a new variable name with the colon (:) syntax like we saw before, then we tack on the default value with its syntax.

What if we want to overwrite the value of an existing variable with a value retrieved while destructuring? That is also possible by assigning without declaring. The syntax does look a little different, but it is an alternative method of destructuring that can be put to good use. I have used something similar to this in the past to override existing variables that were previously defined as a way of using default values. This also works well for destructuring in a condition but declaring the variable with let outside of the condition’s block.

let hello;
let whatAmI;
let randomNumber;

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
};

({
  hello,
  other: whatAmI,
  numberValue: randomNumber = 88,
} = initialObject);

console.log(hello); // 'world'
console.log(whatAmI); // 'iamavalue'
console.log(randomNumber); // 88

Putting parentheses around the destructuring statement executes this code as a statement instead of seeing the { ... } on the left of the equals (=) as a block. It is also worth noting that a semicolon (;) needs to be placed after the preceding statement or the destructuring will be viewed as trying to call a function. The previous syntax additions that I have examples for are also available: renaming variables and providing default values.

Destructuring also works without statically defining the key names we want to pull out. Using similar square bracket ([...]) notation as we would for accessing an object’s values, we can destructure using dynamic values.

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
  numberValue: 42,
};

let firstProperty = 'hello';
let secondProperty = 'other';

if (Math.random() > 0.5) {
  firstProperty = 'numberValue';
  secondProperty = 'hello';
}

const {

  [secondProperty]: second = 'defaultValue',
} = initialObject;

console.log(first);
console.log(second);
/**
 * Either 'world' and 'iamavalue'
 * or 42 and 'world'
 */

Destructuring this way requires us to provide new variables names and optionally provide default values (in this example the default behavior would not be encountered, I simply wanted to show that it is an option). CHECK THAT THIS IS TRUE.

So far I have only shown examples with a flat object, but what about destructuring more complex nested objects?

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
  numberValue: 42,
  nestedObject: {
    nestedKey: 'nestedValue',
    ECHO: {
      ECHo: {
        ECho: {
          Echo: 'echo',
        },
      },
    },
    unusedNestedKey: 901,
  },
};

const {
  hello,
  nestedObject: {
    nestedKey,
    ECHO: {
      ECHo: partialEchoObject
    }
  },
} = initialObject;

console.log(hello); // 'world'
console.log(nestedKey); // 'nestedValue'
console.log(partialEchoObject); // ECho: { Echo: 'echo' }

As we can see, it is fairly straightforward to destructure nested objects. One thing to note is that we do not have to destructure until we hit the end of the nested objects, which I demonstrated with the nested echoing objects.

One final use for this syntax that I wanted to highlight is copying an object but “removing” certain keys from the copy. Instead of using delete, we can use a combination of destructuring and spreading to pull out those unwanted keys.

const initialObject = {
  hello: 'world',
  other: 'iamavalue',
  numberValue: 42,
};

const {
  hello,
  ...rest
} = initialObject;

console.log(hello); // 'world'
console.log(rest); // { other: 'iamavalue', numberValue: 42 }

The value placed in the rest variable, which is spread, is the initialObject minus the properties that have already been used in the destructuring (hello in this example). I love finding excuses to use this nifty pattern. Note that ...rest does not have a comma (,) after it, this is done on purpose.

These are all examples that I have personally used to destructure objects in Javascript. Other patterns exist that I have not used, but I believe these are some of the more common ones. Arrays can also be destructured, but that is a topic for another day. Hopefully, this gave you some context and new tools to add to your kit.

Writing Asynchronous Lambda Function with Node

Thomas Step — Tue, 10 Aug 2021 00:00:00 +0000

When I first started diving into cloud engineering and learning about microservices, I kept hearing about “event-driven architecture”. I understood how to write asynchronous code in Node.js, and I understood how to write code for Lambda functions. But I caught myself awaiting async code before I returned from a Lambda handler without using the result of that awaited function. Something felt off, but I did not know what else to do.

The way that async Lambda handlers work (at least with Node.js) is by running code and then “finishing execution” whenever the handler returns. The Lambda might finish execution, but if an async function was still running in the background (waiting on an API call for example) then that asynchronous execution may or may not finish before the Lambda itself shuts down. There are loads of explanations and documentation out there about the Node.js event loop and how it works with AWS Lambda. That documentation has better explanations of what is happening than I am prepared to offer at this time. The quick and dirty solution that most people use (myself included for a long time) is to simply await all async functions before returning from the Lambda handler. However, there is a better way to handle asynchronous execution. After all, we are talking about the cloud and Amazon, and surely they have run into this problem before.

My experience with async Node.js code and Lambda was limited to the above solution until I started working on a side project. I wanted to focus on speed and lower latency, so I naturally needed to learn more about writing asynchronous systems and code. After reading and experimenting I found a solution that was staring me right in the face this entire time: AWS SNS. For some reason, I had never completely connected the dots before, but SNS allowed me to call an async function without awaiting the result to make sure that execution finished. I had interacted with SNS before, but I failed to think of it as a convenient way to run code in the background without slowing down a Lambda function.

One thing that I knew but never completely put into practice was the fact that I was writing Lambda functions. Instead of writing an async function and running it in my critical path’s Lambda, I could take that same async function, deploy it as its own Lambda function, create an SNS topic, make that Lambda a consumer of the SNS topic, and call my function asynchronously by publishing to SNS from my critical path.

Calling the SNS API in my main Lambda is faster than calling the async function and awaiting it, so my main Lambda is allowed to return to the user with minimum latency. My async function is then triggered by SNS to run without needing to worry about how long it takes. It’s a win-win.

Lambda is great. SNS is great. For whatever reason, it took me a while to realize the power that combining Lambda and SNS provides. Learn from my mistake and enjoy this combination made in heaven. Use it. Love it. Create asynchronous, event-driven systems.

Centering a Div With Tailwind CSS

Thomas Step — Fri, 23 Jul 2021 00:00:00 +0000

Somehow centering a div still seems to trouble people. I have mostly been working with Tailwind CSS lately and I wanted to quickly share how I center a div with Tailwind. I’ll first show the examples and then describe what is happening.

<div class="flex flex-row min-h-screen justify-center items-center">
  I am centered
</div>

The same classes work with either flex-row or flex-col, which set the flexbox’s main axis either horizontally or vertically, respectively. Setting the height with min-h-screen is just an easy way to take up an entire screen’s view. The final two classes are where I needed to learn a tiny amount of CSS.

I played around with justification and alignment far too long before I finally looked into what they affect. justify-content refers to how the content should be positioned along the main axis, while align-content refers to how the content should be positioned along the cross axis. Justifying content and aligning content with Tailwind are simple one-to-one mappings between their classes and the actual CSS, so once I understood how the CSS worked, I understood how the utility worked.

A Guide To Using JWT In Javascript

Thomas Step — Wed, 21 Jul 2021 00:00:00 +0000

JWTs are a popular form of authenticating users for web applications. I won’t get into the debate about whether or not we should be using JWTs for this purpose because that is a different discussion entirely. There are use cases that are better suited for using JWTs than others, and I am going to expect that you have landed here after already determining that JWTs are a good fit for what you hope to accomplish.

JWTs are a convenient way for us to store a small bit of public information generated by a server acting as the source of truth to be distributed to potentially anything. The server that generates and signs the JWT has access to a private key while the public key will need to be distributed to servers consuming the JWTs. By using asymmetrical keys we can ensure that everyone can read the information contained in the JWT while making sure that only the authentication server can sign those JWTs.

For simplicity, I will assume that the intent of signing and verifying JWTs is for authentication purposes. While JWTs can be used for other reasons, I imagine authentication is the widest used case and what anyone who found this article is planning on doing. I am also going to assume that you have none of this set up yet. I will walk you through the following: creating public and private keys, creating a JWKS, signing a JWT, and finally verifying a JWT.

One prerequisite for all of this is the npm package jose

Creating Public and Private Keys

I have seen a small amount of debate out on the interwebs about what algorithms should be used for creating your keys. From my point of view (and as of this writing) the best way to do this is using RSA. The JWA alg for this is RS256. There are two ways of generating these key pairs: using the command line and using jose. My personal preference is to use jose, but the command line tools are just as effective.

To generate the key pairs using the command line run the following commands.

ssh-keygen -t rsa -b 4096 -m PEM -f jwt.key -N ""
openssl rsa -in jwt.key -pubout -outform PEM -out jwt.key.pub
cat jwt.key
cat jwt.key.pub

To generate the key pairs using jose run the following script or some variation of them. It would be beneficial to store the produced strings in files.

const { generateKeyPair } = require('jose/util/generate_key_pair');

(async () => {
  const { publicKey, privateKey } = await generateKeyPair('RS256');
  // https://nodejs.org/api/crypto.html#crypto_class_keyobject
  const publicKeyString = publicKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  const privateKeyString = privateKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  console.log(publicKeyString);
  console.log(privateKeyString);
})();

Both of these methods provide the same outcome. They generate RSA asymmetric keys. These are what allow us to properly sign a JWT.

The private key must not be given out to anything other than your authentication server. The private key is used to sign the JWTs and the JWT consumers use the public key to verify that the JWT came from our auth server, so if anyone else gains access to it, then they can pretend that they are our authentication server. This is obviously a no-no.

The way this all works is that the private key is used to encrypt the JWT. The public key can then be used to unencrypt the JWT. A public key can only unencrypt what its corresponding private key encrypted. No other public key can unencrypt the JWT. Conversely, no other private key can encrypt the same way that our new private key can. This means that whenever our new public key is used to successfully verify a JWT, we gain the knowledge that our authentication server approved the JWT’s authenticity from the start.

This is the premise of asymmetric encryption. While it’s not necessary to know anything more than what I outlined, it would not hurt to obtain a deeper understanding of this concept. Asymmetric encryption is a deep rabbit hole to go down though so learn at your own pace.

Back to the task at hand. As you might have guessed by now, the public key should be given out. This allows anyone to verify that the JWT given to them did in fact come from our authentication server. That’s a good thing. While it is valid to give out the public key in the form of a file (like what we just created) there is another type of “JSON Web Thing” that allows us to distribute the public key at scale: JWKS.

Creating a JWKS

JWKS stands for JSON Web Key Set and they are a convenient way to distribute public keys. Here is an example of a JWKS in the wild. The idea here is that JWKSs can be hosted as a simple JSON file to then be accessed by anything using a corresponding JWT. Building them off of public keys in jose is just as easy as creating the initial key pairs.

If the public key is in a file (if it was created using the command line), then the file will need to be read in, converted to the Node crypto library’s representation of a key, and converted using jose.

const fs = require('fs');
const path = require('path');
const crypto = require('crypto');
const { fromKeyLike } = require('jose/jwk/from_key_like');

const pubKeyPath = path.resolve(__dirname, 'path/to/jwt.key.pub');
const pubKey = fs.readFileSync(pubKeyPath, 'utf8');
const cryptoPublicKey = crypto.createPublicKey(pubKey);
fromKeyLike(cryptoPublicKey).then((publicJwk) => console.log(publicJwk));

We can also create these directly after generating the public key from jose.

const { generateKeyPair } = require('jose/util/generate_key_pair');
const { fromKeyLike } = require('jose/jwk/from_key_like');

(async () => {
  const { publicKey, privateKey } = await generateKeyPair('RS256');
  // https://nodejs.org/api/crypto.html#crypto_class_keyobject
  const publicKeyString = publicKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  const privateKeyString = privateKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  console.log(publicKeyString);
  console.log(privateKeyString);

  const publicJwk = await fromKeyLike(publicKey);
  console.log(publicJwk);
})();

In both cases, the output should look similar to the following JSON Web Key.

{

  "kty": "RSA",
  "n": "wi5rLEy1U7m8rU8bQn2GeJ_g_XisJesbzI0N0QbYF3BNaEuQUwXOnh2ME8bOyKBrpXLik5AvljKp8HjwKG1x456kueJJoullYEBtrSRydnNaOQmUno1GQEcreCnRBZzodi9kw0YgsQvEfyxGwxI1NYSS8mdCSgT_BjOw5veHFfK-kdbJSa4mBkncKQCImArdAptKuvMciB3uSCfGqq1lZdBnsDR1O4isltDMBCLlAA9LQaXpksvQ99OROp965J0AFn9Vy64mwrhuonZ2c0C_dAAHJ_NSmmzI59xhB5QmCasINzGNNYBqxSzqRxCOpPcXt_D16il7nvsSIZoVuQDp3jsXO5fWONx5HtApO5zm7NpfSv800cFQjgQ8GHPqdVdufpKgAXaMxFlnLhgWP2QHTIpY2Mmy5zbKAEtTraWmYQs-cMhhj7sAzXNk6Wt25r9fyFVhmzGwhNmp4eUiVhiLKRTDuDSsFMaky__mtHcOEUZSvtyUEYQ2fqnHzlsBP4Ai8_Hr6d-qPQLmidR-69U5VQb6ftBGOeivzClSVRDfbKW7jtez2zB39FPx6Wm_FZqR1vBMmSNt1mJH2laIxkIh2qrkMCgLmMlkspZX8r3_VTRfUJcTcMWkzX16O8XzJeuI9YGgupF4K7wmzEmj1qZWgyXCSrB6L3873W8kPEui7lc",
  "e": "AQAB"
}

We’re almost there. A JWKS is an array of JSON Web Keys that form the key set. It is possible to have multiple public keys in one JWKS that are identified by the kid (Key ID) parameter. That being said, I have only ever seen single entry JWKSs. Putting our new JSON Web Key into a JWKS could look something like the following.

{
  "keys": [
    {
      "use": "sig",
      "kty": "RSA",
      "n": "wi5rLEy1U7m8rU8bQn2GeJ_g_XisJesbzI0N0QbYF3BNaEuQUwXOnh2ME8bOyKBrpXLik5AvljKp8HjwKG1x456kueJJoullYEBtrSRydnNaOQmUno1GQEcreCnRBZzodi9kw0YgsQvEfyxGwxI1NYSS8mdCSgT_BjOw5veHFfK-kdbJSa4mBkncKQCImArdAptKuvMciB3uSCfGqq1lZdBnsDR1O4isltDMBCLlAA9LQaXpksvQ99OROp965J0AFn9Vy64mwrhuonZ2c0C_dAAHJ_NSmmzI59xhB5QmCasINzGNNYBqxSzqRxCOpPcXt_D16il7nvsSIZoVuQDp3jsXO5fWONx5HtApO5zm7NpfSv800cFQjgQ8GHPqdVdufpKgAXaMxFlnLhgWP2QHTIpY2Mmy5zbKAEtTraWmYQs-cMhhj7sAzXNk6Wt25r9fyFVhmzGwhNmp4eUiVhiLKRTDuDSsFMaky__mtHcOEUZSvtyUEYQ2fqnHzlsBP4Ai8_Hr6d-qPQLmidR-69U5VQb6ftBGOeivzClSVRDfbKW7jtez2zB39FPx6Wm_FZqR1vBMmSNt1mJH2laIxkIh2qrkMCgLmMlkspZX8r3_VTRfUJcTcMWkzX16O8XzJeuI9YGgupF4K7wmzEmj1qZWgyXCSrB6L3873W8kPEui7lc",
      "e": "AQAB",
      "kid": "3d911ijttg0k80u2k74ax0hxeuhnd9njad7oa6nf",
      "alg": "RS256",
      "key_ops": [
        "verify"
      ]
    }
  ]
}

There are a few tweaks that need to be made (either programmatically or manually). The first and most obvious is that a JWKS needs to have a top-level key named keys with a value that is an array of JSON Web Keys. The added keys to the generated JSON Web Key are use, kid, alg, and key_ops which stand for Public Key Use, Key ID, Algorithm, and Key Operations, respectively.

use can either be sig (signature) or enc (encryption). sig indicates that the key is used for verifying the signature on data. enc indicates that the key is used for encrypting data.

kid is a unique string that I generated used to identify JSON Web Keys within a JWKS.

alg identifies the algorithm intended for use with the key. This is the same value as what was passed into generateKeyPair.

key_ops identifies the intended operations upon which the JSON Web Key can be used. Some values for this array are sign, verify, and encrypt among others. This is a link to the RFC defining the values for this key.

Once all of this is constructed, we have our JWKS. I suggested writing this information to a file and hosting it at an endpoint. It would also be completely valid to write it as a BLOB to a database and return it on request.

Signing a JWT

If we complete this piece correctly, then verifying our JWT will be a breeze. If something goes awry…well, let’s just hope that doesn’t happen. First, let’s go back to what we learned about asymmetric encryption in the first section. We use our private key to encrypt a payload and our public key (now in the form of a JWKS) to unencrypt the payload. To sign our JWT, this code will need access to the private key we created in the first section. In addition, we can set claims and create a payload for our JWT as outlined in the RFC.

(async () => {
  const { publicKey, privateKey } = await generateKeyPair('RS256');
  // https://nodejs.org/api/crypto.html#crypto_class_keyobject
  const publicKeyString = publicKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  const privateKeyString = privateKey.export({
    type: 'pkcs1',
    format: 'pem',
  });

  const token = await new SignJWT({
      myClaim: true,
    })
    .setProtectedHeader({
      typ: 'JWT',
      alg: 'RS256',
    })
    .setIssuer('https://example.com')
    .setSubject('uniqueUserId')
    .setAudience('myapp.com')
    .setExpirationTime('6h')
    .setIssuedAt()
    .sign(privateKey);
  console.log(token);
})();

The logged token should look something like this

eyJhbGciOiJSUzI1NiJ9.eyJteUNsYWltIjp0cnVlLCJpc3MiOiJodHRwczovL2V4YW1wbGUuY29tIiwic3ViIjoiYXNkZiIsImF1ZCI6Im15VXNlciIsImV4cCI6MTYyNjg5OTM3MCwiaWF0IjoxNjI2ODc3NzcwfQ.z7RSsY34eyO_sOsebR92M7P2piqoP9vRDw0kp2VUCkU-2ZcGeA2Jvf4GpJDSwmjxSuXSptYwnF-Qvw9A7hb6BH6XmH9ZG3bFLR-UEUjqjAKL5LRleh3EKES2LqVvng89p9xzFsDePTyzzVmc4yWV0fGC1-lMTLAmnDXxhRFIZZdyBbtHoxt7bmgrdCkk8jV0qVy-SoxWb0KvC8A24Pkkb7eWAS1CQDwVxBTWJDa9ixc0-eKSt2xtzw6jL8o_bkoAHJV2Zk1Cu04752Z9eAExdNq3zI6_wQkwap44MR0kpNF2pMPZz6kNLEUECt_QAzobV7WKYuPtkLLKN_67P-OaLg

The payload (which is anything custom that I want to be added to the JWT) is the piece that looks like { myClaim: true }. The server gets to decide what it wants to add, and that payload will be available after verifying the JWT in another server. The other claims that are set are iss (Issuer), sub (Subject), aud (Audience), exp (Expiration Time), and iat (Issued At). The protected header is pretty self-explanatory but there is more information and an example in the RFC.

iss should be set to your service’s URL. For example, Crow Authentication sets the Issuer to https://api.crowauth.com which is the API URL. This claim is optional.

sub should uniquely identify the principal that is the subject of the JWT. This could be a user’s ID or email address. This claim is optional.

aud should name the intended recipient of the JWT. For example, the application server that is requesting it. This claim is optional.

exp represents the time at which the JWT should no longer be deemed valid. This claim is optional, but I highly recommend using it. For security reasons it is not a good idea to spit out a JWT that will forever validate the holder as being who they are. We need to either force the user to reauthenticate or use refresh tokens. Information abounds about reasons for using expiration claims and refresh tokens.

iat simply claims when the JWT was signed.

Verifying a JWT

Remember that JWKS that we created a couple of sections ago? Now we get to use it. For the sake of keeping this guide focused on JWTs I am going to assume that either A) you have taken the time to create a working authentication server that can provide JWTs and hosts the JWKS (while this might require a decent amount of time to get set up, it is ultimately what we’re going for here) or B) you do not have a server set up. Situation A is best, but if you are limited on time or simply want to read code then I will provide small scripts to show the concepts working.

At a high level, we need to grab the distributed public key then verify the JWT provided to the application server, so under normal circumstances, this code would be run by an application server to verify the identity of the client. In the example, I will be using the JWKS that I have referenced a couple of times throughout this guide. Please substitute the URL for the appropriate JWKS which corresponds to the private key used by the authentication server that signed the JWT.

const { createRemoteJWKSet } = require('jose/jwks/remote');
const { jwtVerify } = require('jose/jwt/verify');

const jwks = createRemoteJWKSet(new URL('https://crowauth.com/v1/jwks.json'));
const { payload, protectedHeader } = await jwtVerify(jwt, jwks); // The jwt variable needs to be passed in from somewhere; cookie, hard coded, parameter, etc.

What happens here, is that jose pulls the JWKS, finds the key based on use being sig, key_ops including verify, and some user-provided options. It then uses the JWK found from the JWKS to verify that the signature on the JWT did in fact come from the correct private key/authentication server. jwtVerify can also take in which claims to verify like the Issuer or Audience. The payload returned will contain the application-specific claims or information provided while signing the JWT. Based on my previous signing example, the payload would be { myClaim: true }.

In case you do not have a remote JWKS set up and the rest of the authentication server ready, here is a script that shows the same concept but at a local level.

(async () => {
  const { publicKey, privateKey } = await generateKeyPair('RS256');
  // https://nodejs.org/api/crypto.html#crypto_class_keyobject
  const publicKeyString = publicKey.export({
    type: 'pkcs1',
    format: 'pem',
  });
  const privateKeyString = privateKey.export({
    type: 'pkcs1',
    format: 'pem',
  });

  const publicJwk = await fromKeyLike(publicKey);

  const token = await new SignJWT({
      myClaim: true,
    })
    .setProtectedHeader({
      typ: 'JWT',
      alg: 'RS256',
    })
    .setIssuer('https://example.com')
    .setSubject('asdf')
    .setAudience('myUser')
    .setExpirationTime('6h')
    .setIssuedAt()
    .sign(privateKey);

  const parsedJwk = await parseJwk(publicJwk, 'RS256'); // For the sake of the example, parse the JWK instead of simply using the generated publicKey
  const { payload, protectedHeader } = await jwtVerify(token, parsedJwk);
  console.log(protectedHeader);
  console.log(payload);
})();

While parseJwk is not entirely necessary (we could just use the publicKey directly), this shows the same idea of verifying a JWT based on a JWK.

The End

You’ve done it. Good job. Hopefully, this guide made learning about JWTs with Javascript easy and you did not need to do much Googling on the side. At the beginning of the guide, I mentioned that JWTs are a popular form of authenticating users for web applications. Did you notice that I did not once talk about JWTs in the context of web applications? Whoops.

Understanding JWTs and how to operate with them was enough for this guide. I will be putting out another guide soon about using JWTs with web applications. That guide will cover how to authenticate users, where JWTs should be stored, and the relationships between the authentication server, browser, and application servers. Whenever it is written and published, I will edit this post with a link to it.

If you enjoyed this guide and it was helpful, please let me know somehow. Send an email, tweet at me, comment on one of the cross-posts (if you’re not reading this in my personal blog), whatever floats your boat. If it wasn’t helpful, there are errors, or you have suggestions, the same rule applies. If you need authentication for a new web application and this is your first introduction to it, might I suggest saving yourself the hassle and going with an off-the-shelf solution like Crow Authentication? Until next time!

Three New Things I Tested To Improve My SEO

Thomas Step — Thu, 15 Jul 2021 00:00:00 +0000

I’m writing this post with a little hesitancy. The title might read like generic clickbait, but I hope to inject some originality into this subject. What you probably clicked on to read about is further on down past my rant. Feel free to skip ahead.

When I first started hearing about SEO I imagined it as a large abstract construct. The quick and dirty explanation that I go back to for myself is that SEO is simply the process of ranking higher on search engines. That’s it. There are tons of hacky methods and instructors out there that pawn off regurgitated content as new courses that will “help buyers and bloggers make more money.” That’s not what I’m trying to do here. I remember reading through posts about finding keywords to write about, investigating a niche with ahrefs, and writing consistently. After reading these “tricks” for the Nth time, I stopped clicking on SEO articles. I prefer to simply write about what I want and what I wish was out there when I was looking for guidance or to solve a problem.

In my opinion, there is way too much recycled content out there “written” by people who claim to have subject knowledge, but who have never practiced or seen success with their methods. I’ll argue that they just want a few quick clicks and use them to get a few easy pennies. Swyx wrote about this class of authors as meta-creators.

All that just to say, that I hope these tips are new and original to you. There have only been three jumps in my traffic history so far and each one corresponds to a different tip. I saw more success with my traffic and rankings after I completed these, so maybe they can help you too. But who knows 🤷‍♂️, I’m no SEO expert.

Use Canonical URLs

When I first started to cross-post to DEV.to I did not realize that they offered canonical URLs. I’m pretty sure I didn’t even know what they were at the time. Instead, I used to manually add in my page’s URL as a backlink to try and drive traffic to me, but that did not do near as much as a link tag for a canonical URL. I know that adding canonical URLs to my DEV.to posts worked because my article views there stopped increasing as rapidly, and I saw more organic traffic being driven to me about a week after I added them to my existing posts. I guess that it just took a few days for Google to realize where my original content was posted before it started ranking me above the DEV.to versions of my articles. Most publishers (like DEV.to and Medium) have a setting for an article’s canonical URL. However, if you have access to the cross-posted article’s HTML you can add the following in the <head> instead.

<link rel="canonical" href="https://your.site/original-article">

Changed To A `.com` Domain Name

I have heard that Google does not care about the TLD, which is why I initially used a .dev TLD. I switched TLDs for a different reason, but whenever I did switch, I started getting a noticeable jump in organic traffic. Since Google themselves say that it doesn’t affect rankings, I am left to take a shot in the dark for why this worked. I guess that my traffic was less affected by Google and more affected by people. My thought is that people trust a .com TLD more than a .dev one, which causes them to click on it before other options. While Google might not care about TLD, they do care about clicks, which results in a higher ranking. People and their behaviors also play a part in SEO.

Leave Breadcrumbs Every Where You Go

Another significant boost to my organic traffic came after I reorganized my site’s pages. What is now my /blog page used to be my homepage. I reorganized it to become my /blog page and then I created my home page (/) based on my old /about page. Again, I did this not for SEO, but my own reasons. However, after I did it, I received my latest surge in organic traffic. My only guess as to why this worked was because Google could now follow breadcrumbs to all of my pages within each other. All of my blog posts are under URLs like /blog/this-is-a-blog-post. My thought is that Google wants breadcrumbs to each path and since I did not have a /blog page before, there was a gap between my home page and my blog posts.

I hope that helps and I hope someone can benefit from my observations. It definitely helped me. Feel free to reach out if you have any other questions!

DEV Community: Thomas Step

How to Create a Custom Error Class in Javascript

How I Implement Hexagonal Architecture in AWS Lambda

How To Write AWS CDK Tests

How To Use The DynamoDB Document Client

Creating the Document Client

PutItem

BatchWriteItem

GetItem

Query

UpdateItem

DeleteItem

API Gateway Models

AWS CDK Example For API Gateway And SNS Integration

I Wrote a Guide to Serverless AWS

Destructuring Objects in Javascript

Writing Asynchronous Lambda Function with Node

Centering a Div With Tailwind CSS

A Guide To Using JWT In Javascript

Creating Public and Private Keys

Creating a JWKS

Signing a JWT

Verifying a JWT

The End

Three New Things I Tested To Improve My SEO

Use Canonical URLs

Changed To A .com Domain Name

Leave Breadcrumbs Every Where You Go

`PutItem`

`BatchWriteItem`

`GetItem`

`Query`

`UpdateItem`

`DeleteItem`

Changed To A `.com` Domain Name