Simplify secrets management with LaconiaJS

Thomas Ankcorn — Tue, 30 Jun 2020 21:56:58 +0000

The boilerplate code required to retrieve secrets from the various AWS services makes serverless applications harder than they should be to build, and test.

Not only are there two services to choose from but they don’t provide a way to provide the value of the secret to a lambda at runtime. We have to choose and do the work to retrieve the secrets ourselves. Outrageous!

Let's look at an example:

I want to create a function that needs to retrieve 2 secrets

A Stripe key that is stored in Secrets Manager
An api key for our companies serious business service in SSM Parameter Store

Here's the code using just the AWS SDK to retrieve the values:

const SecretsManager = require('aws-sdk/clients/secretsmanager');
const SSM = require('aws-sdk/clients/ssm');

module.exports.handler = async () => {

        // retrieve stripe keys
        const secretsManager = new SecretsManager({ region: 'eu-west-1' });
        const { SecretString } = await secretsManager.getSecretValue({ SecretId: 'external/stripe' }).promise();
        const stripe = JSON.parse(SecretString);

        // retrieve api key
        const ssm = new SSM({ region: 'eu-west-1' })
        const { Value: apiKey } = await ssm.getParameter({ Name: 'sb-api-key' }).promise();

        // serious business logic follows 🐒
        // ...
}

Other than the fact it's a bit long there are some serious problems with this approach.

If we want to move the API key to secrets manager we have to change some application code - that's probably why it's still in SSM
We cannot unit test the business logic here without first mocking out the calls to the SSM and secrets manager.

LaconiaJS solves these problems really neatly. Here's the example solution using LaconiaJS:

Define 2 environment variables using your deployment framework of choice

LACONIA_CONFIG_STRIPE=secretsManager:external/stripe
LACONIA_CONFIG_SB_API_KEY=ssm:sb-api-key

const laconia = require('@laconia/core');
const config = require('@laconia/config');

function seriousBusiness(lambdaInput, { stripe, sbApiKey }) {
  // stripe and sbApiKey are passed into the function by laconia
}

module.exports.handler = laconia(seriousBusiness)
         .register(config.envVarInstances());

As you can see there is a lot less code, it's more malleable, and it's much clearer how we would test the business logic of this function.

Now when testing the seriousBusiness function we can pass in arbitrary values for the secrets instead of having to connect to production AWS resources.


test('serious business test', async () => {
        const input = {};
        expect(await seriousBusiness(input, { stripe: 'abc', sbApiKey: '123' })).toEqual({ message: 'Serious Business Complete' });
})

Finally

Even though I am pretty new to LaconiaJS its making serverless development much more productive and happy for me. I feel that I write less code and get to focus more on the business logic whilst using it. If you are want to learn more have a look at their website https://laconiajs.io/

Gradually Typing Serverless Applications

Thomas Ankcorn — Sat, 22 Feb 2020 22:24:24 +0000

I love creating applications with Javascript. The code is fresh. The feedback loop fast. We remember the properties of that object because we created it just 5 minutes ago.

Maintaining them, not so fun, once the dust settles and our brains are foggy. Attempting to modify and change the code can feel like trying to paint a window with a baseball bat.

We can write lots of unit tests. It helps us catch bugs and prevent regression, but it's time-consuming, and we can only really test the cases we think of first.

The other option is typing our code!

"But hey Thomas we have already written 100,000 lines of javascript, we can't rewrite it all?."

Enter gradual typing!

You can't have your cake and eat it, but you can gain the benefits of static and dynamic types in the same application using Typescripts --allowJs flag. Now we can make a strategic choice about when to use Typescript in our codebase.

Adopting Typescript like most engineering decisions is all about tradeoffs. In this case, we increase the initial cost of development, in the hope of reducing the cost of maintaining the code.

So we need to apply it precisely to where we know it will have the maximum value. Using monitoring tools we know about the bugs in our software, we must do a route cause analysis to see which bit of code caused the problem. If the problem area is

Highly dynamic.
Hard to reason about
Filled with assumptions

Then it's a candidate to convert to Typescript.

The configuration

First, we are going to do a small amount of configuration to our serverless project. I'm going to use serverless webpack to add typescript support. There are other options, serverless-plugin-typescript for example. But I already use webpack so will take that approach and combine it with the migrating from javascript docs from the typescript website. https://www.typescriptlang.org/docs/handbook/migrating-from-javascript.html
https://serverless.com/plugins/serverless-plugin-typescript/

My notes are linked below, but the typescript and serverless webpack docs are much more complete resources on this.

Installing the dependencies

yarn add -D webpack typescript serverless-webpack awesome-typescript-loader

Configuring the serverless framework

plugins:
  - serverless-webpack
  - serverless-offline

custom:
  webpack:
    webpackConfig: 'webpack.config.js'   # Name of webpack configuration file
    includeModules:
      packagePath: './package.json'   # Node modules configuration for packaging
    packager: 'yarn'  # Packager that will be used to package your external modules
    excludeFiles: src/**/*.test.js

Enabling typescript in webpack

const path = require('path');
const webpack = require('webpack');
const slsw = require('serverless-webpack');

const rootDir = path.resolve(__dirname);
const serviceName = slsw.lib.serverless.service.service;

const srcDir = path.resolve(rootDir, 'src');
const nodeModulesDir = path.resolve(rootDir, 'node_modules');
const outputPath =  path.resolve(rootDir, '.webpack', serviceName);

process.env = Object.assign(
  process.env,
  slsw.lib.serverless.service.provider.environment,
  { NODE_ENV: slsw.lib.serverless.service.provider.stage }
);

module.exports = {
  target: 'node',
  entry: slsw.lib.entries,
  mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
  devtool: "source-map",
  optimization: {
    // We no not want to minimize our code.
    minimize: false
  },
  externals: [
    'aws-sdk' 
  ],
  resolve: {
    // Add '.ts' and '.tsx' as resolvable extensions.
    extensions: [".webpack.js", ".web.js", ".ts", ".tsx", ".js"]
  },
  module: {
    rules: [
      {
        test: /\.js$/,
        include: [
          srcDir
        ],
        exclude: [
          nodeModulesDir
        ]
      },
      { test: /\.js$/, loader: "source-map-loader" },
      { test: /\.tsx?$/, loader: "awesome-typescript-loader" }
    ]
  },
  plugins: [
    new webpack.EnvironmentPlugin(['NODE_ENV'])
  ],
  output: {
    path: outputPath,
    filename: '[name].js',
    libraryTarget: 'commonjs2',
    pathinfo: true
  }
};

Conclusion

Typescript is super powerful at eliminating a particular class of bugs. It helps get us faster feedback on our code which is key to programmer happiness and productivity. Gradually typing helps us avoid some of the costs. This makes it an easier sell to teams and businesses who already have large javascript applications.