DEV Community

Daniel Fyhr
Daniel Fyhr

Posted on

4 1

Simple Cache for AWS Secrets Manager

Introduction

If you need to access some secrets from AWS Secrets Manager, it's a good idea to cache the values. This way you will fetch them less frequently and save on costs. Unfortunately, it's not built-in in aws-sdk for NodeJS. Fortunately, it's pretty straightforward to implement.

In this post we will take a look at why it's a good idea and one way to do it when using AWS Lambda functions. The provided implementation is in TypeScript.

Why you should cache the values

Every external call is a risk and there are many things that can go wrong. The network is not reliable. I once hit the rate quota for fetching values and the service was just waiting for an answer, eventually timing out.

Retrieving a cached value is faster and you can save money with just a few lines of code. Not only do you save on calls to AWS Secrets Manager but you will also have shorter duration.

Strategy

The first time an AWS Lambda function is run it creates an execution environment if there isn't one already. When the execution is done that environment will remain available for some time for subsequent executions.

We can use this as a simple caching mechanism by creating an object in the environment. When we put values in that object, they can be accessed the next time the function is invoked.

Implementation

Let's break it down into two components. First a component for caching:

class SimpleCache {
  private cache: Record<string, string> = {};
  constructor(private readonly loadValue: (key: string) => Promise<string | undefined>) {}
  async get(key: string) {
    // if we find the value in the cache, return immediately
    if (this.cache[key]) {
      return this.cache[key];
    }
    // load it with the provided function
    const res = await this.loadValue(key);
    if (res == null) {
      return res;
    }
    // put the value in the cache and return.
    // The next time we need the value, we don't have to fetch it again.
    this.cache[key] = res;
    return res;
  }
}
Enter fullscreen mode Exit fullscreen mode

Then a component for fetching the value of a secret with a given key:

import SecretsManager from 'aws-sdk/clients/secretsmanager';

const secretsClient = new SecretsManager();
const client = new SimpleCache((key) =>
  secretsClient
    .getSecretValue({ SecretId: key })
    .promise()
    .then((x) => x.SecretString),
);
Enter fullscreen mode Exit fullscreen mode

Putting it all together:

import SecretsManager from 'aws-sdk/clients/secretsmanager';

class SimpleCache {
  private cache: Record<string, string> = {};
  constructor(private readonly loadValue: (key: string) => Promise<string | undefined>) {}
  async get(key: string) {
    if (this.cache[key]) {
      return this.cache[key];
    }
    const res = await this.loadValue(key);
    if (res == null) {
      return res;
    }
    this.cache[key] = res;
    return res;
  }
}

// When we create these two instances outside of the handler 
// function, they are only created the first time a new 
// execution environment is created. This allows us to use it as a cache.
const secretsClient = new SecretsManager();
const client = new SimpleCache((key) =>
  secretsClient
    .getSecretValue({ SecretId: key })
    .promise()
    .then((x) => x.SecretString),
);

export const handler = async () => {
  // the client instance will be reused across execution environments
  const secretValue = await client.get('MySecret');
  return {
    statusCode: 200,
    body: JSON.stringify({
      message: secretValue,
    }),
  };
};
Enter fullscreen mode Exit fullscreen mode

Additional usage

We can use the SimpleCache implementation above for other integrations. If we fetch some other static values over the network, we can use the same caching mechanism.

For example, if we decide to use aws-sdk v3 instead we can use the same SimpleCache but change the component related to SecretsManager. V3 has a nicer syntax where we don't have to mess around with .promise().

Don't put secrets in environment variables

You can resolve the values from AWS Secrets Manager during deployment and put them in the environment variables.

Unfortunately this means your secrets are available in plain text for attackers. It's one of the first place they would look. It has happened before and will probably happen again. Here's an example of an attack like that..

Conclusion

In this post we have covered why you should cache values from AWS Secrets Manager. It will save you money and make your code more reliable and performant. There's also an implementation of how to achieve this.

If you found this post helpful, please consider following me on here as well as on Twitter.

Thanks for reading!

Speedy emails, satisfied customers

Postmark Image

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up

Top comments (2)

Collapse
 
rodrigocprates profile image
Rodrigo Prates

Awesome!

Do you have a version to expire the cache every X minutes/hours? Otherwise that would eternally lookup for the same keys, even if they got changed.

Collapse
 
danielfy profile image
Daniel Fyhr

So far I have not needed that. Because the lambda function resets after some time, you get this behaviour for free.

Another way to expire the cache is to deploy a new version of the function. Chances are, if you are updating a secret, updating the function is not too much extra work.

If you still need it, I would mark a timestamp in the constructor of SimpleCache. Then in the get method, if there is a cached value, check if too much time has elapsed.

Billboard image

Deploy and scale your apps on AWS and GCP with a world class developer experience

Coherence makes it easy to set up and maintain cloud infrastructure. Harness the extensibility, compliance and cost efficiency of the cloud.

Learn more

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay