The Old Faithful: Why SSM Parameter Store still reigns over Secrets Manager

Managing and securing application secrets is a crucial part of any cloud-native application. AWS offers two primary services: the Systems Manager (SSM) Parameter Store and the more recent Secrets Manager.

You might think Secrets Manager is the better choice for managing secrets because it’s a newer service and offers more advanced features such as cross-region replication. After all, as its name suggests, it’s not just a simple “store”, but a manager of secrets.

But you know what. Five years after the initial launch of Secrets Manager in April 2018, I’m still using SSM Parameter Store for most of my secrets. And I’m far from alone in this.

In this post, let’s explore why the tried-and-true SSM Parameter Store is still the preferred choice for many developers and dive into the advantages it has over Secrets Manager.

Cost-effectiveness

One of the most significant differences between the two services is their pricing.

Secrets Manager charges you based on usage (no. of API calls) as well as an uptime cost of $0.40 per secret per month.

With SSM Parameter Store, if you stick to Standard parameters and do not enable the Higher Throughput mode then there’s no charge for using it.

You can have up to 10,000 Standard parameters in a region, with a maximum parameter size of 4kb. These limits should be sufficient for most use cases.

However, the Standard Throughput mode limits you to 40 ops/s. So for the production environments, you should enable the Higher Throughput mode and pay $0.05 per 10,000 API calls (same as Secrets Manager). This raises the throughput limit to 3000 ops/s.

It’s worth noting that $0.05 per 10,000 API calls translates to $5 per million. Which is relatively high compared to other pay-per-use services such as Lambda. In fact, even DynamoDB’s on-demand mode only charges you $0.25 per million read requests!

The expectation is that these application configurations or secrets are slow-changing values and should be cached by your application.

However, as this thread by Luc Van Donkersgoed has highlighted, even experienced AWS users can make the occasional mistake and be left with a surprise.

The story serves to highlight the need for cloud cost controls and that there is still a lot to be done in this space by both AWS and its ecosystem of supporting vendors. CloudZero is one of the better tools out there for managing your cloud costs and should have picked up the spending abnormality within a few hours.

The cost consideration between SSM Parameter Store and Secrets Manager boils down to the monthly uptime cost.

On its own, $0.40 per secret (+ per replica) per month might not seem a lot. But it’s cumulative and can add up when you use it for all your configurations and secrets. Especially when it’s multiplied by the no. of environments (ie. AWS accounts) and regions you have.

I’m not advocating using a cheaper and less capable service just to save money. After all, you get what you pay for. And some of Secrets Manager’s features more than pays for themselves (see “When should you use Secrets Manager?” section below) WHEN you need them.

But for most of my application configurations and secrets, SSM Parameter Store would suffice. It has all the basic features you need: encryption-at-rest, integration with CloudFormation and built-in audit history.

And if your requirements were to change at a later date, it’s straightforward to swap out SSM Parameter Store with Secrets Manager there and then. Especially if you’re accessing the relevant service through a middleware layer such as Middy for javascript Lambda functions.

Again, I’m not against spending on the right tool, I’m uncomfortable with wasteful spending. I don’t want to cut cloud spending, I want to cut waste. Paying for premium features that I don’t need is wasteful.

Simplicity

On paper, Secrets Manager gives you built-in secret rotation. But the devil is in the details.

The rotation process is not fully managed. You still need to provide the Lambda function that Secrets Manager would call when it’s time to rotate the secret.

So if you still have to do the heavy lifting, what does Secrets Manager offer?

It lets you specify the rotation schedule, and it provides a structure for building a robust secret rotation process:

1. Create a new version of the secret.
2. Change the credential in the database or service.
3. Test the new version.
4. Finish the rotation.

(see official documentation here)

This is a very reasonable process for the intended use case?—?changing the root user credentials for database services such as RDS, DocumentDB or Redshift. But it’s overkill for almost every other type of secret and leads to a more verbose and complicated rotation Lambda function.

With SSM Parameter Store, you can write a rotation Lambda function and invoke it with a custom EventBridge schedule. In my experience, this is far simpler to implement and many developers are already familiar with implementing cron jobs with Lambda and EventBridge.

Because it’s such a common pattern, it’s also very well supported by most frameworks. In the Serverless framework, it can be configured with just a few lines of yml:

functions:
  rotateSecret:
    handler: /functions/rotate-secret.handler
    events:
      - schedule: rate(5 minutes)

You also get two retries and dead-letter queue support (preferably using Lambda failure destinations) because this is an async Lambda invocation. So the rotation process is still pretty robust.

And if you need to make it even more robust, then you can always introduce a custom Step Functions state machine instead. But at that point, it would be easier to just use Secrets Manager.

Besides the rotation process, SSM parameters are also simpler to configure in CloudFormation and easier to use. After all, it’s only a store of parameters. There are fewer features, and therefore, fewer decision points that you need to consider.

Flexibility

Your application would likely have many configurations and not all of them are sensitive and need to be encrypted.

Both SSM Parameter Store and Secrets Manager lets you control access through granular IAM permissions.

Not every component (e.g. Lambda functions) needs to access all your configurations. To follow the principle of least privilege , you should not put all your configurations and secrets in one value. Instead, you should separate them into sensible groupings and make sure a component is only given access to the ones it actually needs.

However, this incurs additional costs with Secrets Manager because you will end up paying $0.40 per secret (+ $0.40 per replica) per month for each of these secrets. Because of this, I have often seen customers group secrets together to reduce costs. But it also weakens their data security.

Other the other hand, SSM Parameter Store can handle both plain text parameters as well as encrypted strings. This makes it the more flexible option.

Its Intelligent Tiering feature also means you don’t have to constantly choose between Standard and Advanced parameters. The system would default to Standard unless your parameter is bigger than 4kb or you have exceeded the 10,000 parameters limit.

Again, fewer decision points for you to concern yourself with.

When should you use Secrets Manager?

However, there are a number of use cases where you should definitely prefer Secrets Manager:

1. Replicating secrets for multi-region applications. Secrets Manager supports cross-region replication out of the box.
2. When working with large (> 8kb) secrets. SSM Parameter Store has a parameter size limit of 8kb (for advanced parameters). Secrets Manager, on the other hand, has a maximum secret size of 64kb.
3. If you need to share secrets cross-account. Secrets Manager allows you to control cross-account access with resource policies. This is very useful in an enterprise environment and helps you control access centrally and eliminate the need to duplicate the secret value in different accounts. See the official documentation here for more detail. (Thanks to Heitor for pointing this out!)

SSM Parameters Store lacks cross-region replication and can’t handle large secret values. And while it’s possible to work around them by building custom solutions yourself, it’s easier and faster to simply switch to Secrets Manager for the relevant secrets instead.

After all, both are useful services and you don’t need to limit yourself to just using one. Use the right tool for the job, and sometimes that means using both tools because they’re right in different contexts.

Wrap up

I hope you have found this article insightful and helped you better understand the difference between SSM Parameter Store and Secrets Manager.

If you want to learn more about building serverless architecture, then check out my upcoming workshop where I would be covering topics such as testing, security, observability and much more.

Hope to see you there.

The post The Old Faithful: Why SSM Parameter Store still reigns over Secrets Manager appeared first on theburningmonk.com.