Workflow in AWS Step Functions is represented by a state machine which consists of a number of states (steps), their relationships and their input and output. When we are constructing the state machine using Amazon States Language, we might define the application secrets as part of the state machine code.
One of the possible situations is that we are using a Pass state to prepare an input JSON for the next state. In the following example, we are passing the remote FTP username and password to the next state, which is a Lambda function to download files from remote FTP server.
"ftp-download-lambda-config": {
"Type": "Pass",
"Result": {
"ftpHost" : "ftp-server.com",
"ftpUsername" : "username",
"ftpPassword" : "password",
"withDecryption": true
},
"ResultPath": "$",
"Next": "ftp-download-lambda"
},
"ftp-download-lambda": {
"Type": "Task",
"Resource": "arn:aws:lambda:xxx:xxx:function:ftp-download-lambda",
"Next": "next-step"
},
This will raise a concern if we are to version control the state machine code into any version control system like Git. It is because the application secrets(the FTP username and password) will be exposed to the users who have access to the codebase. As a good security practise, it is advisable to store the secrets in a separate environment, which will ensure that it is safe with limited access and possible encrypted where they are stored.
There are few prominent services in market: Vault(from Hashicorp), Secrets Manager and Parameter Store (from AWS). Since a managed service is preferred, Vault will not be taken into consideration. Comparing Secrets Manager with Parameter Store, Secrets Manager costs $0.40 per secret per month $0.05 per 10,000 API calls and Parameter Store costs nothing for usage. Although Secrets Manager comes with a secret rotation feature which allows you to automatically rotate API keys, passwords and more, it is not required for our requirement. Therefore, AWS Parameter Store is more preferable for this use case.
So, once we have decided to store our secrets in the AWS Parameter Store, we have to modify our application(Lambda function in this case) to accept the reference of secrets stored in Parameter Store and retrieve the secrets from Lambda function. The change in the state machine code is showed as below:
"ftp-download-lambda-config": {
"Type": "Pass",
"Result": {
"ftpHost" : "ftp-server.com",
"ftpUsername" : "<Reference to username in Parameter Store>",
"ftpPassword" : "<Reference to password in Parameter Store>",
"withDecryption": true
},
"ResultPath": "$",
"Next": "ftp-download-lambda"
},
"ftp-download-lambda": {
"Type": "Task",
"Resource": "arn:aws:lambda:xxx:xxx:function:ftp-download-lambda",
"Next": "next-step"
},
Or we can have additional Lambda function to retrieve the secrets from Parameters Store and pass the output to the next state in the state machine. Thus, the existing Lambda functions/tasks can remain as it is with the usage of new Lambda functions. The usage of new Lambda function is showed as below:
"ssm-parameters-getter-lambda-config": {
"Type": "Pass",
"Result": {
"param": [
"parameters-store-ftpUsername",
"parameters-store-ftpPassword"
],
"withDecryption": true
},
"ResultPath": "$",
"Next": "ssm-parameters-getter-lambda"
},
"ssm-parameters-getter-lambda": {
"Type": "Task",
"Resource": "arn:aws:lambda:xxx:xxx:function:ssm-parameters-getter-lambda",
"Next": "ftp-download-lambda"
},
"ftp-download-lambda": {
"Type": "Task",
"Parameters" : {
"ftpHost": "ftp-server.com",
"ftpUsername.$": "$.parameters-store-ftpUsername",
"ftpPassword.$": "$.parameters-store-ftpPassword",
},
"Resource": "arn:aws:lambda:xxx:xxx:function:ftp-download-lambda",
"Next": "next-step"
},
The ssm-parameters-getter-lambda accepts the following as input:
{
"param": [
"random-input-a",
"random-input-b"
],
"withDecryption": true
}
And produce an object with param name as attribute and value from parameter store. The value can be
{
"random-input-a": "<Value from Parameter Store>"
"random-input-b": "<Value from Parameter Store>"
}
The Lambda function is open sourced on Github. It can be deployed manually to your AWS account using the source code: index.js or deploy directly from the AWS Serverless Application Repository with name: ssm-parameters-getter-lambda. Remember to check Show apps that create custom IAM roles or resource policies as if this Lambda requires additional role.
mengjiann
/
ssm-parameters-getter-lambda
To retrieve parameters from AWS Systems Manager's Parameter Store.
ssm-parameters-getter-lambda
To retrieve parameters from AWS Systems Manager's Parameter Store.
Use case
The best use case is to place this before the state in a Step Function's workflow that accepts configuration as input such as username and password. As such, the application secrets are kept safely in AWS Parameter Store rather than in the Step Function's state machine code.
Parameters Access Restriction
Following the principle of least privilege, there are two options that allows restriction on the AWS Lambda function to access certain parameters from the Parameters Store.
- ParametersPrefix - Only allow access to parameters with certain prefix in the Parameters Store.
- Tag Keys List - Only allow access to parameters with certain tags for the parameters. Allows comma-delimited list of tag keys.
Sample Input and Output
Input
{
"param": [
"random-input-a",
"random-input-b"
],
"withDecryption": true
}
Output
{
"random-input-a": "<Value from Parameter Store>"
"random-input-b": "<Value from Parameter Store>"
}
References:
Top comments (0)