AWS Step Function Background
AWS Step Functions is one of the AWS services which allows us to define workflow by integrating the existing AWS services such as Lambda, Fargate, SNS and etc. Each workflow is based on the concept of a state machine and it is defined using Amazon States Language (ASL) (JSON-based).
Workflow on Step Functions is made up of series of steps, with the output of one step acting as input into the next step. Thus, you can perform a series of actions to the workflow input. Due to the limitation of data passed between each steps, it is best to use a separate service like AWS S3 to transfer the large files (Limit).
The benefit of using Step Function is that the application logic is implemented within the workflow and it is separated from the implementation of individual component (step). This actually promotes the reusability of each component for other workflows.
A simple state machine is defined as below. StartAt and States are the compulsory building blocks. StartAt defines the entry point of your workflow and States defines the steps for your customised workflow. There are other optional components that you can have within the state machine.
{
"Comment": "A Hello World example of the ASL.", // Optional
"StartAt": "HelloWorld", // Compulsory
"States": { // Compulsory
"HelloWorld": {
"Type": "Pass",
"Result": "Hello World!",
"End": true
}
}
}
To define the steps within the workflow, these are the available types of states provided by ACL now:
| State | Purpose |
|---|---|
| Task | A Task state represents a single unit of work performed by a state machine. |
| Pass | A Pass state passes its input to its output, without performing work. |
| Choice | A Choice state adds branching logic to a state machine. |
| Wait | A Wait state delays the state machine from continuing for a specified time. You can choose either a relative time, specified in seconds from when the state begins, or an absolute end time, specified as a timestamp. |
| Succeed | A Succeed state stops an execution successfully. The Succeed state is a useful target for Choice state branches that don't do anything but stop the execution. |
| Fail | A Fail state stops the execution of the state machine and marks it as a failure. |
| Parallel | The Parallel state can be used to create parallel branches of execution in your state machine. |
Sample Step Functions Workflow
This article is going to cover some useful features of ASL and how to create a simple workflow by using AWS Lambda functions. Sample AWS Lambda functions are available here. Before starting, you will need to deploy the AWS Lambda functions from the sample code. Do check out here if you need some help on the Lambda deployment.
The sample Step Functions workflow diagram is as below:
The basic idea of this workflow is to:
1) Configure the random-number-generator-lambda to generate a random number from a number range.
2) Then, trigger the random-number-generator-lambda to generate a random number.
3) Based on the output of the random-number-generator-lambda, the next step is to check if the output is above or below number 5.
4) If it is above 5, SNS notification will be triggered to send out SMS.
5) If it is below 5, another Lambda power-of-number-lambda will be triggered to raise the previous input to the power of configured exponents.
6) Then, the workflow will end after that.
Here is the explanation of ACL features being used for this sample workflow:
1) random-number-generator-lambda-config - To configure the random-number-generator-lambda by using the Pass state.
"random-number-generator-lambda-config": {
"Comment": "To configure the random-number-generator-lambda.",
"Type": "Pass",
"Result": {
"min": 1,
"max": 10
},
"ResultPath": "$",
"Next": "random-number-generator-lambda"
},
2) random-number-generator-lambda - To trigger a Lambda function by using the Task state.
"random-number-generator-lambda": {
"Comment": "Generate a number based on input.",
"Type": "Task",
"Resource": "${random-number-generator-lambda-aws-arn}",
"Next": "send-notification-if-less-than-5"
},
3) send-notification-if-less-than-5 - To check if the output from previous step is less than 5 using the Choice state.
"send-notification-if-less-than-5": {
"Comment": "A choice state to decide to send out notification for <5 or trigger power of three lambda for >5.",
"Type": "Choice",
"Choices": [
{
"Variable": "$",
"NumericGreaterThanEquals": 5,
"Next": "power-of-three-lambda"
},
{
"Variable": "$",
"NumericLessThan": 5,
"Next": "send-multiple-notification"
}
]
},
4) power-of-number-lambda - If output is more than 5, trigger another Lambda function using Parameters to customise the input JSON object.
"power-of-three-lambda": {
"Comment": "Increase the input to power of 3 with customized input.",
"Type": "Task",
"Parameters" : {
"base.$": "$",
"exponent": 3
},
"Resource": "${power-of-number-lambda-aws-arn}",
"End": true
},
5) send-multiple-notification - A Parallel state to trigger multiple actions.
6) send-sms-notification - To trigger SNS notification to send SMS.
7) send-email-notification - To trigger SNS notification to send Email. (Not include as part of the sample)
"send-multiple-notification": {
"Comment": "Trigger multiple notification using AWS SNS",
"Type": "Parallel",
"End": true,
"Branches": [
{
"StartAt": "send-sms-notification",
"States": {
"send-sms-notification": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message": "SMS: Random number is less than 5 $",
"PhoneNumber": "${valid-handphone-number}"
},
"End": true
}
}
},
{
"StartAt": "send-sns-topic",
"States": {
"send-sns-topic": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message": "Email: Random number is less than 5: $",
"TopicArn": "${aws-sns-topic-to-send-out-email}"
},
"End": true
}
}
}
]
}
You can trigger the workflow by using the Start Execution button within the Step Functions console. Each invocation allows JSON object as input to the workflow, but it is not compulsory. Or Cloudwatch Schedule event can be configured to trigger the workflow.
Before that, it is also important to ensure that you have granted the appropriate permission to the IAM role for the Step Function execution. For this sample workflow, you will need the following
- AWS Managed Policy -
AmazonSNSFullAccess- To trigger SNS notification SMS or Email. - AWS Lambda Invocation Permission.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction",
"lambda:InvokeAsync"
],
"Resource": "*"
}
]
}
You should see something as below if the execution is successful. Do try a few more rounds to see you get a SMS from the SNS notification!
Not all states are being covered in the above sample workflow, more can be found from official doc on the different ASL features.
Hope this is useful for your exploration on Step Functions.
mengjiann
/
aws-step-functions-sample
Sample Workflow for AWS Step Function (with terraform)
Sample Project for AWS Step Functions
This is to demonstrate the usage of several AWS Lambda functions to create a workflow on AWS Step Function. Terraform configuration files are also provided to simplify the process of setting up the required AWS resources.
AWS Lambda Function
- power-of-number-lambda - To calculate the power of a number.
- random-number-generator-lambda - To generate a random number between MIN and MAX.
Terraform - Prerequisite
- To use terraform, go to here to install and learn Terraform by Hashicorp.
- For homebrew user:
brew install terraform
- For homebrew user:
- AWS user (programmatic access) with AdministratorAccess. Copy the
access key IDandsecret access keyto setup theawsclilater. - Install
awsclifrom official guide- For homebrew user:
brew install awscli
- For homebrew user:
- Using the
access key IDandsecret access key, follow this guide to step a new profile for yourawscli.
Terraform - Guide
- After
git clone, change directory to…
Reference:
Top comments (0)