The recent addition to the AWS Step Functions service sparked many conversations in the AWS serverless community. This is very much understandable as having the option to integrate AWS Step Functions with almost every AWS service directly is like having superpowers.
This blog post will walk you through creating direct integration between AWS Step Functions and Amazon API Gateway (REST APIs). By utilizing Step Functions and API Gateway VTL transformations, the architecture will allow you to create the whole APIs without deploying any AWS Lambda functions at all!
Let us dive in.
All the code examples will be written in TypeScript. You can find the GitHub repository with code from this blog here.
The API
Creating the API Gateway REST API with AWS CDK is pretty much painless.
The first step is to create the RestApi resource.
import * as apigw from "@aws-cdk/aws-apigateway";
// Stack definition and the constructor ...
const API = new apigw.RestApi(this, "API", {
defaultCorsPreflightOptions: {
/**
* The allow rules are a bit relaxed.
* I would strongly advise you to narrow them down in your applications.
*/
allowOrigins: apigw.Cors.ALL_ORIGINS,
allowMethods: apigw.Cors.ALL_METHODS,
allowHeaders: ["*"],
allowCredentials: true
}
});
Since our example will be using the POST HTTP method, I've opted into specifying the defaultCorsPreflightOptions. Please note that this property alone does not mean are done with CORS. The defaultCorsPreflightOptions and the addCorsPreflight on the method level create an OPTIONS method alongside the method you initially created. This means that the API Gateway service will handle OPTIONS part of the CORS flow for you, but you will still need to return correct headers from within your integration. We will address this part later on.
The second step is to create a resource which is nothing more than an API route.
Let us create a route with a path of /create.
const API = // the API definition from earlier.
const createPetResource = API.root.addResource("create");
We have not yet integrated our API path with any HTTP verb nor AWS service. We will do this later on after defining the orchestrator powering our API - the AWS Step Functions state machine.
The Step Functions state machine
Since this blog post is not a tutorial on Step Functions our Step Functions state machine will be minimalistic.
Here is how one might define such a state machine using AWS CDK.
import * as sfn from "@aws-cdk/aws-stepfunctions";
import * as logs from "@aws-cdk/aws-logs";
// Previously written code and imports...
const APIOrchestratorMachine = new sfn.StateMachine(
this,
"APIOrchestratorMachine",
{
stateMachineType: sfn.StateMachineType.EXPRESS,
definition: new sfn.Pass(this, "PassTask"),
logs: {
level: sfn.LogLevel.ALL,
destination: new logs.LogGroup(this, "SFNLogGroup", {
retention: logs.RetentionDays.ONE_DAY
}),
includeExecutionData: true
}
}
);
Since we are building synchronous API, I've defined the type of the state machine as EXPRESS. If you are not sure what the difference is between the EXPRESS and STANDARD (default) types, please refer to this AWS documentation page.
In a real-world scenario, I would not use the EXPRESS version of the state machine from the get-go. The EXPRESS type is excellent for cost and performance, but I find the "regular" state machine type better for development purposes due to rich workflow visualization features.
As I eluded earlier, the definition of the state machine is minimalistic. The PassTask will return everything from machine input as the output.
I encourage you to give it a try and extend the definition to include calls to different AWS services. Remember that you most likely do not need an AWS Lambda function to do that.
The Integration
Defining direct integration between Amazon API Gateway and AWS Step Functions will take up most of our code. The Amazon API Gateway is a feature-rich service. It exposes a lot of knobs one might adjust to their needs. We will distill all the settings to only the ones relevant to our use case.
const createPetResource = API.root.addResource("create");
createPetResource.addMethod(
"POST",
new apigw.Integration({
type: apigw.IntegrationType.AWS,
integrationHttpMethod: "POST",
uri: `arn:aws:apigateway:${cdk.Aws.REGION}:states:action/StartSyncExecution`,
options: {}
}),
// Method options \/. We will take care of them later.
{}
);
The most important part of this snippet is the uri property. This property tells the API Gateway what AWS service to invoke whenever the route is invoked. The documentation around uri is, in my opinion, not easily discoverable. I found the Amazon API Gateway API reference page helpful with learning what the uri is about.
With the skeleton out of the way, we are ready to dive into the options parameter.
The integration options
Our integration tells the Amazon API Gateway service to invoke a Step Functions state machine, but we never specified which one!
This is where the requestTemplates property comes in handy. To target the APIOrchestratorMachine resource that will power our API, the ARN of that state machine has to be forwarded to the Step Functions service.
const APIOrchestratorMachine = // state machine defined earlier...
const createPetResource = API.root.addResource("create");
createPetResource.addMethod(
"POST",
new apigw.Integration({
// other properties ...
options: {
passthroughBehavior: apigw.PassthroughBehavior.NEVER,
requestTemplates: {
"application/json": `{
"input": "{\\"actionType\\": \\"create\\", \\"body\\": $util.escapeJavaScript($input.json('$'))}",
"stateMachineArn": "${APIOrchestratorMachine.stateMachineArn}"
}`
}
}
})
);
The requestTemplates is a key: value structure that specifies a given mapping template for an input data encoding scheme – I assumed that every request made to the endpoint would be encoded as application/json.
The data I'm forming in the request template must obey the StartSyncExecution API call validation rules.
Let us tackle AWS IAM next.
Our definition tells the Amazon API Gateway which state machine to invoke (via the requestTemplates property). It is not specifying the role API Gateway could assume so that it has permissions to invoke the state machine.
Enter the credentialsRole parameter. The Amazon API Gateway service will use this role to invoke the state machine when specified.
Luckily for us, creating such a role using AWS CDK is not much of a hassle.
import * as iam from "@aws-cdk/aws-iam";
const APIOrchestratorMachine = // state machine definition
const invokeSFNAPIRole = new iam.Role(this, "invokeSFNAPIRole", {
assumedBy: new iam.ServicePrincipal("apigateway.amazonaws.com"),
inlinePolicies: {
allowSFNInvoke: new iam.PolicyDocument({
statements: [
new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
actions: ["states:StartSyncExecution"],
resources: [APIOrchestratorMachine.stateMachineArn]
})
]
})
}
});
const createPetResource = API.root.addResource("create");
createPetResource.addMethod(
"POST",
new apigw.Integration({
// Properties we have specified so far...
options: {
credentialsRole: invokeSFNAPIRole
}
}),
// Method options \/. We will take care of them later.
{}
);
The role is allowed to be assumed by Amazon API Gateway (the assumedBy property) service and provides permissions for invoking the state machine we have created (the states:StartSyncExecution statement).
Since our /create route accepts POST requests, the response must contain appropriate CORS headers. Otherwise, users of our API might not be able to use our API.
Just like we have modified the incoming request to fit the StartSyncExecution API call validation rules (via the requestTemplates parameter), we can validate the response from the AWS Step Functions service.
This is done by specifying the integrationResponses parameter. The integrationResponses is an array of response transformations. Each transformation corresponds to the status code returned by the integration, not the Amazon API Gateway service.
const createPetResource = API.root.addResource("create");
createPetResource.addMethod(
"POST",
new apigw.Integration({
// Properties we have specified so far...
options: {
integrationResponses: [
{
selectionPattern: "200",
statusCode: "201",
responseTemplates: {
"application/json": `
#set($inputRoot = $input.path('$'))
#if($input.path('$.status').toString().equals("FAILED"))
#set($context.responseOverride.status = 500)
{
"error": "$input.path('$.error')",
"cause": "$input.path('$.cause')"
}
#else
{
"id": "$context.requestId",
"output": "$util.escapeJavaScript($input.path('$.output'))"
}
#end
`
},
responseParameters: {
"method.response.header.Access-Control-Allow-Methods":
"'OPTIONS,GET,PUT,POST,DELETE,PATCH,HEAD'",
"method.response.header.Access-Control-Allow-Headers":
"'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent'",
"method.response.header.Access-Control-Allow-Origin": "'*'"
}
}
]
}
}),
// Method options \/. We will take care of them later.
{}
);
The above integrationResponse specifies that if the AWS Step Functions service returns with a statusCode of 200 (controlled by the selectionPattern and not the statusCode), the transformations within the responseTemplates and responseParameters will be applied to the response and the Amazon API Gateway will return with the statusCode of 201 to the caller.
Take note of the responseParameters section where we specify CORS-related response headers. This is only an example. In a real-world scenario, I would not recommend putting * as the value for Access-Control-Allow-Origin header.
The mapping template is a bit involved.
The following block of code
#if($input.path('$.status').toString().equals("FAILED"))
is responsible for checking if a given execution failed. This condition has nothing to do with checking if AWS Step Functions service failed.
The conditions checks whether given state machine execution failed or not.
For AWS Step Functions service failure, we need to create another mapping template to handle such a scenario. This is out of the scope of this blog post.
The method options
The method options (AWS CloudFormation reference) contain properties that allow you to specify request/response validation, which responseParameters are allowed for which statusCode (very relevant for us), and various other settings regarding Amazon API Gateway route method.
In the previous section, we have specified the responseParameters so that the response surfaced by Amazon API Gateway contains CORS-related headers.
To make the responseParameters fully functional, one needs to specify correct methodResponse.
createPetResource.addMethod(
"POST",
new apigw.Integration({
// Properties we have specified so far...
options: {
integrationResponses: [
{
// Other `integrationResponse` parameters we have specified ...
responseParameters: {
"method.response.header.Access-Control-Allow-Methods":
"'OPTIONS,GET,PUT,POST,DELETE,PATCH,HEAD'",
"method.response.header.Access-Control-Allow-Headers":
"'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent'",
"method.response.header.Access-Control-Allow-Origin": "'*'"
}
}
]
}
}),
{
methodResponses: [
{
statusCode: "201",
// Allows the following `responseParameters` be specified in the `integrationResponses` section.
responseParameters: {
"method.response.header.Access-Control-Allow-Methods": true,
"method.response.header.Access-Control-Allow-Headers": true,
"method.response.header.Access-Control-Allow-Origin": true
}
}
]
}
);
In my opinion, the resource definition would be less confusing if the methodResponses (and the whole method options section) would come before the integration. I find it a bit awkward that we first have to specify the responseParameters in the request and THEN specify which ones can be returned in the methodResponse section.
Bringing it all together
Phew! That was a relatively large amount of code to write. Luckily, engineers contributing to the AWS CDK are already preparing an abstraction for us to make this process much more manageable. Follow this PR to see the progress made.
Here is all the code that we have written together. The code is also available on my GitHub.
const API = new apigw.RestApi(this, "API", {
defaultCorsPreflightOptions: {
/**
* The allow rules are a bit relaxed.
* I would strongly advise you to narrow them down in your applications.
*/
allowOrigins: apigw.Cors.ALL_ORIGINS,
allowMethods: apigw.Cors.ALL_METHODS,
allowHeaders: ["*"],
allowCredentials: true
}
});
new cdk.CfnOutput(this, "APIEndpoint", {
value: API.urlForPath("/create")
});
const APIOrchestratorMachine = new sfn.StateMachine(
this,
"APIOrchestratorMachine",
{
stateMachineType: sfn.StateMachineType.EXPRESS,
definition: new sfn.Pass(this, "PassTask"),
logs: {
level: sfn.LogLevel.ALL,
destination: new logs.LogGroup(this, "SFNLogGroup", {
retention: logs.RetentionDays.ONE_DAY
}),
includeExecutionData: true
}
}
);
const invokeSFNAPIRole = new iam.Role(this, "invokeSFNAPIRole", {
assumedBy: new iam.ServicePrincipal("apigateway.amazonaws.com"),
inlinePolicies: {
allowSFNInvoke: new iam.PolicyDocument({
statements: [
new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
actions: ["states:StartSyncExecution"],
resources: [APIOrchestratorMachine.stateMachineArn]
})
]
})
}
});
const createPetResource = API.root.addResource("create");
createPetResource.addMethod(
"POST",
new apigw.Integration({
type: apigw.IntegrationType.AWS,
integrationHttpMethod: "POST",
uri: `arn:aws:apigateway:${cdk.Aws.REGION}:states:action/StartSyncExecution`,
options: {
credentialsRole: invokeSFNAPIRole,
passthroughBehavior: apigw.PassthroughBehavior.NEVER,
requestTemplates: {
"application/json": `{
"input": "{\\"actionType\\": \\"create\\", \\"body\\": $util.escapeJavaScript($input.json('$'))}",
"stateMachineArn": "${APIOrchestratorMachine.stateMachineArn}"
}`
},
integrationResponses: [
{
selectionPattern: "200",
statusCode: "201",
responseTemplates: {
"application/json": `
#set($inputRoot = $input.path('$'))
#if($input.path('$.status').toString().equals("FAILED"))
#set($context.responseOverride.status = 500)
{
"error": "$input.path('$.error')",
"cause": "$input.path('$.cause')"
}
#else
{
"id": "$context.requestId",
"output": "$util.escapeJavaScript($input.path('$.output'))"
}
#end
`
},
responseParameters: {
"method.response.header.Access-Control-Allow-Methods":
"'OPTIONS,GET,PUT,POST,DELETE,PATCH,HEAD'",
"method.response.header.Access-Control-Allow-Headers":
"'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent'",
"method.response.header.Access-Control-Allow-Origin": "'*'"
}
}
]
}
}),
{
methodResponses: [
{
statusCode: "201",
responseParameters: {
"method.response.header.Access-Control-Allow-Methods": true,
"method.response.header.Access-Control-Allow-Headers": true,
"method.response.header.Access-Control-Allow-Origin": true
}
}
]
}
);
Next steps
To make sure this blog post is not overly long, I've omitted some of the code I would usually write in addition to what we already have.
Here are some ideas what you might want to include in the integration definition:
- Add request and response validation via API models.
- Amend the state machine definition to leverage the SDK service integrations.
- Add more entries in the
integrationResponses/methodResponseserrors from the AWS Step Functions service itself.
Closing words
Integrating various AWS services directly is a great way to save yourself from one of the most significant liabilities in serverless systems – AWS Lambdas code. You might be skeptical at first, as writing those services together is usually not the easiest thing to do, but believe me, the upfront effort is worth it.
I hope that you found this blog post helpful.
Thank you for your time.
For questions, comments, and more serverless content, check out my Twitter – @wm_matuszewski
Top comments (0)