loading...

AWS CloudFormation, Swagger, and Broken Dreams

forstmeier profile image John Forstmeier ・2 min read

AWS is the core infrastructure resource that we use at my work and we're using it to build our platform as a microservices architecture. There are lots of interesting features provided by AWS and one of the ones we use literally every day is CloudFormation.

All of our individual microservices use CloudFormation templates to launch their necessary resources. I am a fan of these single, cohesive files that bind everything together - it's sort of a "design scaffold" in my mind that we then put our code into. As much as possible, our team tries to keep things contained in these files.

Another cool resource that I was recently introduced to is Swagger (a.k.a. OpenAPI but "Swagger" is just such a better name in my humble opinion). Swagger is essentially a uniformed way to define an API in a single file. Having this stand-alone file is pretty cool because it can then be used in a bunch of different places (e.g. generating documentation, building tests, etc.) outside of strictly constructing the production API.

AWS pretty niftily supports using a Swagger definition to create your API - basically you replace the AWS::ApiGateway::Resource and AWS::ApiGateway::Method resources in your CloudFormation template file with Swagger which is pretty cool.

Sort of.

Now in our case, because we're building microservices we're constantly launching and tearing down individual resource stacks so important resource references (e.g. Lambda ARNs) are changing frequently.

This poses a problem for the two options available in AWS for including Swagger in your template.

  • Option 1: in-line where the Swagger file contents are directly included in the Body field of the AWS::ApiGateway::RestApi template resource
    • pros: the "Ref' statements can be used to catch dynamic ARN/resource references, flexibility is maintained in between stack launches/teardowns
    • cons: the Swagger contents can't be dropped anywhere else since they're hard-coded into the template, the benefits of an independent Swagger file are lost
  • Option 2: external S3 file where the Swagger file is stored in S3 and referenced through the BodyS3Location field on AWS::ApiGateway::RestApi
    • pros: the Swagger file is separate, this is ideal for sharing and using in other services
    • cons: the ARN references must be hard-coded in the Swagger file which assumes those resources are permanently available

Neither option is particularly good since they sort of cut off the left arm to make the right arm stronger or vice versa. The current workaround I've implemented is to provide a small Python script to insert the Swagger definition. Basically, it takes the CloudFormation template file contents and inserts the Swagger JSON file contents into the Body field at the build stage of our CI pipeline.

It's not an ideal solution but hopefully AWS provides something better soon!

AWS FRUSTRATION

Discussion

markdown guide
 

This is where AWS Serverless Application Model (or SAM) is extremely useful. Think of it as CloudFormation with easier Lambda function definitions and API Gateways. The best part is that it can be deployed like any other CloudFormation document, it's just an extra transform layer on top of it.

So you can define APIs using Swagger if you'd like, and you can use normal CloudFormation intrinsic functions in those swagger definitions. So when you add your x-amazon-integration details, you can do a !Sub MyLambda.Arn in its place.

 

I'm certainly late to this party, but check out this link:

medium.com/@nabtechblog/integratin...

The template formatting is a bit hard to read, but it's a hybrid of the approaches listed. You include the body using AWS Sam in the definitionBody and the FN::Transform to include and process the AWS stack variables from the S3 file. I think this is pretty similar to what Turner is suggesting, but the link has an example of it working.

 

Could you not put have the Swagger inline to gain the advantage of version control and Cf referencing; as part of the post API Gateway complete phase trigger a push of the SWagger content to S3 for sharing? I feel like this is a not-trivial use case and has a solution.

 

That's definitely an option and essentially the reverse of what we implemented - keeping the file in S3 and then adding it to the template instead of keeping the Swagger content in the template and then exporting it to S3 (I assume via an AWS::CustomResource or something).

 

Sam eliminates this problem. Tried, tested and deployed.