TL;DR
I created a new CloudFormation resource in the AwsCommunity organization with Python, created a new Troposphere resource for it, and now can use it in production.
The why
AWS CloudFormation (CFN) is my favorite Infrastructure as Code (IaC) service to use with AWS. However, some resources might not have a CloudFormation equivalent created by the respective product team, for various possible reasons.
So then, you have to either use another IaC tool (i.e. Ansible), create a CustomResource (lambda function invoked by CFN), or you could create your own AWS Resource and publish it for everyone else on AWS to use, with the AWS CloudFormation Registry
I wanted the ability to create ScheduledActions, resources managed by the ApplicationAutoscaling service, but there wasn't a resource for that. So I created it.
The how
The CFN team has created a set of tools and libraries in different languages to allow people to author their own resources.
I created a few myself before, and published these out of my own AWS Accounts. But this time instead of doing this all on my own, I had the opportunity to contribute to the Open Source community driven repository on GitHub. Great place to see what already existed, and have other people actually help and review my code.
If you are used the workflows of CloudFormation resources provisioning and the different stages it goes through, this is a very painless process. And to make the whole process even less so painful, you can (and definitely should) run contract testing locally using the sam cli which will validate that your code passes all the different use-cases it should.
Publishing your resource
As mentioned, being a seasoned CloudFormation user, creating a CFN template to publish my resource and create a StackSet to publish it in all AWS regions isn't difficult, but it can be very daunting.
Joining the AWS Community driven repository will remove that concern away from you as the friendly & helpful team at AWS have automated the whole process for you, and will be publishing your resource as part of the AwsCommunity "Organization".
Using the resource
Once the resource is published on AWS, you need only to activate it in your account. This will require you to create an IAM role to allow the resource to perform actions on your behalf. You can also, optionally, have logging output sent to your AWS Account, which can help with your audits and security posture.
After you have activated the resource, you can start using it in your CloudFormation templates, like any other resource. The resource will return attributes, so you can use functions like Fn::GetAtt, Fn::Ref or Fn::Sub and so on.
Here is a snippet of how the resource would be used in a CloudFormation template
AWSTemplateFormatVersion: "2010-09-09"
Description: Template to test AwsCommunity::ApplicationAutoscaling::ScheduledAction
Parameters:
ScheduledActionName:
Type: String
Default: cfn-testing-resource
ServiceNamespace:
Type: String
AllowedValues: [
"ecs",
"elasticmapreduce",
"ec2",
"appstream",
"dynamodb",
"rds",
"sagemaker",
"custom-resource",
"comprehend",
"lambda",
"cassandra",
"kafka",
"elasticache",
"neptune"
]
ScalableDimension:
Type: String
AllowedValues: [
"ecs:service:DesiredCount",
"ec2:spot-fleet-request:TargetCapacity",
"elasticmapreduce:instancegroup:InstanceCount",
"appstream:fleet:DesiredCapacity",
"dynamodb:table:ReadCapacityUnits",
"dynamodb:table:WriteCapacityUnits",
"dynamodb:index:ReadCapacityUnits",
"dynamodb:index:WriteCapacityUnits",
"rds:cluster:ReadReplicaCount",
"sagemaker:variant:DesiredInstanceCount",
"custom-resource:ResourceType:Property",
"comprehend:document-classifier-endpoint:DesiredInferenceUnits",
"comprehend:entity-recognizer-endpoint:DesiredInferenceUnits",
"lambda:function:ProvisionedConcurrency",
"cassandra:table:ReadCapacityUnits",
"cassandra:table:WriteCapacityUnits",
"kafka:broker-storage:VolumeSize",
"elasticache:replication-group:NodeGroups",
"elasticache:replication-group:Replicas",
"neptune:cluster:ReadReplicaCount"
]
MinCapacity:
Type: Number
MinValue: 0
MaxCapacity:
Type: Number
MinValue: 0
ScheduleExpression:
Type: String
Description: the cron(), rate() or at() expression.
EndTime:
Type: String
Default: none
Description: When using cron() or rate(), timestamp of when the rule expires.
StartTime:
Type: String
Default: none
Description: When using cron() or rate(), timestamp of when the rule starts.
TimeZone:
Type: String
Default: "UTC"
ResourceId:
Type: String
Description: The Scalable Target ID.
Conditions:
NoEndTime:
!Equals [ !Ref EndTime, "none" ]
NoStartTime:
!Equals [ !Ref StartTime, "none" ]
Resources:
ScheduledActionForScalableTarget:
Type: AwsCommunity::ApplicationAutoscaling::ScheduledAction
Properties:
ScheduledActionName: !Ref ScheduledActionName
ServiceNamespace: !Ref ServiceNamespace
ScalableDimension: !Ref ScalableDimension
Schedule: !Ref ScheduleExpression
ScalableTargetAction:
MinCapacity: !Ref MinCapacity
MaxCapacity: !Ref MaxCapacity
StartTime: !If
- NoStartTime
- !Ref AWS::NoValue
- !Ref StartTime
EndTime: !If
- NoEndTime
- !Ref AWS::NoValue
- !Ref StartTime
Timezone: !Ref TimeZone
ResourceId: !Ref ResourceId
Creating a troposphere resource and integration to ECS Compose-X
If you have read my earlier posts or heard me talk with Corey on Screaming in the Cloud, I am the author of a tool called ECS Compose-X. It's designed to support all the docker compose features, and allow for extensions that make it easier to deploy to AWS.
Because ECS Compose-X uses Troposphere, I was able to create a very light and simple python library(https://github.com/JohnPreston/troposphere-awscommunity-applicationautoscaling-scheduledaction) to distribute the resource for other Troposphere users to re-use.
Naturally, as autoscaling was already implemented in the project, this was a natural addition to the x-scaling services feature. This particular resource allows to plan for scaling your ECS service when you need to, and scale back down accordingly.
Conclusion
Creating AWS Resource for CloudFormation might seem like something like you shouldn't have to do, but I see it as a great way to contribute back to the community and to a service I love. And doing so, I hope, will help people who might trying to achieve the same thing.
But it is not limited to AWS resources: MongoDB, NewRelic, DataDog and other AWS partners have taken advantage of this capability to publish their own resources, allowing AWS customers to rely on AWS CloudFormation to provision their resources.
I am not a CDK user, but one could also now create a CDK extension to provision that resource in their code.
Big shoutout to the AWS CloudFormation team (@ericzbeard, @kddejong) & contributors for having helped me out and look forward to seeing more exciting resources and projects coming up!
Top comments (1)
Really helpful post