We have all been there before, well at least I have, multiple times now in fact. Send off your old faithful CloudFormation template that creates a few EC2 instances and a few seconds later hit with the: “API: ec2:RunInstances Not authorized for images: ‘ami-12345abcdef’ …”. I hope for some of you out there this hasn’t occurred and hopefully after reading this will never occur. For others enjoy some coffee and hear my struggles of keeping my stack AMI values up to date.
In this article, I am going to share with you an automation trick one of my colleagues enlightened me too, which involves a crafty way of always retrieving the latest AMI-ID published by AWS to be used for my deployments. By using AWS Systems Manager Parameter Store, I can use unique identifiers to specifically select my desired AMI-ID.
What is an AMI?
An Amazon Image ID or AMI is a template that encompasses the required software to launch an EC2 instance. From theses AMI’s you can launch a instance, and what is created is an instance of that AMI, so a running copy. When selecting an AMI, you can select either an:
- AWS Provided AMI — supported and maintained by AWS.
- Paid AMI — Purchase a developer provided AMI on AWS MarketPlace.
- Community AMI — AMI’s shared with the community or public.
- Custom AMI — Using existing AMI’s customize it further with required application and security software. Creating a Golden-Image to launch instances from. You can also create and sell AMI’s.
Any time you create an EC2 instance whether it be by console, Command Line Interface (CLI), or infrastructure as code (IaC) you must specify an AMI at launch.
Some common characteristics to look for when selecting an AMI:
- AWS Region
- Operating System (OS)
- Architecture (32 vs. 64 bit)
Note: For Linux AMI’s you can also define a virtualization type depending on what the needs are. Linux offers hardware virtual machines (HVM) as well as paravirtual (PV). See here for more information regarding Linux virtualization types.
What is Infrastructure as code?
Infrastructure as code or IaC is a process of provisioning and managing cloud resources through code. To use IaC, a configuration (blueprint) template file needs to be defined that contains instructions for the exact resources and specifications to build and deploy projects. With the building of these IaC templates, many benefits can be achieved such as improved consistency, version control, and faster time to deployment. Some popular IaC services are:
The Problem at hand:
I built and continue to build a lot of IaC templates that create various EC2 instances that range in OS flavors and releases. A lot of these templates are used in workshops I have created to highlight different security concepts and workflows. A work colleague of mine was using one of the previous workshops I built and if you are familiar with the infamous Murphy’s law [anything that can go wrong, will go wrong] when it comes to live demos then as you can guess he ran into an error stating “the image id ‘ami-12345abcdef’ does not exist”. This error happens periodically for a few reasons. AWS performs OS updates to their AMI’s they offer to customers. Each AMI is available to a particular region as well. So when these updates occur a new AMI ID is allocated while the older one is set to be deprecated.
Now, this is something that has occurred to me before, so I already knew the solution was to manually go into AWS EC2 and just grab the latest AMI-ID for that specific OS and update the CloudFormation stack with the value and I am good to go for a brief while. However, I was building multiple different workshops each with its own needs. This became a weekly event for me to ensure all my templates had updated and accurate AMI’s, this grew to become a pain as the templates scaled.
In my previous iterations, I chose to leverage a technique that involved mappings. In CloudFormation, this allowed me to set values based on a specific AWS region it was to be deployed. In my case, I could define a region(s) with AMI-ID values for each OS that was needed in my case. AWS has 27 regions, I covered just 4 using this approach. In addition, each time I updated the AMI-ID values for each respective OS in each respective region I was also subjected to potential human error with copy and paste. A lot of different things could go wrong and worst of all, it was a monotonous process that went directly against the DRY method or “Don’t Repeat Yourself” method.
Solution:
The very same work colleague that had to navigate the template error also later enlightened me on a better approach when deploying different OS types and as well as easier management scaling availability for a global audience. He suggested I update my template and abandon the mappings approach in lieu for Systems Manager Parameter Store.
With SSM Parameter Store, it turns out that it hides a gold mine of strings that can be queried to return the latest AMI image ids for different OS and releases. All that was needed on my end was to locate the needed string values. I navigated to SSM Parameter Store, selected the option for “Public Parameters” and selected the service for which I am looking for values to leverage.
Once I acquired my OS-specific parameter string values, all that is left is to update my IaC template to reflect this new approach. In the example below, I created a parameter for each OS that was needed, in this case an Ubuntu 18 AMI and an Amazon Linux AMI. Defined in the example is a default value for each respective parameter. For each OS, I applied the value acquired from my Systems Manager Parameter Store search.
Description: A template to deploy a Linux & Ubuntuinstance.
#High lights SSM parameter store public AMI-ID strings.
Parameters:
#CFT Parameter created with default string for SSM AMI string.
LatestUbuntuAmiId:
Type: 'AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>'
Default: '/aws/service/canonical/ubuntu/server/18.04/stable/20220810/amd64/hvm/ebs-gp2/ami-id'
Description: DO NOT CHANGE THIS VALUE
Resources:
#Amazon Linux Resource created with resolve string for SSM AMI string.
AmazonLinux:
Type: AWS::EC2::Instance
Properties:
ImageId: resolve:ssm:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
InstanceType: t3.micro
Tags:
- Key: Name
Value: AmazonLinux
- Key: Project
Value: SSM
#Ubuntu Resource via CFT input parameter to link for SSM AMI string.
UbuntuLinux:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref LatestUbuntuAmiId
InstanceType: t3.micro
Tags:
- Key: Name
Value: Ubuntu
- Key: Project
Value: SSM
https://gist.github.com/JustinDPerkins/09001a21f974432ecb57d7c29daa7a25
Conclusion:
Some struggles do come to an end. I have yet to see the issue recur now for a few months and it is now a method I employ on all new templates I create that require EC2 instances. I encourage anyone who is currently developing or planning on developing to employ the AMI-ID retrieval using SSM parameter store. Using this method will help prevent you from dealing with failed deployments and that irritating AMI not found error. Last, I want to thank my colleague Raphael Bottino for shedding light on this awesome trick and saving me countless future headaches, so make sure to give him a follow as well.
Image Credit:
Unsplash- Ankush Minda - https://miro.medium.com/max/640/0*Yw785vmT_KD5Yx1S
Top comments (0)