DEV Community: Andrew May

Faster, Cheaper - AWS Graviton 2

Andrew May — Sat, 20 Nov 2021 20:29:55 +0000

Graviton: Hypothetical quantum of gravity, a Marvel character, and now an ARM based CPU from AWS

There have been multiple attempts to bring ARM based CPUs into the data center in recent years due to their perceived efficiency compared to x86_64 designs. Generally they have been seen as lower performance designs suitable for running webservers and other light workloads.

In 2018, AWS released Graviton 1 CPUs that powered the A1 instance family. These were cheaper than Intel/AMD instance types, but also lower performing. I think I launched one to take a look, but never really considered using them again.

In 2019, AWS released Graviton 2 CPUs, and these are a very different beast. AWS claims that they are up to 40% faster than the equivalent Intel instance types, and prices them 20% less. Instead of being limited to one instance family, they are available in a whole range (T4g, C6g, M6g, R6g, X2gd) and they're no longer limited to EC2 instances.

In many ways you can think of what AWS is doing with Graviton to be similar to what Apple is doing with their M1 line of chips. Both are using custom ARM designs developed in house (by companies they bought) and are redefining what we expect from ARM based CPUs.

Supported Services

Some of the services (the list is growing over time, and with re:Invent on the horizon this could well grow in the next couple of weeks) that support Graviton 2:

EC2 - Run servers with a range of instance types
Lambda - Run serverless applications in a range of languages
ECS/EKS - ARM based EC2 AMIs are available to work with both container platforms in AWS, and it's hopefully just a matter of time before Fargate support appears

Update: 11/23 - Graviton 2 support for AWS Fargate announced before re:Invent! Currently this does not include Fargate Spot.

RDS - Supports MySQL, Postgres and Aurora
ElastiCache - Redis and Memcached
OpenSearch - AWS's fork of ElasticSearch
EMR - Elastic Map Reduce
CodeBuild - Build code to run on ARM on Graviton instances

With a lot of these services you will need to be on a recent version to be able to use Graviton instances. For example you need to use a recent version of MySQL 8 to select Graviton instance types in RDS, and the very latest version to use the burstable T4g instance family.

Where to start using Graviton 2?

If you are starting a new project in AWS, and you're using some of the services listed above, I would recommend starting with Graviton 2 instances where possible. That will give you cost savings and performance benefits from day 1. In some cases AWS is already defaulting the instance type to a Graviton 2 processor in the AWS console.

This does make some assumptions:

You're using services/applications that can run on Linux
The runtimes and tools you need are available for ARM (true of pretty much all major languages and Open Source tools)
You're willing to accept there may be some extra work in compiling/packaging applications to run on a different architecture than you're developing on (unless you're on a new Mac)

APIs are (generally) easy

For services where you interact with an API and don't run your own code (RDS, ElastiCache, OpenSearch), consumers of the API don't care whether they're calling a service running on x86_64 or ARM, so these are often the easiest to switch.

If you're already running some of these services, you may find that you need to upgrade the version and this can be disruptive depending upon how far from the latest version you are and the type of service. But perhaps this is the time to do the upgrade you've been putting off for the last 6 months.

Open Source applications

Linux has supported ARM for a long time (doesn't everyone have a stack of Raspberry Pis sitting around?), and if you can install a package on an x86_64 machine you can almost certainly install it on Graviton 2 instances.

Nginx example

This snippet of CloudFormation launches an EC2 instance running Amazon Linux 2, installs Nginx then starts the service:

  NginxInstance:
    Type: AWS::EC2::Instance
    Properties:
      IamInstanceProfile: !Ref InstanceProfile
      ImageId: !Ref AmiParameter
      InstanceType: !Ref InstanceType
      SecurityGroups:
        - !Ref SecurityGroup
      Tags:
        - Key: Name
          Value: !Ref 'AWS::StackName'
      UserData:
        Fn::Base64: !Sub |
          #!/bin/bash -xe
          amazon-linux-extras install -y nginx1
          systemctl start nginx

This works identically whether the AMI/Instance Type are for x86_64 or for Graviton.

Custom Applications

For your own applications, whether running on EC2 or Lambda, how much work it is to run on Graviton 2 will depend upon the language:

Languages: No changes necessary

Java
.NET Core, .NET 5 or 6
NodeJS

All of these run on top of a runtime and your code is either not compiled (NodeJS) or compiled into an intermediary form that works across platforms.

Java in particular has always had the concept of build once, run anywhere, and it's over 20 years ago that I first compiled Java code on an Intel/Windows machine and ran it on a Sun/Solaris server.

Languages: Repackage (sometimes)

Python
Ruby

These languages can run on multiple platforms, often without changing, but may make use of some packages that use native code. This means that you may need to re-package your application for ARM, and some packages may need to be compiled if they're not available pre-compiled in the package manager.

I'm not that familiar with PHP, but I think it may fall into this list as well. This blog post talks about some of the work AWS has done with the PHP community to improve the performance on ARM64 CPUs like Graviton 2.

Languages: Recompile

Golang
Rust
C, C++

These languages will need to be recompiled to run on a Graviton instance, using an ARM64 server to compile (e.g. in CodeBuild), or cross-compiling from another machine.

Containers

Although containers are considered portable (run the same container many different places), they are architecture specific and you cannot run the same container on x86_64 and ARM architectures. Even if your application code is in a language like Java, it's going to reply on a JVM being installed in the container that uses native code.

Containers can either be built on the target architecture, or there are options for building multiple architectures using emulation.

AWS ECR has added support for multi-architecture container images. This allows you to store different architectures in the same repository and when you pull a multi-architecture tag it picks to correct container to download based upon the client architecture.

At the moment the UI for ECR doesn't do a good job of displaying architectures, and I'm not sure how lifecycle policies are affected.

Lambda functions

Depending upon the language you're using, it may be as easy as changing the Architecture value to switch from x86_64 to ARM64 (i.e. Graviton 2). As discussed above, if you're using native packages or code you will need to repackage or recompile to switch.

One particularly nice feature is the ability to gradually shift traffic from one architecture to another by using weights to distribute traffic between two versions of the function. This would allow you to send a small percentage of your traffic to the Graviton 2 version of the function until you are satisfied that there are no errors and the performance is satisfactory (hopefully better).

Graviton Challenge

AWS is running the Graviton Challenge which walks you through migrating an application to Graviton 2 over 4 days. Some parts of the challenge have finished, but you can still run a single T4g.micro instance for free until 12/31.

Looking forward to re:invent

The re:invent catalog contains a number of sessions related to Graviton and custom silicon:

CMP301: The journey of silicon innovation at AWS
CMP213: Lessons learned from customers who have adopted AWS Graviton
AMZ301: How Amazon migrated a large ecommerce platform to AWS Graviton
CMP214: Run containerized workloads on AWS Graviton2 for better price-performance
DEM002-S: Workload analysis on Arm-based AWS Graviton2 processors

I'd also be very surprised if there aren't some more announcements in this area.

GCP Cloud Digital Leader, a certification in search of an identity

Andrew May — Thu, 09 Sep 2021 00:03:36 +0000

I recently took and passed the exam for the new GCP Cloud Digital Leader certification, and I thought I'd share a few thoughts.

The certification is the first "Foundation" Level certification for GCP, but it seems to sit there uneasily.

Similar Certifications

I've previously taken both the AWS Cloud Practitioner and Microsoft Azure Fundamentals certifications (along with a bunch of other AWS certs), and these both provide fairly easy introductions to cloud concepts and a high level overview of the core services in the platform.

Digital Leader?

The name of the certification seems to be an indication of the aspirations Google had for this new certification, and the training material they provide makes it seem that the certification is squarely aimed at decision makers who are considering moving to the cloud but either need convincing, or need ammunition to convince others.

The four training courses in their learning path are titled:

Introduction to Digital Transformation with Google Cloud
Innovating with Data and Google Cloud
Infrastructure and Application Modernization with Google Cloud
Understanding Google Cloud Security and Operations

The courses take about 8-10 hours to go through and have a number of simple tests at the end of sections. In the videos you'll hear a lot of reasons for why you should move to the Cloud, and GCP in particular, but you'll learn practically nothing about the individual services available in GCP. Some of the services are mentioned by name, but with no detail - for example Cloud SQL gets a mention, but no details about which RDBMS engines it supports.

The 10 practice test questions they provide might lead you to think these videos have prepared you reasonably well for the exam, as they ask you to chose between adding resources in your datacenter or creating them in a "public cloud platform" (wink, wink, we mean GCP).

Hint: the answer is always moving to the cloud!

There are actually a number of these types of questions in the exam, but it feels like they got part way and then realized they had nowhere near enough questions of that type to create a certification.

Many of the other questions are of a more traditional foundation style, expecting you to select the right service to meet a particular use case. It does seem like they expect you to know every service in GCP unlike the AWS and Azure certs, but at least there are a lot less services in GCP.

Then there are questions that feel like they were pulled from an Associate Architect exam that doesn't exist, that require you to have a bit deeper understanding of how to select services, or configure IAM etc. The addition of these questions makes the exam feel more difficult as a whole than the other foundation certs, and had me doubting myself a few times.

Preparation

A group of us at Leading EDJE all decided to get this certification, so we watched the provided videos together. Fortunately before diving headlong into the exam, we did some research and found some reviews of the beta exam that made us realize we were not yet prepared.

There isn't a lot of third party training available yet because it's such a new certification. I ended up using the ExamPro training course that includes 5.5 hours of videos and practice questions that are designed to be closer to the real exam.

I also spent some time reading my Associate Cloud Engineer Study guide that I bought a while ago but never got around to using.

After that I felt pretty prepared for the exam and went ahead and booked it to take a few hours after I finished going through the ExamPro course.

The exam

GCP uses Webassessor/Kryterion for their exams, and while the process was similar for a remotely proctored exam was similar to the companies used for AWS and Azure tests, the actual pre-exam screening was made significantly more difficult because you can't use your phone to take pictures of the room and have to use your webcam - so there was a lot of trying to scan the room (and under the table) using my laptop, and in a few cases (like verifying my id) I had to take a photo with my phone, zoom in and then hold it up to my webcam.

The actual exam software and type of question was similar to the AWS Cloud Practitioner, multiple choice questions most with a single answer - none of the variety of question styles that the Azure Fundamentals exam uses. As with AWS, they ask you to take a questionnaire before telling you if you passed or failed.

Currently I just know I've passed, I'm still waiting on the official email from Google.

Some of the questions were very straightforward and I answered in seconds, others I wasn't entirely sure about. I answered all the questions in about 30 of the allotted 90 minutes.

Summing up

If you are a decision maker trying to decide whether to move to the Cloud or not, the provided GCP training videos may be of use, but I'm not convinced studying for the certification exam itself will be a great deal of use.

If you already know you're going to be using GCP, and want to learn more about the platform, then the provided training is of little use and you're probably better served by studying for the Associate Cloud Engineer certification.

Jamming with AWS

Andrew May — Sun, 22 Aug 2021 16:20:45 +0000

The upcoming AWS Summit Online (Americas) next week (August 24th-26th) will have the usual Keynotes and Sessions, but the most interesting/fun part of the summit will be the AWS Jam.

An AWS Jam is a "Capture the Flag" style event that runs over two days (on the 25th/26th August) where you complete challenges in AWS to earn points, either competing as an individual or as part of a team. The challenges vary in complexity and difficulty and can take anywhere from a few minutes to a couple of hours. If you're stuck you can get hints, but these decrease the number of points you get for completing the challenge.

Example challenges from the AWS Jam at the May 2021 summit:

At the last summit I wasn't entirely sure what an AWS Jam was, so I took part by myself and didn't have time to organize a team (I was also limited in how much time I could spend on it), but I still managed to reach 38th on the leader board by quickly completing a number of challenges at the last minute!

I found it was a great opportunity to experiment with parts of AWS I'd not used before, in particular some of the Data Science and ML challenges where you're given a pre-configured Notebook and Dataset and they work you through the steps of data analysis.

This time I'm hoping to have a team of EDJErs to compete in the challenge and see how much further we can get up the leaderboard!

Joining the Jam

In order to take part in the AWS Jam you'll need a Jam account (sign up here) and you'll also need to code for the Jam which will be available once the AWS Summit starts (so you'll need at least one person in your team to register with the summit to get the code, create a team, and then you can invite people to the Jam and direct them to your team).

The rest of the summit

There should be an interesting keynote from the VP of Machine Learning at AWS (Swami Sivasubramanian), and possibly a few new announcements, and then there's a selection of interesting sessions in all areas of AWS.

Additionally there are two hour workshops in a range of technologies (e.g. Build and deploy web applications with AWS App Runner) that you need to register for separately. They're great if that particular workshop is something you want to learn about, but won't give you experience with the range of AWS services that the Jam will.

Feature flags for Infrastructure as Code

Andrew May — Sun, 23 May 2021 18:23:58 +0000

Feature what?

Feature flags or toggles are often* used in application code to allow for rapid integration of new changes without necessarily enabling them in production.

*I have no real idea how commonly used they really are

Feature flags can prevent the dreaded long running feature branch that keeps accumulating changes that you hold off from merging because you're not ready for it to go into production in the next release.

These same challenges can apply to your Infrastructure as Code (IaC) where having changes to shared resources you want to test in a lower environment but aren't ready to promote to production can be a real problem.

Consider this scenario:

You're making changes to your network in a cloud environment.
You've got your changes in a development environment and everything seems to be working, but you can't go to production until the next maintenance window because it's going to cause an outage.
Then you suddenly have to make a different change to that same IaC template that has to go into production immediately, but you want to test it before you apply it to production.

Well, now you've got a problem because development has your other changes in it, so you're going to have to branch your template for production, but perhaps also update what's in development with the new change, or rollback your other changes to test.

🤷 Either way it's not ideal.

Adding flags to your IaC

IaC tools have conditional logic of varying degrees of sophistication. This might be Conditions in CloudFormation, Conditional Expressions in Terraform or the full power of a programming language using the AWS Cloud Development Kit (CDK).

Once you've got conditional logic, you can turn parts of your infrastructure on/of or configured on way or another based upon some kind of input. The easiest option here is to have a parameter to your template act as your feature flag.

Your goal with the feature flags is to be able to merge changes and then use the flags to enable the features progressively in environments while still allowing additional changes to be made and tested.

To create or not to create, that is the question

With feature flags in application software, the new code that is not yet enabled is actually deployed to production, it's just not active. With IaC it's a little different, because your new change may be creating additional resources (or deleting old ones).

There are a few factors to consider when you decide whether your flag should control the existence of resources or just whether they're configured to be used:

Are you adding new resources of significant cost that you aren't sure when they're going to be used?
Can the change exist in parallel with existing resources?
Does pre-creating new resources make the eventual switchover faster and less risky?

In other words it's going to depend upon the situation.

If the resources are cheap (perhaps even free if not being used) and can co-exist then you probably don't need to bother with a flag at all and just go ahead and create the new resources.

Testability

You can often test your changes by going ahead and deploying them in a non-production environment. Depending upon the change you may be able to deploy it in parallel to existing resources and not break anything if you get it wrong, but certain changes can have impacts (going back to our network changes), and some changes are only applied in production.

Depending upon the IaC tool you are using there are different levels of testing you can perform without deploying anything:

Guidelines for Testing HashiCorp Terraform
Testing AWS CDK Constructs

With CDK you can write regular unit tests that can easily be automated to ensure you create the correct infrastructure, and you can vary your inputs (your feature flags) to ensure they do the right thing when turned on or off. This can give you the confidence to push your new template to production and know that it won't change anything unexpectedly until you toggle your flag 🚩.

As I'm writing this, I'm doing a good job of convincing myself that I should look more seriously into using CDK for new projects.

Your level of confidence and familiarity with techniques like unit testing are going to depend upon your background, but robust testing for your Infrastructure code is just as important as your Application code.

Cleaning up

There is some conditional logic in your IaC that will always be there to handle differences between environments.

Feature flags are different - they should be transient and cleaned-up once the flag has been turned on in every environment.

Once again unit testing is really useful here as you can ensure that removing the flag won't make unexpected changes.

Summing up

Using feature flags in your Infrastructure as Code can allow you to merge changes and promote them to production without updating your infrastructure until you are ready. This can reduce the number of conflicting changes in your templates and avoid problems with when particular changes go live.

I wrote this article because I've been struggling with exactly the problem I described with changes in development that aren't ready to go to production.

It seems to me now like an obvious solution, but I wasn't using it before, and I couldn't find a lot of articles suggesting this approach.

Let me know in the comments if you're already been doing something like this.

Cloud Migration 101: Developer Edition

Andrew May — Sat, 13 Mar 2021 00:37:48 +0000

So your company is moving to the cloud, that's cool,
but how does it affect YOU?

How things work on-prem

When applications are on-premises, developers typically have little say about the infrastructure their applications run on. The choice of languages and frameworks may be set in stone, and applications are more likely to be monolithic.

Even with the increasing adoption of DevOps practices, there may still be a significant divide between application developers and the infrastructure/operations team - your deployments may be automated, but setting up a new server probably isn't.

How things change in the Cloud

The Performance Pillar of the AWS Well Architected Framework includes these Design Principals:

Democratize advanced technologies: Make advanced technology implementation easier for your team
Use serverless architectures
Experiment more often
Consider mechanical sympathy

This isn't just true of AWS, the ability to quickly spin up new resources, initially for experimentation and then all the way to production, is a feature of all Cloud platforms.

The architecture of applications built to take advantage of running in the cloud can be radically different to what you've been previously building, and while you can deploy an ASP.NET core API to AWS Lambda (for example), how it behaves in that environment is significantly different to running it in IIS.

AWS provides a NuGet package (see Amazon.Lambda.AspNetCoreServer) that makes it easy to do this. It works pretty well, but instead of one copy of the API serving many clients you have many copies of the API each serving one client at a time. There are also limitations in areas like file upload and download (which vary depending upon whether this is being used with API Gateway or an Application Load Balancer).

Become a Cloud Developer

For Leading EDJE I've worked on a number of cloud migration projects, as an architect, developer and cloud engineer. I've seen the difficulties that developers can face getting up to speed with the cloud, and the miscommunication that can occur between application and infrastructure teams.

It's more important than ever for developers to understand how their applications run when deployed and not just on their machine when developing. This requires developers to have an understanding of the Cloud platform that's being used and it's features and quirks. It's also important (this has always been important) for developers to be involved in supporting applications all the way through their lifecycle including production support.

Training and certifications

Hopefully, if your company has decided to move to the cloud they've recognized that there's a skill gap and have a plan for training. If so, make sure you take advantage of it - even if you don't use it immediately, it shows a willingness to learn that looks good at review time. If unfortunately your company hasn't got a plan in place, or they are limiting who has access to the training to a select few (perhaps they think developers don't need to know the details), then it behooves you to still do what you can to learn about the cloud.

This unfortunately may mean that you need to learn about the cloud in your own time, and I recognize that not everyone has free time to spend on this self-study, but if you do there can be many benefits for your career.

If your company doesn't want to make use of your new found cloud skills, there are lots of other companies that are looking for them.

Studying for a cloud certification is a good way of getting an overview of a cloud platform. There's a variety of free and paid training (much of it inexpensive), and you should also go hands-on and sign up for a free trial of the cloud platform your company is using so that you can try some things out.

Mechanical Sympathy

“You don’t have to be an engineer to be be a racing driver, but you do have to have Mechanical Sympathy.” – Jackie Stewart, racing driver

If you don't understand how the cloud platform works, then you can build an application that runs in the cloud, but you can't build a cloud native application that makes efficient usage of cloud resources.

Cloud Native is an overused term that has multiple different definitions.

The Cloud Native Computing Foundation has a heavy focus on Kubernetes and Linux, but when I talk about it I'm thinking more of what's described in this Azure Cloud-Native overview, with a combination of managed services, containers and serverless.

New applications you build in the cloud can take advantage of services provided by the platform, but you can only do this effectively if you understand the strengths and weaknesses of those services. It may make sense to decompose your application into microservices or even smaller units of work and use messaging to decouple processing.

The implication of this is that as a developer you need to be more involved in architectural decisions even when you are working on relatively small applications. You will hopefully have someone in your organization setting guidelines for Cloud architecture, but each application has different needs.

You may have the opportunity to get involved in provisioning the infrastructure using Infrastructure as Code tools like Terraform, CloudFormation (AWS), Azure Resource Manager or Cloud Deployment Manager (GCP). If you can help to build the infrastructure you are far less likely to be held up waiting on another team to do the work or to have difficulties communicating what you need.

Infrastructure as Code may be foreign and new to you, but it's the best way to really understand how the building blocks in a cloud platform fit together.

Organizational resistance

A cloud migration is a journey for a company and it may require changes that not everyone is ready to accept.

As a cloud developer who learns how the cloud platform works, you can be in a position to influence how your company adapts and this can be a great career opportunity. That may put you at odds with those who oppose change, but having a deep understanding of the platform allows you to educate those around you and overcome some of those difficulties.

A change of role?

For much of my career I was a Java developer and then an application architect specializing in Java. I learned about AWS while I worked on my first Cloud project and found myself splitting my time between architecting/developing parts of the application, and building out the infrastructure using CloudFormation.

Personally I enjoy the mix of all these different roles, but you might find out that you have an affinity for building cloud infrastructure. The relatively new field of Site Reliability Engineering can be a good fit for those with a development background.

You owe it to yourself

I firmly believe that Cloud platforms are here to stay and will become the dominant deployment platform for the applications we all develop. Even those organizations who do not want to use a public platform are likely to adopt technologies where infrastructure can be provisioned on demand.

So, if you consider yourself "just a developer" you might want to rethink the boundaries of what that means.

The relevance of the AWS Certified DevOps Engineer - Professional certification

Andrew May — Sun, 21 Feb 2021 20:57:56 +0000

I recently passed the AWS Certified DevOps Engineer - Professional certification, and honestly I did it more to get a sense of completion (and to finally add the sticker I picked up at re:Invent 2018) than an expectation that I would learn a lot, but I was surprised how many gaps in my knowledge it filled in and how relevant they would end up being.

Certification topics

The certification focuses on a number of services that I haven't used a great deal, and aren't necessarily the ones I would use for green-fields development, but have an important role when it comes to cloud migrations of existing applications. In particular you can expect to see a lot of questions about Elastic Beanstalk and CodeDeploy, and learning their different deployment options is a large part of preparing for this certification.

You also need to learn about CodeCommit, CodeBuild and CodePipeline, but while these services have their uses (and generally fairly attractive pay-as-you-go pricing), their functionality and ease of use is unfortunately far behind other CI/CD solutions (e.g. Azure DevOps, GitHub Actions, GitLab).

As part of my certification prep, I used CodeBuild to deploy the container image for stackmanager to the new Amazon ECR public gallery. Having to configure some elements within AWS (using CloudFormation) and others within the stackmanager repository (in GitHub) made configuring the build much more complicated and error prone than solutions where everything can be configured in one place.

A large part of the certification focuses on deployments to EC2 instances within Auto-Scaling Groups, or best practices around migrating on-premises applications to EC2. Understanding some of the details of ASGs, in particular lifecycle hooks is important to be able to pass the exam.

Honestly, I got tired of the repetition within the practice tests and the actual exam and the questions started to blur together. I had to take a brief break during the exam to take my hands away from the keyboard and close my eyes for a few minutes to re-focus - it would have been nice to get up and stretch, but that was not possible in the proctored exam.

One other service that I really needed to focus on was EventBridge (aka CloudWatch events). I've used CloudWatch events for triggering other services (mostly Lambda functions and Step Functions) based upon a schedule or something occurring within the AWS account, but it wasn't until studying for this certification that I really appreciated the difference between services that natively create events, and those where events are available via CloudTrails. It's also interesting that there are a small number of API calls you can make directly from an event (e.g. Create an EC2 snapshot) that probably predate Lambda functions. There are also a lot more targets for events than I was aware of.

There is a focus on monitoring and governance, with AWS Config and Trusted advisor being used to trigger remediations for different types of issues.

One of the hardest parts of the certification is remembering exactly what mechanisms the different services have for triggering other activities - whether it's AWS Config directly triggering SSM Automation, or the types of notifications CodePipeline supports.

It's also important to be familiar with Lambda, ECS, CloudFormation and a number of other services, but these are services I use on a regular basis and were not a large part of the exam.

Relevance

OK, so that's a lot about what the certification covers, but by itself that doesn't make a convincing case for relevance.

Leading EDJE is an AWS Select Consulting Partner, and as I become more involved in our partnership and work on projects at our clients, it has become clear how early many companies are in the overall cloud adoption process.

The migration path can vary a lot, but it is common for enterprises to have applications that are moved to the cloud but are not immediately rearchitected to take full advantage of cloud functionality. These may be applications that are due to be retired or replaced but are still business critical and may still be updated on a regular basis.

These applications will run on EC2 instances, either managed directly or using Elastic Beanstalk. Understanding the different deployment options as covered by this certification is essential to knowing how best to configure and manage applications that may have restrictions due to their architecture or licensing.

Recently I've been able to offer advice on where CodeDeploy might be an appropriate tool for deployments, and some of the limitations it has (for example it is not recommended to have multiple deployment groups targeting a single ASG).

A convincing case?

I learned quite a lot while studying for this certification, and there was less overlap with the other certifications than I expected. I was familiar with the services covered, but had never gone into this much depth with most of them before.

Most of my preparation was a combination of practice tests and going through the AWS documentation for the services included in the tests and making notes, plus my experimentation with CodeBuild mentioned above.

If you are coming from a System Administrator background and are learning about AWS, there is a lot of important information here and it is the obvious choice after the AWS Certified SysOps Administrator - Associate certification.

Let me know in the comments if you've taken this certification and felt it was worthwhile (or not).

stackmanager: Yet another tool for CloudFormation

Andrew May — Thu, 29 Oct 2020 00:55:29 +0000

I recently started building stackmanager, an Open Source project for managing CloudFormation stacks.

There are already a number of other tools for doing this, and I've used or tried several of them, but none quite fit how I wanted them to work.

Functional Requirements

Managing all the configuration used to create/update a CloudFormation stack in a single file

The CloudFormation create-stack command has lots of separate values you need supply, and with the CLI it's a pain to supply Stack Name, Template File, Parameters, Targets, Capabilities, etc.

Support for deploying the same stack to different environments (e.g. dev, prod, or different regions) while reducing duplication of configuration

Almost all the CloudFormation I write is used in multiple different environments (often different accounts), and sometimes is deployed to multiple regions for failover.

Uses ChangeSets, with control over whether to immediately apply or apply later

ChangeSets allow you to preview (to a limited degree) the changes you're making, and especially if you're using the AWS Serverless Application Model transformation, or a CloudFormation macro, you don't necessarily know exactly what resources are going to be created until the CloudFormation service process the template.

In some CI/CD workflows, there may be an approval requirement before changes go into production, and the ChangeSet allows you to preview changes.

Log progress

Print a summary of the ChangeSet and show the CloudFormation events whether the stack update succeeds or fails. I really like how SAM shows this information.

Support Lambda Functions

This is somewhat outside the core functionality, but as I started writing this while working on a project that deployed Lambda functions using CloudFormation (where the S3 Key is changed when there is new code, triggering an update), I wanted to be able to use a single tool to build, upload and deploy Lambda functions, similar to the SAM CLI, but better suited to more general CloudFormation management and CI/CD.

Non-functional requirements

This is not the first utility of this sort I've written, but previous versions have belonged to the client who I wrote them for. I wanted to create an Open Source utility that I could use at multiple clients and for Leading EDJE infrastructure.

Writing stackmanager from scratch allowed me to improve on what I'd written previously, and include things I'd previously neglected like unit tests.

Even if I end up the only one contributing to the code base I also wanted to run it like a regular open source project with a list of issues, pull requests and releases when I reached a suitable milestone.

Getting to 1.0

A lot of Open Source projects never reach a 1.0 release, but I just wanted to release something relatively complete and well tested and not necessarily include everything that stackmanager could possibly do. The hardest thing to finish in the list of issues I'd included in the 1.0 milestone was to get acceptable code coverage, as my experience writing Python unit tests is somewhat limited, especially when it comes to mocking.

Using stackmanager

I don't really want to rewrite the README here, but stackmanager allows you to do things like:

$ stackmanager --profile dev --region us-east-1 \
build-lambda --source-dir integration/functions/python/hello_world/ --output-dir /c/dev/temp/ --runtime python3.7 \
upload --bucket stackmanager-lambda-files-us-east-1 --key python/hello_world.zip \
deploy --environment dev --config-file integration/functions/python/config.yaml --auto-apply

Building python3.7 Lambda function from integration/functions/python/hello_world/

Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource

Built Lambda Archive C:\dev\temp\hello_world.zip
2020-10-28 20:33:21 Found credentials in shared credentials file: ~/.aws/credentials

Uploaded C:\dev\temp\hello_world.zip to s3://stackmanager-lambda-files-us-east-1/python/hello_world.zip

Stack: d-StackManager-PythonFunction, Status: does not exist

Creating ChangeSet c8a1e3704d0da48f69903bafcaf239c86

Action    LogicalResourceId    ResourceType           Replacement
--------  -------------------  ---------------------  -------------
Add       FunctionRole         AWS::IAM::Role         -
Add       Function             AWS::Lambda::Function  -

Executing ChangeSet c8a1e3704d0da48f69903bafcaf239c86 for d-StackManager-PythonFunction

ChangeSet c8a1e3704d0da48f69903bafcaf239c86 for d-StackManager-PythonFunction successfully completed:

Timestamp                  LogicalResourceId              ResourceType                ResourceStatus      Reason
-------------------------  -----------------------------  --------------------------  ------------------  ---------------------------
2020-10-28 20:33:24-04:00  d-StackManager-PythonFunction  AWS::CloudFormation::Stack  REVIEW_IN_PROGRESS  User Initiated
2020-10-28 20:33:35-04:00  d-StackManager-PythonFunction  AWS::CloudFormation::Stack  CREATE_IN_PROGRESS  User Initiated
2020-10-28 20:33:39-04:00  FunctionRole                   AWS::IAM::Role              CREATE_IN_PROGRESS  -
2020-10-28 20:33:39-04:00  FunctionRole                   AWS::IAM::Role              CREATE_IN_PROGRESS  Resource creation Initiated
2020-10-28 20:33:53-04:00  FunctionRole                   AWS::IAM::Role              CREATE_COMPLETE     -
2020-10-28 20:33:56-04:00  Function                       AWS::Lambda::Function       CREATE_IN_PROGRESS  -
2020-10-28 20:33:57-04:00  Function                       AWS::Lambda::Function       CREATE_IN_PROGRESS  Resource creation Initiated
2020-10-28 20:33:57-04:00  Function                       AWS::Lambda::Function       CREATE_COMPLETE     -
2020-10-28 20:33:59-04:00  d-StackManager-PythonFunction  AWS::CloudFormation::Stack  CREATE_COMPLETE     -

This built a Python Lambda function, uploaded the zip file to S3 and then deployed a CloudFormation stack that used the file in S3, previewing the change-set and then running it.

In this case the functionality is similar to the SAM CLI for Lambda functions, but the focus of stackmanager is really the deploy command that is for any CloudFormation stack and not just Lambda functions.

At the heart of stackmanager is the configuration file that can encapsulate the configuration for multiple environments. For the example I ran above this is the configuration file:

---
Environment: all
StackName: "{{ EnvironmentCode }}-StackManager-PythonFunction"
Parameters:
  Environment: "{{ Environment }}"
Tags:
  Application: StackManager
  Environment: "{{ Environment }}"
Template: integration/functions/python/template.yaml
Capabilities:
  - CAPABILITY_IAM
---
Environment: dev
Region: us-east-1
Variables:
  EnvironmentCode: d

In this simple example the inheritance between the dev environment and the all environment isn't particular useful, but once I have multiple environments and a mix of common and unique parameters this really cuts down on the total amount of configuration (and the number of separate files to edit).

Is stackmanager right for you?

If you're already using CloudFormation, but you have a mess of parameter files and other arguments you're using to run your CloudFormation then I'd like to think this will help you clean up the mess.

Moving forward there's a possibility that CDK will obsolete writing CloudFormation directly, but there's a lot of CloudFormation already out there so it's going to be around for a while.

PyPI release using GitHub Actions

Andrew May — Sun, 16 Aug 2020 18:53:18 +0000

My Workflow

I'm making use of the release event in a GitHub action to release my open source stackmanager project to PyPI. This allows me to use the GitHub tag name for the PyPI release and just have one place to manage versions.

I don't always want to release a new version after a PR is merged to master, so this gives me manual control over releases.

This has been my first time using GitHub Actions and I've been generally pleased with how it worked but found some of the documentation to be a bit cryptic.

Submission Category:

Maintainer Must-Haves

Yaml File or Link to Code

LeadingEDJE / stackmanager

Utility for managing Cloudformation stacks

stackmanager

Utility to manage CloudFormation stacks based upon a Template (either local or in S3) and a YAML configuration file.

Uses ChangeSets to create or update CloudFormation stacks, allowing the ChangeSets to either be automatically applied or applied later (e.g. during a later phase of a build pipeline after review of the ChangeSet).

There are also some utility methods for building a lambda file zip and uploading files to S3 These are to provide some of the AWS SAM CLI functionality while fitting into the workflow and configuration style of stackmanager.

Configuration

The configuration file can either be a single YAML document containing the configuration for a stack for a specific environment and region, or can contain multiple documents for different deployments of that stack to different environments and regions.

Single Environment

The configuration combines together the different values that are typically passed to the CloudFormation command line when creating…

View on GitHub

Specifically, the release.yml workflow is triggered on release.

Additional Info

The version of the release is taken from the github.event.release.tag_name property of the release event and passed to setup.py as the STACKMANAGER_VERSION environment variable.

I'm using a PyPI API token restricted to this project so that I don't have to store a username and password in GitHub secrets (this also allows me to turn on 2FA for my PyPI account).

    - name: Package and Upload
      env:
        STACKMANAGER_VERSION: ${{ github.event.release.tag_name }}
        TWINE_USERNAME: __token__
        TWINE_PASSWORD: ${{ secrets.PYPI_APIKEY }}
      run: |
        python setup.py sdist bdist_wheel
        twine upload dist/*

AWS Lambda Layer for Private Certificates

Andrew May — Mon, 10 Aug 2020 20:02:55 +0000

The problem

How can code running in the managed AWS Lambda environment call services that use private certificates for HTTPS?

The majority of enterprises moving to AWS or other cloud platforms have existing on-premises applications, and there is often a need for the new cloud based applications to talk back to services on-prem. Typically this done with a hybrid network where the corporate network and AWS VPC(s) are connected using a VPN or Direct Connect.

Let's assume that the network has been set-up and the on-prem service is either using public DNS or a solution like Route 53 resolver has been configured and it's possible to establish a connection to the on-premises service.

If the service is using a public certificate (issued by a public certificate authority that the Lambda environment is aware of and has a copy of the root certificate public key) then there will be no problems, but if the enterprise is using their own private Certificate Authority then there will be an error making HTTPS calls.

Enterprises may have their own CA for services that were only ever expected to be used internally. It gives greater control over issuing and revoking certificates. Typically the CA certificate will be installed onto desktops and services at the enterprise to ensure that connections to these services are trusted in browsers and between services.

If this were an EC2 instance calling back to on-prem then you would probably install the certificate as part of the instance set-up, but we have less control in the transient Lambda environment.

It's possible to write code to handle the TLS failure, or even to disable certificate validation entirely (please don't do this), but what if there was a better way to have the lambda function transparently trust the private certificate?

The solution

Lambda Layers to the rescue!

Many TLS/SSL libraries or application frameworks have mechanisms to add additional root certificates to the "trust store". Once this is configured, any connections using certificates that were signed by the private root certificate will be automatically trusted.

On Linux (and macOS) .NET Core uses OpenSSL for cryptography and OpenSSL allows you to add additional root certificates from a file (in PEM format) using the SSL_CERT_FILE environment variable. The root certificate doesn't need to be "installed" into the environment.

Setting environment variables is easy for Lambda functions, but where should this certificate file be, and how do we get it into the Lambda environment?

The contents of the Lambda zip file are extracted to /var/task, and it's possible to include the certificate file here and point SSL_CERT_FILE to a location in this directory, however this has the drawback that every lambda zip file needs to contain the certificate file and you either need to include it into every repository, or include it as part of your CI/CD process.

Lambda Layers have two main use-cases: sharing dependencies (typically code or libraries, but can be configuration like this) or creating a custom runtime. The contents of the layer are extracted to /opt.

If we build a layer containing our certificate file it can be re-used across as many lambda functions as we like.

Building and sharing the layer

This SAM template will build a Certificate Lambda Layer including files in the certs sub-directory in the layer zip file.

The ARN for the layer is placed in a Parameter Store value that can be referenced by the templates for Lambda functions.



AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: SAM Template for certificate layer

Resources:
  CertificateLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      CompatibleRuntimes:
        - dotnetcore2.1
        - dotnetcore3.1
      ContentUri: ./certs/
      Description: Layer containing additional certificates
      RetentionPolicy: Retain

  LayerParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Description: ARN for latest Certificate Layer
      Name: /Lambda/Layers/Certificate
      Type: String
      Value: !Ref CertificateLayer

Within the certs sub-directory there should be a file containing one or more root certificates in PEM format. Typically this file will have a .crt extension, so let's call it additional-certificates.crt.

If we need to add additional root certificates (or remove expired ones), the layer can be updated, which will create a new version. Existing functions will continue to use the old version, but the Parameter Store key will now point to the ARN of the new version so any new deployments will pick up the change.

Why not a CloudFormation export? CloudFormation exports are good for things that never change, but you can't update the value of an export that's in use by another stack.

In the parameters of your CloudFormation stacks containing your Lambda functions, include a parameter to pull in the value from Parameter store and then use it with the function, also setting the SSL_CERT_FILE environment variable:



AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Lambda Function

Parameters:
  CertificateLayer:
    Default: /Lambda/Layers/Certificate
    Description: Certificate Layer ARN
    Type: AWS::SSM::Parameter::Value<String>

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Environment:
        Variables:
          SSL_CERT_FILE: /opt/additional-certificates.crt
      Layers:
        - !Ref CertificateLayer
      # Plus all the other properties

Once again we're using SAM here, although there is unfortunately an issue with the SAM CLI build command where it does not like the use of parameter store values for layers. There are some work arounds described in the GitHub issue.

Other runtimes

I've used this approach with .NET Core lambda functions, but this layer may be useful for other lambda runtimes.

For example, the popular Python requests library also allows you to configure certificates using the REQUESTS_CA_BUNDLE environment variable.

Using the AWS Serverless Application Model (SAM)

Andrew May — Sat, 18 Jul 2020 21:38:27 +0000

I've recently done some work to compare the AWS Serverless Application Model to the Serverless Framework for building and deploying .NET Core based Lambda functions to AWS, and have started to deploy a number of Lambda functions that were built using the Serverless Application Model (SAM).

There are a number of existing articles out there that compare the two frameworks, so rather than write another one I'll cut to the chase and say that I prefer SAM over the Serverless Framework for a number of reasons. I would suggest you evaluate both frameworks for yourself because you may have different criteria for evaluation. This article is going to give an overview of SAM and at the end I'll offer a brief comparison with the Serverless Framework. Other articles in this series will talk more about deploying services built with SAM, and other things that we've encountered while building these applications.

AWS Serverless Application Model

There are two main parts to SAM - the Template Specification and the Command Line interface.

SAM Templates

If you're at all familiar with CloudFormation then you will find writing SAM templates very familiar, because they are just CloudFormation templates with some custom resources and a Transform:

AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: A simple SAM Template

Globals:
  Timeout: 60
  MemorySize: 512

Resources:
  ListFunction:                                                                  
    Type: AWS::Serverless::Function                                                       
    Properties:                                                                           
      CodeUri: ./src/ListFunctions/                                                   
      Handler: ListFunctions::ListFunctions.LambdaEntryPoint::FunctionHandlerAsync
      Runtime: dotnetcore3.1

Although the AWS::Serverless:: resources may look like custom Resource Providers, they actually have far more in common with Macros that can manipulate a template using a transform built using a Lambda function. The SAM Translator is the code that is responsible for converting SAM templates into CloudFormation templates.

Because this is a CloudFormation template with some additions, you can add standard CloudFormation resources intermingled with the SAM resources. Developer tools that work with CloudFormation templates for editing and linting (cfn-lint) also support SAM templates.

In fact you can combine your own macros with the SAM Serverless transform to manipulate regular CloudFormation resources or even SAM resources, which can be useful because there are gaps in what the SAM resources support.

Something that's important to note here is that you submit the template containing the AWS::Serverless:: resources to the CloudFormation service and it handles the transformation server-side. There are a few modifications that the SAM CLI may make to the template client-side to replace local references to code with an S3 Bucket and Key, but these are similar to the existing functionality that's part of the aws cloudformation package command.

Differences between `AWS::Serverless::Function` and `AWS::Lambda::Function`

Many properties of AWS::Serverless::Function match the properties of AWS::Lambda::Function, which is unsurprising because after the transform is applied the one is replaced with the other plus additional resources.

Some of the biggest benefits to using the AWS::Serverless:Function are these properties:

CodeUri

The CodeUri property can be a link to a relative directory, and when used with the sam build command it will perform the appropriate build for the specified Runtime and then package the Lambda code.

Events

The Events allows you to specify the triggers for a Lambda function that you would normally need to specify in separate CloudFormation resources. There are current 14 different Event Sources including API Gateway (both the REST and HTTP versions), S3 and SQS.

One big limitation of the S3 event is that it only works if the S3 bucket is declared in the same SAM template. This is because CloudFormation only allows you to specify S3 events directly on the S3 bucket resource. In a lot of cases the S3 bucket will have a different lifecyle than the lambda function and you would not want CloudFormation to try and delete it (which will fail if it contains objects) if you delete the Lambda function.

CloudFormation Custom Resources can be used to add S3 lambda triggers separately from the bucket creation.

With the Api and HttpApi events there's some extra magic going on - if you don't specify the RestApiId (for the Api event) or ApiId (for the HttpApi event) it will automatically create an API Gateway using the mappings in your events.

Events:
  HttpApiEvent:
    Type: HttpApi
    Properties:
      Path: /
      Method: GET

Example HttpApi event

There is a lot more you can do with API Gateways using the AWS::Serverless::Api and AWS::Serverless::HttpApi resources, but that's a whole separate topic.

Unfortunately the Events section lacks supports for Application Load Balancers, which are becoming a popular alternative to API Gateway for Lambda functions because they are significantly cheaper than REST API Gateways and can be used to easily build internal APIs (something that is possible but more difficult with REST API Gateways and not currently possible with HTTP API Gateways). That doesn't mean you can't define a Lambda function using AWS::Serverless:Function and target it with an ALB, but you will need to create the AWS::Lambda::Permission, AWS::ElasticLoadBalancerV2:TargetGroup and AWS::ElasticLoadBalancerV2::ListenerRule resources yourself.

This is one of those gaps that can be plugged with CloudFormation macros that operate on the SAM resources, like this example: https://github.com/glassechidna/sam-alb

Policies

Doesn't everyone love writing IAM policies? The Policies property can be used in a number of ways, but one really useful feature is being able to use Policy Templates that provide well designed sets of permissions for certain resources.

Policies:
  - DynamoDBReadPolicy:
      TableName: !Ref DyanamoDBTable
  - KMSDecryptPolicy:
      KeyId: !Ref DynamoDBKmsKey

Granting a Lambda function to read from a specific table and decrypt data using the related KMS key

It will also add the necesary permissions for CloudWatch logs, and other resources declared on the AWS::Serverless:Function resource - i.e. if you specify VPC subnets to run the function within your VPC then it will add the necessary permissions automatically.

SAM CLI

The standard workflow for SAM applications is to run sam build to build and package any code referenced in AWS::Serverless::Function resources, then sam deploy to upload the lambda zip files to S3 and then create a CloudFormation stack (by default using change sets) to deploy everything.

`sam init`

Create a new SAM project. There are quick start examples for most if not all of the supported runtimes:

$ sam init
Which template source would you like to use?
        1 - AWS Quick Start Templates
        2 - Custom Template Location
Choice: 1

Which runtime would you like to use?
        1 - nodejs12.x
        2 - python3.8
        3 - ruby2.7
        4 - go1.x
        5 - java11
        6 - dotnetcore3.1
        7 - nodejs10.x
        8 - python3.7
        9 - python3.6
        10 - python2.7
        11 - ruby2.5
        12 - java8
        13 - dotnetcore2.1

While these aren't the most complicated examples, they set up the directory structure in a way that works well with SAM and include testing set-up. You can also use custom templates if you have your own way of organizing your projects.

`sam build`

The build command looks for template.yaml or template.yml in the current directory by default, and figures out what to build based upon the contents of the template. The template could potentially contain several lambda functions using different runtimes, and it would package the code pointed to be the CodeUri property using an appropriate builder for the runtime. All of these files end up in an .aws-sam/build directory.

There are few arguments you might need to specify in certain cases:

Set --use-container for runtimes that have native dependencies. It will pull down a docker image based upon the Lambda environment for the function runtime and build within that. This can be important for Python where some dependencies may need to be compiled and if built on another platform (e.g. Windows) will not work in the Lambda runtime which is based upon Amazon Linux (1 or 2 depending upon the runtime).
Set --profile/--region if using Lambda layers - it will retrieve the layer from your AWS account. If the Layer ARN is provided via a Parameter you will also need to provide the value using --parameter-overrides.

Other parameters can override the build directory and the manifest file (e.g. requirements.txt for Python, a .csproj file for C#, pom.xml or build.gradle for Java) location or the template name if you're not using template.[yaml|yml].

`sam package`

The package command is less commonly used now because it's functionality has been rolled into the deploy command. This command packages the results of the build and uploads to s3 and then replaces the local references in the template with references to the objects uploaded to S3 and returns the new template (either to standard out or a file). The command supports these resource types and attributes:

  Resource : AWS::ServerlessRepo::Application | Location : LicenseUrl
  Resource : AWS::ServerlessRepo::Application | Location : ReadmeUrl
  Resource : AWS::Serverless::Function | Location : CodeUri
  Resource : AWS::Serverless::Api | Location : DefinitionUri
  Resource : AWS::Serverless::StateMachine | Location : DefinitionUri
  Resource : AWS::AppSync::GraphQLSchema | Location : DefinitionS3Location
  Resource : AWS::AppSync::Resolver | Location : RequestMappingTemplateS3Location
  Resource : AWS::AppSync::Resolver | Location : ResponseMappingTemplateS3Location
  Resource : AWS::AppSync::FunctionConfiguration | Location : RequestMappingTemplateS3Location
  Resource : AWS::AppSync::FunctionConfiguration | Location : ResponseMappingTemplateS3Location
  Resource : AWS::Lambda::Function | Location : Code
  Resource : AWS::ApiGateway::RestApi | Location : BodyS3Location
  Resource : AWS::ElasticBeanstalk::ApplicationVersion | Location : SourceBundle
  Resource : AWS::CloudFormation::Stack | Location : TemplateURL
  Resource : AWS::Serverless::Application | Location : Location
  Resource : AWS::Lambda::LayerVersion | Location : Content
  Resource : AWS::Serverless::LayerVersion | Location : ContentUri
  Resource : AWS::Glue::Job | Location : Command.ScriptLocation
  Resource : AWS::StepFunctions::StateMachine | Location : DefinitionS3Location

You are required to specify the --s3-bucket parameter and you can supply and optional --s3-prefix (useful if you want to share a bucket between multiple serverless applications but want to group related artifacts). The S3 key will be automatically generated using an md5 of the uploaded resource, and the command will by default skip uploading if the md5 of the new resource matches an existing S3 key.

One thing to note about this approach is that it works much better running locally than from a CI/CD pipeline. Locally the build may skip recompilation if nothing has changed or copy files and retaining their timestamps; but when using a fresh checkout in a pipeline you are typically performing a fresh build, and even if all the files in the resulting artifact are identical, if they're zipped up for a Lambda function the different timestamps will cause the md5 to be different than any previous build.

`sam deploy`

The deploy command includes the functionality in package in newer versions of the SAM CLI, and also deploys the application using CloudFormation. It makes use of ChangeSets and will display the pending changes before executing them and then displays the CloudFormation events as they occur. The command has quite a lot of options (many of them are familiar from the related CloudFormation commands), so when using SAM locally you can specify the --guided option and it will prompt you to make certain decisions and ask for values for all the parameters in the CloudFormation template. These values are then stored in a toml configuration file which will be read the next time you run sam deploy.

Unlike the package command, you are not required to specify the S3 bucket to upload the files to (although you can), and by default it will create a new bucket in your account for SAM deployments if one does not already exist.

Local Testing

In addition to packaging and deploying to AWS, the sam local commands allow you to run Lambda functions locally within a Docker container that matches the AWS runtime. It also has some basic support for REST API Gateway applications.

sam local generate-event can generate sample events from 22 different services. This can save you from scouring the documentation to find examples, but typically these events are just one of the possible events from the service and you will probably need to tweak it for use in testing your lambda functions.
sam local invoke will run your Lambda function in a docker container. You supply an event from either generate-event or a local file. If you have a single function in your template it will automatically call that, but if there are multiple functions you will need to supply it's LogicalId from the template. The output from the function is almost identical to what you would see in CloudWatch logs, including the timing and utilized memory information. Note that because it starts the container each time you're incurring the Lambda cold start penalty every time.
sam local start-api can be used for templates containing a REST API Gateway either directly defined or where the Lambda functions use the Api event type. It does not support HTTP API Gateways currently. It starts a server in Docker and listens on localhost:3000 by default. This way you can make HTTP calls rather than using the API Gateway events that you need to supply to sam local invoke.
sam local start-lambda also starts a HTTP server, but this one allows you to invoke the lambda functions using the CLI or SDKs as if they were running in AWS. This may be useful for running a series of tests against a lambda without running sam local invoke each time.

Of course there are limitations to what this supports. If your lambda function calls another AWS service (as many do), then it will call out to AWS and you will need to have valid credentials (and those resources will need to have been deployed). It is possible with some work to invoke certain local versions of AWS services (e.g. those from localstack), and this may be useful for automated testing, but nothing will be a perfect emulation of the AWS environment.

This is particularly true when you need to verify that your IAM permissions for calling other services are correct, as when running a function locally and invoking services in AWS it will be using the AWS credentials you supply it with rather than the role that has been defined for the function.

There are some integrations with a number of IDEs (e.g. Visual Studio Code) that allow you to debug your Lambda function when it's run locally like this. I've had mixed luck getting this to work consistently with .NET Core, but it can be useful.

Together or Apart

Because the Serverless Transform used in SAM templates is essentially a pre-defined macro that is implemented as part of the CloudFormation service, there is no requirement that you use the SAM CLI with your SAM templates.

For example, you might be building the Lambda Zip files as part of your CI/CD pipeline and then use the same Zip file in multiple accounts (e.g. dev/test/prod) without rebuilding it. The SAM CLI doesn't really support this workflow, but you can still use the AWS::Serverless:: resources in your template.

You will not be able to use the substitutions for local resources that the SAM CLI supports if you are using the CloudFormation commands instead, although aws cloudformation package does support uploading local Zip files to S3 and returning a transformed template. If you upload to S3 as part of your CI/CD pipeline you can then make the Bucket name and Key CloudFormation parameters that the servless resources reference.

Comparisons to the Serverless Framework

Unlike SAM, the Serverless Framework supports multiple Cloud platforms. If you have a multi-cloud strategy this can be useful.
Serverless supports a plugin model that extends what the platform can do - with SAM you are somewhat constrained by what has been implemented by the platform and it can sometimes be frustrating waiting for new functionality to be supported.
There is a Commercial (Pro) version that offers a monitoring platform, CI/CD support and other administration features. If your Cloud applications are largely built up of functions this might be a useful option instead of using Cloud specific or third party tools to monitor your applications.
There are a lot of examples for using the Serverless Framework on their website, but they are dominated by nodeJS (that I believe was the only language supported when it first launched) - if you're looking for C# examples you won't find anything more than the most basic Hello World.
The CLI is responsible for interacting with the Cloud platform. In the case of AWS it still uses CloudFormation, but the templates are generated by the CLI from the serverless.yml template.

For me this last point is the most significant. The full serverless.yml is pretty comprehensive and provides support for a lot of things that SAM does not, and you can even include raw CloudFormation resources in the resources section of the template, but you are still reliant upon the serverless CLI to generate the CloudFormation template. This means you are required to use the CLI as part of your CI/CD pipelines, and if there are problems with the CloudFormation stack you are dealing with an auto-generated template and trying to understand how that related back to your serverless template.

Because I am very familiar with CloudFormation my bias is towards SAM which allows me to write better CloudFormation rather than Serverless that just makes use of it behind the scenes. Like I said at the start of this article, that's my personal bias and if you don't have the same background in CloudFormation you may find that Serverless works better for you.

In future articles in this series I'll be writing about some different approaches to using .NET Core to build Lambda functions, and some of the challenges of deploying and using these functions in AWS.

Success! Passing the AWS Solutions Architect Professional Certification

Andrew May — Wed, 01 Jul 2020 21:37:30 +0000

After much procrastination, I finally took and passed the AWS Solutions Architect Professional certification, using the "virtual" testing option from Pearson Vue.

I'd been reluctant to take the test virtually after reading a few horror stories about long waits for the proctor to be available. I also wasn't sure that I could get 3 hours undisturbed at home. However, I was able to take advantage of some time when our office was empty and the virtual experience went fairly smoothly.

Additional Preparation

Having taken the Advanced Architecting on AWS course I felt somewhat prepared but needed a way to find the gaps in my knowledge. I'd seen a recommendation for the Tutorials DOJO Practice Tests, and because they were inexpensive I decided to give them a go.

I took 3 of the timed tests, and they helped me build up a list of topics where I wasn't confident in my knowledge:

I then went through the AWS documentation and FAQs for these topics, and in a few cases tried a few things out (e.g. Patch baselines in System Manager). This was a lot to go through, but I used the questions from the test as a guide to which details were important to remember.

Testing Virtually

Scheduling

AWS currently has 2 testing providers, PSI and Pearson VUE. Both are in the process of re-opening their testing centers in Ohio, but with limited capacity and a backlog of people wanting to complete tests, there was a wait of a couple of months for availability at the local testing centers. So I opted to use the virtual testing option from Pearson VUE that has availability 6 days a week.

I picked a time when our office was going to be empty and booked the test for a couple of weeks later. I knew that a conference room in our office was going to be clear of the forbidden clutter that fills my home office.

You will receive an email after scheduling the test recommending that you do a system test on the hardware and in the location where you will be taking the test. This checks internet speed and access to your camera and microphone, but also takes you through the steps of taking photos with your phone of yourself, your id and test location. I did the hardware test at home, but skipped the rest knowing that I was going to be in a different location.

You will need a webcam!

Starting the test

You can start the test up to half an hour before your allotted time, but being late can cause you to fail the test. I turned up at the office in good time and got myself set-up.

I made the mistake of taking the entire system test before the test. This did the hardware check and then walked me through taking and uploading the photos I needed, but didn't save any of that information.

Once I'd repeated this process the second time to take the test for real, the kiosk browser opened for the test and I was instructed to wait for a proctor to be available. Fortunately it was just a couple of minutes before they appeared on the chat and asked if they could start a call with me to prepare for the test.

I can see why a delay at this point would be annoying - I'd already stowed my phone and everything else away, so I was just siting there staring at the screen.

I was asked a few questions about the location I was taking test test in, what I had plugged into USB (a mouse), and had to scan the room by turning my laptop around. They did ask if I could cover the windows of the conference room and unplug the TV on the wall, but neither of those were possible and they seemed OK with that.

The test itself

When you're running the test there's a bar at the top showing your camera image and with a chat button to talk to the proctor. The format of the tests is a little different to the software used for the office practice tests and at PSI testing locations (the button to submit an answer is annoyingly all the way down in the bottom right), but it basically works the same way with a question counter and time countdown at the top right and an option to mark questions for review.

At one point near the end of the 3 hours when I was reviewing one of the questions I'd marked, I had covered my mouth with my hand and the chat window popped up and I was told not to do that, but otherwise I had no further interactions with the proctor.

This was a difficult test, and there were only a small number of questions where I thought the answer was immediately obvious. For most of the questions it was a process of elimination with many answers being several sentences that are identical apart from a few words. Some of the answers are wrong because they suggest doing something that is not possible, but others are wrong because they don't meet the criteria in the question (e.g. most cost effective).

I made sure to answer every question as I went, but marked some of them for review later in case I had extra time. I saw that I was doing OK for time, so I reviewed as I went, and after picking an answer I'd double check that it matched the criteria in the question, and in a few cases that caused me to change my answer because I wasn't really answering the question that had been asked.

I had about 30 minutes left after answering the 75 questions, and wasn't sure whether I was going to pass or fail. I quickly reviewed the questions I'd marked (perhaps 10-15?), but only changed one answer.

Then after confirming that I was done reviewing my answers I got the normal questionnaire about my testing experience and how long I've used AWS.

Finally it told me I'd passed (yay!) and to click the button at the bottom left to end the test.

I wish I'd been able to take a screenshot of the message saying that I'd passed, because honestly I started to doubt it as soon as it was no longer on the screen.

I received the email confirmation 2 days after the test.

Question Topics

Some of the topics that came up in my exam were:

Networking, in particular connectivity between enterprise networks and AWS using Direct Connect and VPNs
Accessing S3 using VPC endpoints
Controlling access across multiple accounts using Organizations and Service Control Policies
Using AWS Config to track resources and verify compliance
The pattern of using SQS and Lambda consumers to decouple applications and handle spikes in traffic
Debugging network connectivity issues
Restricting deployments to approved applications using Service Catalog
Improving performance with various kinds of caching: CloudFront, API Gateway, Elasticache, DAX
Reducing costs with reserved and/or spot instances
Migrating databases to AWS

End of the journey?

I finally managed to add the Solutions Architect Professional sticker to my laptop that I picked up at re:Invent 2017, but I've still got a sticker for the DevOps Professional certification, so perhaps that one is next?

If you read this because you're getting ready to take the certification exam, I wish you the best of luck. It's difficult, but achievable.

The practice test for the AWS Certified Solutions Architect Professional

Andrew May — Sun, 10 May 2020 18:15:21 +0000

Last weekend I was working on preparing a talk about AWS certifications (it looks like I will be presenting it at the upcoming AWS Community Day Midwest), and as part of that I was including sample questions for the different certifications.

AWS provides 10 sample questions for each certification, and the ones for the SA Professional certification can be found on the certification page alongside the exam guide.

As I hadn't actually gone through this version of the sample questions before, I decided to treat them like a mini test, and was fairly pleased to get 8 out of 10 right.

In the past AWS did not provide the answers for these sample questions, so you could never be quite sure whether you had them right, but now all of them appear to have the answers and a short explanation (that's the really useful bit for questions you get wrong).

After that positive result, I decided to dive in and take the official practice test that you can register for at http://aws.training and going into the certification sub-portal. The sample tests contain 20 questions and normally cost $20, but I had a coupon for a free practice test from a previous certification I'd taken.

The practice test gives you an hour to answer 20 questions, so 3 minutes for each. The full test lasts for 3 hours, and apparently contains 75-80 questions, so you have less time per question. I've heard that a lot of people struggle with time on the test, so I tried to go through the questions quickly only pausing when I was unsure.

In the end I finished the 20 questions in about 35 minutes, and I felt pretty good about most of my answers and was surprised that some of the questions were relatively straightforward (of course what seems straightforward mostly had to do with which services I've been using for the last 5 years).

Unfortunately my confidence was somewhat misplaced as when I got the results I had only 65% correct (13/20). I had some doubts about a few questions (and I looked up a few things afterwards and knew where I'd gone wrong in those cases), but I think I was confident in answers to more than 13 of the questions.

Here's the breakdown that they sent me:

Topic Level Scoring:
1.0 Design for Organizational Complexity: 50%
2.0 Design for New Solutions: 40%
3.0 Migration Planning: 75%
4.0 Cost Control: 50%
5.0 Continuous Improvement for Existing Solutions: 85%

Some of those (particularly Design for New Solutions) are a little painful.

Because it isn't obvious which questions I got wrong and why, that makes it harder to know where to focus my preparation. I think the reality is that I've got a lot of studying still to do if I want to go into the exam with a high chance of passing (as it currently stands I think I've got about 50/50 odds).

I will probably look into other sets of practice questions (even though I'm not convinced that all of them are very realistic) as it may be a good way to expose gaps in my knowledge for services that are likely to be on the test.

Hopefully the next post in the series will be to say how I passed the test. The testing centers are starting to re-open in Ohio, and while I'm in no rush to take the test, the virtual tests seem to have been a nightmare for some people (stories of people waiting hours for the proctor to start the test, and having to purge your room of extra monitors, paper etc.)

DEV Community: Andrew May

Faster, Cheaper - AWS Graviton 2

Supported Services

Where to start using Graviton 2?

APIs are (generally) easy

Open Source applications

Nginx example

Custom Applications

Languages: No changes necessary

Languages: Repackage (sometimes)

Languages: Recompile

Containers

Lambda functions

Graviton Challenge

Looking forward to re:invent

GCP Cloud Digital Leader, a certification in search of an identity

Similar Certifications

Digital Leader?

Preparation

The exam

Summing up

Jamming with AWS

Joining the Jam

The rest of the summit

Feature flags for Infrastructure as Code

Feature what?

Adding flags to your IaC

To create or not to create, that is the question

Testability

Cleaning up

Summing up

Cloud Migration 101: Developer Edition

How things work on-prem

How things change in the Cloud

Become a Cloud Developer

Training and certifications

Mechanical Sympathy

Organizational resistance

A change of role?

You owe it to yourself

The relevance of the AWS Certified DevOps Engineer - Professional certification

Certification topics

Relevance

A convincing case?

stackmanager: Yet another tool for CloudFormation

Functional Requirements

Managing all the configuration used to create/update a CloudFormation stack in a single file

Support for deploying the same stack to different environments (e.g. dev, prod, or different regions) while reducing duplication of configuration

Uses ChangeSets, with control over whether to immediately apply or apply later

Log progress

Support Lambda Functions

Non-functional requirements

Getting to 1.0

Using stackmanager

Is stackmanager right for you?

PyPI release using GitHub Actions

My Workflow

Submission Category:

Yaml File or Link to Code

LeadingEDJE / stackmanager

Utility for managing Cloudformation stacks

stackmanager

Configuration

Single Environment

Additional Info

AWS Lambda Layer for Private Certificates

The problem

The solution

Building and sharing the layer

Other runtimes

Using the AWS Serverless Application Model (SAM)

AWS Serverless Application Model

SAM Templates

Differences between AWS::Serverless::Function and AWS::Lambda::Function

CodeUri

Events

Policies

SAM CLI

sam init

sam build

Differences between `AWS::Serverless::Function` and `AWS::Lambda::Function`

`sam init`

`sam build`

`sam package`

`sam deploy`