DEV Community: James Stratford

Using CloudFormation Custom Resources to react to coded config changes with AWS Lambda

James Stratford — Tue, 20 Feb 2024 13:17:10 +0000

As ever, engineering in a startup is a matter of working out when you can fully invest in a feature, and when to bridge the gap until a more optimal time arises as the scale doesn't make sense yet and there are more important problems to focus on.

One particular set of examples of this we've faced here at Pelago (formerly Quit Genius) is when can you take expanding config out of code and into a simple CRUD system, and when do you have to keep it there. Adding this CRUD system into production typically requires both backend and frontend resources, and having the spare time in both teams is hard to come by!

Therefore, when given the scenario of “we need to tell a third party when a new element is added to a given config enum”, we had to go to the drawing board to prevent further steps being added to the SOP of expanding this given piece of config. I'd seen examples of using CloudFormation Custom Resources for doing on-stack-changes to set up or update other resources - e.g. filling default entries in a DynamoDB tables - and thought that it could be a solution here too.

Having then had to cobble together different pieces of advice in order to get the end solution to work, I thought I’d write this guide so the next person who faces this problem can tackle it even quicker!

How to set up serverless config file.

First off the easy bit. I’m most familiar with using serverless.ts files these days, but the same config should be easily translated to serverless.yml - just the function capability of the custom InputParameter gets a bit more tricky!

The Lambda (which I’ll get into below) can be declared just like a normal serverless function with no trigger.

functions: {
  ...
  myConfigUpdater: {
    handler: 'handler.myConfigUpdater',
    timeout: 90
  }
}

The fun bit is that you then need to declare a custom resource that will point at that Lambda as the user of its ServiceToken. This is done by adding a resource of type AWS::CloudFormation::CustomResource. As noted in the docs - this can actually be modified to a different Custom name if required.

The properties that aren’t the ServiceToken are entirely customisable here. For simplicity I’ve called mine InputParameter and its presence here means that every time the value calculated there doesn’t match up with the most recent value that CloudFormation has (from the last time you deployed the service), that CloudFormation will re-run the Custom Resource, because it sees the property(s) of the resource have changed. Here, the getSumOfMyConfigEnum is a JavaScript function in a shared library that every time the config enum I’m using is added to the sum of the enum values changes. This change therefore means the Custom Resource is re-run when the config enum changes. If you were to want your Lambda to run every time the service is deployed you could chose to do something like InputParameter: Date.now() to just get the most recent Epoch time value. (Presuming you’re deploying less frequently than once a second! 😀)

Worth noting here that the name of the Resource (MyConfigUpdaterResource) doesn’t actually matter config-wise - it’s just for you to keep track of your resources. Also, as commented in the code, you need to change the name of the Lambda function in both the Fn::GetAtt and DependsOn statements to allow for the CloudFormation to attach.

resources: {
      Resources: {
        // this is a custom resource that will update the config when triggered by a change in the input parameter
        // when this parameter changes it tells CloudFormation that a change has occurred and thus the lambda needs to be triggered
        // see further details in the README.
        MyConfigUpdaterResource: {
          Type: 'AWS::CloudFormation::CustomResource',
          Properties: {
            ServiceToken: {
              'Fn::GetAtt': ['MyConfigUpdaterLambdaFunction', 'Arn'], // N.B. if the function name above is changed this needs to be matched.
            },
            InputParameter: getSumOfMyConfigEnum(),
          },
          DependsOn: ['MyConfigUpdaterLambdaFunction'], // N.B. if the function name is changed above this needs to be matched.
        },
      },
    },

Problem: CloudFormation Hanging? Can’t get a response to work successfully?

Now comes the real fun part, getting the code to work alongside the serverless config. Given if you get it completely wrong you can easily be sat with the CloudFormation stack hanging for 60 minutes to timeout, it can be quite frustrating!

Response format

The first issue is the response format. There is a handly little module that’s been built by AWS for handling the response, but as documented in the CloudFormation docs it’s only included when you use the ZipFile property to write your code!

⚠️ “For code in buckets, you must write your own functions to send responses.”

Therefore you need to either copy across the code directly from AWS - or rely on a package such as this one. Though as documented on GitHub, if AWS changes their code then that package needs to be informed of the change - not exactly ideal!

Async Changes

Having made sure I was using the cfn-response package I thought I had finally gotten past the difficult part of the challenge. Little did I know that in an even more obscure part of the internet is a Stack Overflow thread pointing out that this in fact won’t work if you have any async operations happening inside your Custom Resource. This is because the package from AWS is designed to work synchronously. Given I needed to make an API call to our third party as a part of my config changes, this wasn’t going to work for me!

I then found this package called cfn-response-async which solved the issue. The below code is thus a simplified version of what I’ve settled upon. If my async config changes are made successfully in our third party, then we return a SUCCESS and the CloudFormation succeeds. If there’s an error, this is caught and the CloudFormation deployment is stopped.

Further logic can be inserted here to change what’s being done as the handler will be invoked with three different types of RequestType on the event object - these are Create, Update and Delete. Thus you can use a switch statement if you need to do something more complex (e.g. a setup on the create, and then a deconstruction of something on the delete)

Final Pseudocode

import * as cloudFormationResponse from 'cfn-response-async';
...

const customResourcehandler = async (event, context) => {
  try {
    log.info(`Starting custom resource handler`);
    // make the config changes
    await makeMyAsyncConfigChanges()
    await cloudFormationResponse.send(
      event,
      context,
      'SUCCESS',
      {
        Input: event.ResourceProperties.InputParameter,
      },
      event.StackId
    );
    // respond with a positive 
  } catch (error) {
    log.error(`Error updating config: ${error.message}`)
    await cloudFormationResponse.send(
      event, 
      context,
      'FAILED',
      {
        Input: event.ResourceProperties.InputParameter,
      },
      event.StackId
    )
  }
}

Conclusion

So there you have it - a couple of wrinkles to get asynchronous changes done by CloudFormation, but super useful to help us save time in development!

Importing/Exporting Serverless Custom Authorizers Across Services

James Stratford — Wed, 07 Apr 2021 15:50:46 +0000

Ever been faced with the issue of running out of custom authorizers for your Lambdas with Serverless in AWS? We ran out, and needed to consolidate ours for our API which is used across multiple microservices. AWS only lets you have 10 separate authorizers (though you can ask AWS for more, BUT they're quite against this), so this is an issue many will have when following the pattern of using one common API in this way. Indeed it's a problem that comes up on issues on Serverless' GitHub - e.g. here. The Serverless documentation alludes to how to share an authorizer, but doesn't show how to do it across services. In this article, I'll outline how we got round the problem with some examples in both yaml and TypeScript.

Solution

Our solution was to create a common-authorizer service, with the authorizers in this service exported to allow them to be imported in multiple other Serverless stacks.

Planning

The first step was relatively simple, going to the API Gateway Authorizers console on AWS and listing all the authorizers which were currently live, and then comparing this to the authorizers that were being used in various services to see which were performing the same functionality (e.g. both only allowing the same groups on a specific user pool). Collating this list showed we could go from 20+ authorizers down to 8 separate ones. Luckily, the number that were live in prod was 19, and we had 8 authorizers all fulfilling the same purpose, so could go from 19 down to 12 in one deployment (by adding 1 then removed8), then 12 down to 8 in a second deployment.

Common-authorizer Service

From here it was a matter of bringing all the authorizers into one service. At a basic level, this is defining all the custom auth Lambdas in a common-auth service and then exporting a reference for these Lambdas to be used elsewhere using Outputs and Imports. This is similar to how you'd simply reference the Lambda object with the name of the authorizer if it was in the same Serverless service, just with extra steps!

In order to ensure that the authorizer objects (AWS::ApiGateway::Authorizer) themselves are created in the authorizer stack, it's not good enough to just declare the custom auth Lambda functions in the common authorizer service; they need to become Authorizers here too. There's two main options here, either you can explicitly add it as an AWS::ApiGateway::Authorizer object by using CloudFormation inside your serverless.yml file, or you can attach it to a Lambda as the authorizer on an http endpoint. We chose to go for the latter, as this set-up led to the ability to create short integration tests that could test that our authorizers were serving the appropriate groups/pools correctly.

{
   ...
   functions: {
      basicUserAuthorizer: {
         handler: "handler.basicUserAuthorizer"
      },
      testAuthorizerEndpoint: {
         handler: "handler.testAuthorizerEndpoint",
         events: [
            {
               http: {
                  method: 'get',
                  path: 'test/basicAuth',
                  authorizer: {
                     type: "token",
                     name: BasicUserAuthorizer,
                     arn: { "Fn::GetAtt": ["BasicUserAuthorizerLambdaFunction", "Arn"] },
                     resultTtlInSeconds: 0
                  }
               }
            }
         ]
      }
   }
}

We can then export the AWS::ApiGateway::Authorizer object Serverless has created for us to be used in other services.

{
   ...
   resources: {
      Outputs: {
         "BasicUserAuthorizerId": {
            "Export": {
               "Name": "BasicUserAuthorizerId-dev"
            },
            "Value": {
               "Ref": "BasicUserAuthorizerApiGatewayAuthorizer"
            }
         }
      }
   }
}

Where basicAuthorizer is the name of the custom authorizer Lambda function declared in the same file. (Note, Serverless will capitalise the first letter of your Lambda resource's name, hence our Ref is pointing at the BasicUserAuthorizerApiGatewayAuthorizer object)

Other Services

Warning, once you start deploying services which reference your authorizer-service exports you won't be able to change the export name!

Now we have our BasicAuthorizer being exported, we can import it into our other stacks using the magic of the Intrinsic CloudFormation function ImportValue! This can be done like so:

...
functions:
   saveUserAddress:
      handler: handler.saveUserAddress
      events:
         - http:
            path: /example/save
            authorizer: 
               type: 'CUSTOM'
               authorizerId:
                  'Fn::ImportValue': BasicAuthorizerId-${self:provider.stage}
...

or in TypeScript.. (since due to the age of some of our services we have a mix of TS and yaml stacks)

...
functions: {
   saveUserAddress: {
      handler: "handler.saveAddress",
      events: [
         {
            http: {
               path: "/exampleTwo/save",
               authorizer: {
                  type: "CUSTOM",
                  authorizerId: {
                     "Fn::ImportValue": "BasicAuthorizer-${self:provider.stage}"
                  },
               },
            }
         }
      ]
   }
}

As documented here, if you're specifying the AuthorizerId for an API Gateway method (what Serverless is doing under the hood) you must "specify CUSTOM or COGNITO_USER_POOLS for this property"(i.e. the type). This annoyingly is referring to a different authorier-type property than the one we referred to above as a token!

Also worth noting here is that I've used a reference to the stage to allow this to be used across stages. Additionally, due to a yaml parsing issue with our CI and exclamation marks, I've used the more verbose Fn::ImportValue here - if you don't have the parsing issue then feel free to use !ImportValue inline!

And that's it! Your non-auth service should now be pulling the reference to the authorizer using your imported AuthorizerId, and therefore is able to share authorizers with other services! Make sure to test the authorized endpoints to ensure they're still performing as you expect, and once you're happy you can delete your old authorizer code.

Of Note

Integration/Deployment
A small change had to be made to our pipeline process in order to allow for the authorizer stack to always be deployed first. A simplified example for what occurs in dev is as follows:

diff=$(git whatchanged --name-only --pretty="" origin..HEAD)
matches=$(echo "$diff" | grep --color=never / || echo "")
changedDirectories=$(echo "$matches" | cut -d/ -f1 | sort -u)

authorizationChangedDirectories=$(echo "$changedDirectories" | grep auth- || true)
otherChangedDirectories=$(echo "$changedDirectories" | grep -v auth- || true)

echo "$authorizationChangedDirectories" | xargs --no-run-if-empty -n 1 -P 10 ./.circleci/deploy dev
echo "$otherChangedDirectories" | xargs --no-run-if-empty -n 1 -P 10 ./.circleci/deploy dev

What's going on here is we're using git whatchanged to work out the difference between what's on master and what's currently in the branch, obtaining the directories of the files changed, and then splitting that list into auth and non-auth services. These lists are then split up using xargs to call the deployment script.

resultTtlInSeconds
Due to the way API Gateway by default will cache (300s) the specific policy generated by an authorizer for a user, you get 403s on the second and beyond endpoint a user tries to access with an accessToken (if the authorizer in question is returning a specific resource in its policy like ours does).
E.g. if you have a user who tries to access foo/123, then a policy is generated for them saying they have permission to access foo/123, which is good if that user only ever accesses that endpoint. However, if they then want to go ahead and access foo/345 moments later, then the cached policy remains, saying they only have access to foo/123 returning a 403 Forbidden response.

Helper Functions

Part of the beauty of using TypeScript for our serverless file is that you can use helper functions to keep IaaC dry. We made use of two here in the common authorizer service, and then one to reference common authorizers in other services.

Common authorizer service

(1) Create Authorizer objects for the common-authorizer service:

export const commonAuthorizer = (lambdaName: string) => {
   // N.B. lambdaName string must start with a capital letter
  return {
    type: 'token',
    name: lambdaName,
    arn: { "Fn::GetAtt": [`${lambdaName}LambdaFunction`, "Arn"] },
    resultTtlInSeconds: 0
  };
};

Which when lambdaName is the name of the Lambda function you want to become an authorizer means you can simplify your events property of your testAuthorizerEndpoint

events: [
   http: {
      method: 'get',
      path: 'test/basicAuth',
      authorizer: commonAuthorizer("BasicUserAuthorizer")
   },
   http: {
      method: 'get',
      path: 'test/powerAuth',
      authorizer: commonAuthorizer("PowerUserAuthorizer")
   }
]

(2) Create Authorizer exports for the common-authorizer service to simplify your Outputs:

export const exportAuthorizerId = (authorizerLambdaName: string, stage: string = '${self:provider.stage}') => {
  // N.B. authorizerLambdaName string must start with a capital letter
  return {
    Export: {
      Name: `${authorizerLambdaName}Id-${stage}`
    },
    Value: {
      "Ref": `${authorizerLambdaName}ApiGatewayAuthorizer`
    }
  };
};

Which when authorizerLambdaName is the name of the custom Lambda function, can then simplify your exports section to look like this:

{
   ...
   resources: {
      Outputs: {
         basicUserAuth: exportAuthorizerId('BasicUserAuthorizer'),
         powerUserAuth: exportAuthorizerId('PowerUserAuthorizer')
      }
   }
}

Other services

(3) Create Authorizer imports for other services to be able to use the authorizers from the common authorizer service.

export const authObject = (authorizerName: string, stage: string = '${self:provider.stage}') => {
  return {
    type: 'CUSTOM',
    authorizerId: {
      'Fn::ImportValue': `${authorizerName}Id-${stage}`
    }
  }
}

Which when authorizerName is the name of the required custom Lambda function from the common authorizer service can simplify your authorizer objects to look like this

functions: {
   saveUserAddress: {
      handler: "handler.saveAddress",
      events: [
         {
            http: {
               path: "/exampleTwo/save",
               authorizer: authObject("BasicUserAuthorizer")
            }
         }
      ]
   }
}

An extra bonus here is you can change the type of the authorizerName parameter to a custom type that you define as the names of your authorizers to ensure you're using the correct names!

Thanks for reading

If you got this far, thanks for reading - I hope this helps you if you're stuck with the same situation we were!

Further Info

Serverless's documentation on variables - notably those on Cloudformation stack outputs

Yan's guide to selecting your API Gateway auth method - we're in the bucket of using group-based authentication therefore custom Lambdas.

Building a Serverless (UK) Property Helper using Zoopla - Part 1: Not Serverless Yet

James Stratford — Sun, 19 Jul 2020 22:19:57 +0000

UPDATE Annoyingly, whilst continuing to work on this, I have gotten stuck due to the details listed here: https://medium.com/@mariomenti/how-not-to-run-an-api-looking-at-you-zoopla-bda247e27d15. To cut a long story short, Zoopla no longer actively support the API, which means that the API keys just randomly stop working with no warning. Back to the drawing board I suppose...

Introduction

For a little holiday project, I wanted to try and build something to help with a non-work thing I've found has been inefficient - property searching. My partner and I are hoping to buy a place soon, and we've found it frustrating trying to keep up with property pages, plus who doesn't like finding something that annoys them a little, then spend hours trying to get rid of that itch!

This article is the first of a few (depending on far I take it), and I'll add links to subsequent docs at the bottom. Things that will be covered later will include integrating with the Airtable API for a good spreadsheet + images view, using Lambda and Cloudformation to make the work repeatable for others, then using Cloudwatch Events and Eventbridge to automate the process!

Initial Research

My first thought was a web scraper might be a a good option, and the site I use most frequently when looking at properties is Rightmove.
However, a little googling quickly lead me to this section of Rightmove's T's & C's:

You must not use or attempt to use any automated program (including, without limitation, any spider or other web crawler) to access our system or this Site. You must not use any scraping technology on the Site. Any such use or attempted use of an automated program shall be a misuse of our system and this Site. Obtaining access to any part of our system or this Site by means of any such automated programs is strictly unauthorised.

From Rightmove's terms of use

So that was the end of that train of thought...

Zoopla is another commonly-used property site in the UK, so this was the next option. They have a refreshing opinion on third party development, and have an extensive API with what I've seen so far to be a good set of docs, so this seemed like a good place to start!

Broad Plan

The first version for this won't be serverless, just to make sure I'm getting things right. I want to be able to query Zoopla using a script, via the Node REPL, and then be able to send the results to a suitable place for viewing. I've toyed with a few ideas, but I think to begin with Airtable (on the free version!) should be good enough. This does mean this won't be a totally serverless solution, but we can potentially replace it further down the line. From there, I'll use Lambda to do the communicating with Zoopla/Airtable, and then set it to run at a certain frequency using a Cloudwatch cron-event.

Development

Step 1: Zoopla Test

First step here was to register for the Zoopla Developer API for the API key required to make queries. This was fairly simple, just needing some standard details and some inclination of what you want to do with it.
Then, to see what results I would get I quickly tested the Property Listings Endpoint using Postman.
Adding a bunch of fields which I felt would be useful (postcode, radius, listing_status, maximum_price, minimum_beds and of course the api_key), revealed a quite extensive result (see below).

Step 2: Prototype Code

Though I want to get this into Lambda, I thought it best to try with running a Javascript-script locally to repeat what i've done with Postman, and allowing me to separate out the params for the request.

In order to separate the logic of the params being used for a specific query and the querying itself, for now I have written the params I want to test with to a local params.json file that looks like this:

{
  "api_key": "INSERTAPIKEYHERE",
  "radius": 1,
  "listing_status": "sale",
  "maximum_price": 1221000,
  "minimum_beds": 1,
  "postcode": "NW1 6XE",
  "page_size": 10
}

(I have of course changed the demo params here to be those for Sherlock Holmes, who due to house price rises will now have to do with a budget of 0.001221b£ for a property on Baker St.)

These params can then be used, along with axios, to query the Zoopla endpoint like so:

const axios = require("axios");
const fs = require("fs");

const propertyListingUrl = "https://api.zoopla.co.uk/api/v1/property_listings.js";

const getParams = async () => {
  const params = await fs.readFileSync("./params.json");
  return JSON.parse(params);
};

const buildConfig = async () => {
  return {
    params: await getParams(),
    url: propertyListingUrl,
    headers: {},
    method: "get"
  };
};

const axiosRequest = async config => {
  const result = axios(config);
  return result;
};

const queryZooplaPropertyListing = async () => {
  const config = await buildConfig();
  const result = await axiosRequest(config);
  return result.data;
};

module.exports = {
  queryZoopla: queryZooplaPropertyListing
};

Here, in the main queryZooplaPropertyListing function we are building our config, which involves reading in our params from ./params.json, then we are using this built config with axios to request the property listings from the Zoopla url.(Note I've appended .js to the url in order to receive a JSON response!)

This uses Node's async-await functionality, as both the file-reading, and the Zoopla request itself are asynchronous processes.

After the promises have resolved, the exported queryZoopla function should then returns an object which looks like this:

{
    "country": "England",
    "result_count": 196,
    "longitude": -0.158541,
    "area_name": " NW1",
    "listing": [
        {
            "country_code": "gb",
            "num_floors": 0,
            "image_150_113_url": "https://lid.zoocdn.com/150/113/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "listing_status": "sale",
            "num_bedrooms": 2,
            "location_is_approximate": 0,
            "image_50_38_url": "https://lid.zoocdn.com/50/38/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "latitude": 51.525627,
            "furnished_state": null,
            "agent_address": "24 Charlotte Street, London",
            "category": "Residential",
            "property_type": "Flat",
            "longitude": -0.162988,
            "thumbnail_url": "https://lid.zoocdn.com/80/60/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "description": "360' virtual tour available. A very well presented second floor apartment set within a popular gated development located just moments away from the open spaces of Regent's Park, Baker Street & Marylebone stations and numerous shops, bars and restaurants. The property is quietly located overlooking the courtyard gardens comprising two bedrooms, two bathrooms, a reception room, seperate kitchen with integrated appliances and 2 x private balconys. The apartment is sold with an underground parking space. As a resident of the building benefits include concierge, access to a communal gym, a swimming pool and landscaped communal gardens. Alberts Court is set within the modern Palgrave Gardens development to the west of Regent's Park. The Leaseholders are currently in the process of purchasing the freehold.The building provides easy access to the West End, The City and various transport links around and out of London.",
            "post_town": "London",
            "details_url": "https://www.zoopla.co.uk/for-sale/details/55172443?utm_source=v1:8aaVEj3AGALC-xWzf7867y2rJwMs0-2Y&utm_medium=api",
            "short_description": "360' virtual tour available. A very well presented second floor apartment set within a popular gated development located just moments away from the open spaces of Regent's Park, Baker Street &amp; Marylebone stations and numerous shops, bars (truncated)",
            "outcode": "NW1",
            "image_645_430_url": "https://lid.zoocdn.com/645/430/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "county": "London",
            "price": "1200000",
            "listing_id": "55172443",
            "image_caption": "Picture No. 13",
            "image_80_60_url": "https://lid.zoocdn.com/80/60/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "status": "for_sale",
            "agent_name": "Hudsons Property",
            "num_recepts": 1,
            "country": "England",
            "first_published_date": "2020-07-09 08:44:51",
            "displayable_address": "Alberts Court, 2 Palgrave Gardens, Regent's Park, London NW1",
            "floor_plan": [
                "https://lc.zoocdn.com/4cb0366075b14e99efe3a1a7b24a608f4c7a92f0.jpg"
            ],
            "street_name": "Alberts Court",
            "num_bathrooms": 2,
            "agent_logo": "https://st.zoocdn.com/zoopla_static_agent_logo_(62918).jpeg",
            "price_change": [
                {
                    "direction": "",
                    "date": "2020-06-28 22:30:07",
                    "percent": "0%",
                    "price": 1200000
                }
            ],
            "agent_phone": "020 3641 7089",
            "image_354_255_url": "https://lid.zoocdn.com/354/255/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "image_url": "https://lid.zoocdn.com/354/255/2cd80711fb52d57e85068b025920836abb906b89.jpg",
            "last_published_date": "2020-07-09 08:44:51"
        }
    ],
    "street": "",
    "radius": "0.5",
    "town": "",
    "latitude": 51.523659,
    "county": "London",
    "bounding_box": {
        "longitude_min": "-0.170158861045769",
        "latitude_min": "51.5164304665016",
        "longitude_max": "-0.146923138954231",
        "latitude_max": "51.5308875334984"
    },
    "postcode": "NW1 6XE"
}

And voila. A swanky 2 bed, 2 bath property near Baker St for Sherlock to relocate to! With a whole heap of extra data to boot. Evaluating this will be the first part of the next step, as we aim to get this data into Airtable, so stay tuned!

You can see this code in full at https://github.com/jcts3/serverlessPropertyHelper/tree/workingLocalQuery

How to use the new AWS HTTP API with SAM

James Stratford — Fri, 06 Dec 2019 15:51:24 +0000

Hi, I'm new to the world of serverless this year, and in my role at Solve I've been working on trying out some of the newest additions to the serverless world announced at this year's AWS re:invent to see if they suit us.

One of these new announcements that we are stoked about is the new HTTP API service. As our new app is to have a number of services driven out of AWS, all you had to say was 'faster' and 'up to 70% cheaper' to get our interest! To test it out we picked a service that was going to be particularly busy, and I've been trying out how to implement it with SAM. It ended up being a non-trivial fix, and there were no docs for multi-endpoint services - so I thought I'd write up what I found!

Previously on our API...

Our old API had a POST and GET endpoint, both to be served by lambda functions which would do call(s) to DynamoDB. The SAM template looked like this:

Resources:
...
  GetOneOfTheThing: 
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getOneThing
      Events:
        GetOneThing:
          Type: Api
          Properties:
            Path: /getOneThing/{Key}/
            Method: get
            RestApiId: !Ref MyApi

  PostManyKeysForManyThings:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.postGetManyThings
      Events:
        GetVariationDetail:
          Type: Api
          Properties:
            Path: /getManyThings/
            Method: post
            RestApiId: !Ref MyApi  

  MyApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: !Ref Stage
      MethodSettings:
        - ResourcePath: "/*"
          HttpMethod: "*"

Initial Template Changes

Simple steps from the documentation:

Change the event type from Api to HttpApi for each of the lambda objects
API object type from Serverless::Api to Serverless::HttpApi
Remove MethodSettings from the API object. As described in the docs the properties of this object are a lot simpler than the previous one. For a single API endpoint, this Serverless::HttpApi object isn't actually needed when using SAM, as it can work out how to fill in the gaps. However, I found that when you wanted to use the HTTP API with two endpoints, this no longer caused a connection between the Lambdas and the new API.

Subtler Template Changes

The things that caught me out from there were:

I needed a DefinitionBody for the HttpApi object. There was no real guidance for this beyond an example of a root path option, so after some tinkering what I found to work was as follows:

      DefinitionBody:
        info:
          title:
            Ref: AWS::StackName
          version: "1.0"
        openapi: 3.0.1
        paths:
          /getOneThing/{flagKey}:
            get:
              responses: {}
          /getManyThings:
            post:
              responses: {}

The catch here, was that the path options couldn't end with a /, therefore the lambda events' paths needed to edited to remove the trailing backslash. (i.e. Path: /getOneThing/{flagKey}/->Path: /getOneThing/{flagKey})

  GetOneOfTheThing: 
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getOneThing
      Events:
        GetOneThing:
          Type: HttpApi
          Properties:
            Path: /getOneThing/{Key}
            Method: get
            RestApiId: !Ref MyApi

  PostManyKeysForManyThings:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.postGetManyThings
      Events:
        GetVariationDetail:
          Type: HttpApi
          Properties:
            Path: /getManyThings
            Method: post
            RestApiId: !Ref MyApi

Conclusion

And that was it! No changes at all to the lambdas themselves, and only one change to a test, as the sad-path test of no key being provided to the GET was no longer being caught as a 403 by API Gateway was now returning a 400 by the error catching inside the lambda.

Caveat here of course is that HTTP API is still in beta so some of this may be subject to change, but we are looking forward to its benefits nonetheless!