DEV Community: Mick Jacobsson

One simple trick to win Hackathons

Mick Jacobsson — Sun, 31 Mar 2024 00:36:11 +0000

Everyone has been approaching the AWS PartyRock Hackathon wrong in my opinion. Why try and think of an idea and create a new AI app when you can just use AI to think of the winning idea and write a summary to enter? You can't lose!

AWS PartyRock

AWS is recently hosted a hackathon (ended on March 11, 2024), where participants were challenged to create an AI app using the PartyRock platform. Essentially, this involves leveraging AWS BedRock technology and offers a fun opportunity to explore AWS's generative AI services.

If you're familiar with hackathons, this one follows the same format. Contestants will compete in various categories and have the chance to win CASH prizes. So, if a hackathon has a specific format, can we use AI to help us win?

Hackathon Hack a.k.a HackaThong

My idea is straightforward: you give the user inputs that I'll use as context for the generative AI to create a winning suggestion. Instead of making this specific to just the AWS PartyRock competition, I'll make it more generic so that it can be reused.

Why HackaThong? I'm Australian, and we refer to rubber-style casual footwear, similar to sandals, as thongs or pluggers.

What do hackathons have in common?

Summary of the hackathon
Terms and conditions
Criteria for judging
Examples (often provided to inspire creativity)

This should give enough context for our AI to offer suggestions, but what about the output?

A winning idea
A catchy, unique name
Some artwork to help visualize the idea
A written summary describing the app
An application to the competition that we can use as a starting point

Time to build a thing

I'm going to quickly explain how I built this with PartyRock. But first, I want to recommend a few articles that cover this topic in more detail (and better) than I will:

I'm going to start with an empty app and just start adding widgets:

Instead of starting with an empty app you can write a description and PartyRock will have a crack at creating the app you might need.

Based on what we know, we need a lot of input from the user. So, for the first widgets, we'll create several user input types:

Hopefully this is straightforward, title, placeholders etc.

The first couple of user input widgets are straightforward: they're simple text fields where the user can provide context about what the AI needs to do. Very little setup is neededjust name it and hit save.

This is where the fun starts. The next few widgets will use the inputs from above to perform the AI magic. We then create an AI-Powered widget and link it to the previous input widgets.

A few things to note:

You can change the model based on your requirements, there is a brief description of the models here
You design your prompt

From this point, we continue the process, creating more widgets until the app looks something like what we intended to develop.

Let's talk about the fails

The AI kept suggesting the same thing. For example, it would suggest a cooking app every time. I had to reword the prompt to resolve this, and even so, it can provide things that are pretty similar.
There is a word limit. Not too surprising. The terms and conditions are super long (expected), so there is a point in the T&Cs that I truncate to accept most of the content.
Initially, regenerating ideas was a bit dumb because I'd simply update the inputs. But, there is a little refresh button on the widget to resolve this.

Try it out

You can try out my app on AWS Party Rock, it's a little bit of fun. Your mileage may vary on how useful it is to win a hackathon with, but hopefully, you can see some potential in generative AI.

Wtf is construct hub

Mick Jacobsson — Wed, 14 Feb 2024 01:22:40 +0000

If you've ever seen my github repo you'll know there are a bunch of cdk style projects in there that are usually paired with a blog explaining the details. These are examples of how to develop your own cdk apps but aren't a good example of a reusable construct.

Construct hub was launched towards the end of 2021 (oof, that long already), I vaguely remember the announcement and was keen to understand how it might work. I've always been aware of construct hub, but, tbh it's not front of mind when I'm looking at CDK and I haven't contributed to the catalog.

Let's take a look at construct hub, I'll also provide a very basic example repo that should help others get into construct hub if they wish.

What is construct hub?

What I'm talking about is constructs.dev aka construct hub.

Find and use open-source Cloud Development Kit (CDK) libraries

OK, so what this sounds like is a catalog of all CDK things. This reminds me a little of the npm registry but just for CDK.

When I got to the homepage I notice a few things:

A global search to find CDK libraries
A search by publisher which appears to list top libraries split by community, AWS and Hashicorp
Feature packages

So this looks pretty easy navigate we can assume that if I publish to construct hub that I would appear in the catalog as a community package. It makes sense that there would be other categories e.g. AWS and Hashicorp. When you consider the AWS CDK library this would fall under AWS and Hashicorp makes sense for the terraform providers. Terraform gets a little messy because the providers are often developed by third party companies.

When developing cdk constructs I'm typically following the IntelliSense breadcrumbs and links to the AWS CDK API reference. It looks like constructs hub provides that reference as well (guessing standard doc generation).

TL;DR

Construct hub is a catalog of all CDK libraries that have met the requirements (think package tags, license etc) to be ingested into the catalog. Construct hub provides a standard interface to surface documentation and links about the library that should help developers adopt and contribute to the library.

Note: I mention NPM registry above as a comparison which is still correct imo, but one of the benefits of CDK is the jsii compiler. The benefit here being a library publishing to several different registries e.g. npm and python all discoverable from the same source.

How do you get something into construct hub?

This wasn't super obvious to me, I was guessing it was something along the lines of tagging your project with the correct tags (think npm style package.json) and that would be enough. But, there are a few more things.

If you head over to the contribution docs on construct hub you will see the requirements:

Must be published to NPM
Must have license (certain types)
Must use jsii

There is also a link to a guide that someone has written that probably explains this better than I can, be sure to check it out.

If you do all this right you should see your construct appear in construct hub within about 30 minutes. construct hub seems to be polling at this interval for changes.

While the docs don't say you must use projen it does seem geared around using projen. There are benefits here, projen makes the whole jsii part a tad easier imo. But, projen is super opinionated to the point where it can be frustrating to use. It will come down to where you and your team are at if it's going to work for you. I wrote a piece a while back about project templating (scaffolding) where I touch on projen.

Hello Construct Hub, an npm package

I think the best way to understand construct hub will be to publish something into construct hub. We know we need an NPM package and TypeScript works best for me. I'll be using projen to help get started knowing that we need to use jsii, projen will take care of a few things for us (github actions, build, etc).

I've created a new package hello-construct-hub and published to construct hub. The basic package is my take on the classic hello world, what is a basic IaC package in AWS I wonder? A bucket seems to be the best example I could think of.

The construct will:

Extend the existing AWS S3 Bucket Construct
Take a single property: name, this will be to...you guessed it name the bucket
All defaults remain the same for a bucket (refer to parent class)
A bucket deployment will upload the readme.md into the bucket

Diagram

I like diagrams, here is a totally unnecessary simple diagram showing what the new construct provides:

Now, users can use the published package (npm) to create a bucket with a semi predefined naming standard. Room for improvement but you get the idea. You get a simple consistent naming pattern for buckets with a single doc that might help users understand the intent of the bucket.

Because we are using jsii we would also be able to publish this to python as a pip package.

What issues did I find?

Just projen things, like I said projen is very opinionated so I mainly had issues fighting my existing linters and working within what projen wants me to do (I realise you can override these).
Github actions failed due to the self mutation action that's added in projen. What this does is run the build steps in an action and compare the output. If there are differences it will commit the changes to your branch. To do this you need a new token in your pipeline (PROJEN_GITHUB_TOKEN). This is optional, the step won't pass until fixed though if you care. The changes are likely docgen API docs that will appear in construct hub which I wanted. Also doing this locally would avoid this.
You'll need a NPM registry token to publish your package, not an issue to be expected but just calling it out.
Update your tests, you'll have a boiler plate test that will start failing after you update your construct so be sure to update your tests (not that you'd skip this part...am I right...eh).

Final thoughts

Construct hub is going to suffer the same problem as other registries. Any projen package rightly or wrongly is going to end up in construct hub which may bring the quality of content down or make it harder to find what you're looking for. I wouldn't mind seeing another tag or something as a default for those projects who want to make that effort to publish.

My initial thoughts before digging further into this was that I didn't really see the value in what construct hub was doing. All of this is available in the various registries that you probably already hang out in. But, once I started using it more and understood how things were presented, I concluded that I really should be using this as my go to. I like the standard layout with links to the repo, I like that the AWS CDK library is available here as well and I think the presentation is slightly better than the AWS version.

I think my concerns over using projen with construct hub aren't really a problem, in the end you shouldn't need to use projen to get into construct hub. Just do all the things you need and it will appear. The AWS lib is an example of this.

If you're a heavy CDK TF user I think that the Hashicorp "developed" providers are a little misleading, it would be nice if the original developer flowed through to the platform. The issue likely stems from upstream at the TF provider generation in the AWS CDK TF project. This is minor.

One last thing that is worth a mention is that if you're starting out with CDK this is a good resource and links to other good resources. So when the questions come your way about getting started you can point them at construct hub and they should be able to get started.

TL;DR

Yeah, should probably start using construct even if its to consume other libraries.

Here is a link to the github repo: https://github.com/talkncloud/hello-construct-hub you'll find the construct hub link in this article or in the repo.

Image banner credits: Photo by Matt Ridley on Unsplash

The death of forestry.io

Mick Jacobsson — Sun, 04 Feb 2024 04:24:31 +0000

When I migrated from Ghost Pro to hugo I stumbled across forestry.io, a CMS for my static site. It seemed to be everything I was looking for, something to bridge that small gap between writing markdown content and having a nice editor with workflows. It wasn't perfect but I stuck with it.

What has replaced forestry.io and is it any good?

Why though?

Like I said, It bridges a small gap. You don't need forestry, you can just use the IDE of choice and churn out markdown but it gets a little old. I like the finer things in life and one of those is lightweight CMS.

Something we should clear up before we go further. I'm not happy about this at all. I'm only "trying" out forestry on a small single site for personal use. Considering they've ended forestry and moved onto another project is something to consider before moving to the alternative. Sh*t happens.

I'll talk about something that I'd like when we wrap up at the end.

TinaCMS

If you've used forestry.io you would already know that it has been replaced with TinaCMS.

The Forestry.io team is focused on building TinaCMS, the next iteration of our vision. See the migration guide or test it with the following command.

Not really any clues here as to what has happened. The internet probably knows but I'm not chasing the juicy details, I want progress. That is TinaCMS for now so we'll take a look a migrating.

The good news is that TinaCMS appears to be built some sort of migration around forestry to the new "vision". So lets try that and see how it goes.

Another migration

Yes, I was less than thrilled at migrating to something else.

Good news, the docs look pretty decent. Easily to follow and they include videos for those who prefer that approach. Following the migration guides we can see it's pretty simple, go to your web root:

npx @tinacms/cli@latest init

Pretty much just answer Y to everything.

What you'll hopefully end up with is a new tina configuration file with package management and some shiny new additions to your gitignore. The main config.ts is where I've been finding most of the interesting bits and pieces, I had to play around with the path the most for both root and media. So check that out.

Next you'll need to start your local development server, tina has something for that too, this is what it looks like for my local hugo:

npx tinacms dev - c "hugo server -D -p 1313"

This will launch the web server locally and you can access the web admin site on...you guessed it http://localhost:1313.

Now, if everything went OK you should see your content. You'll see your existing pages, media etc.

Tip: Mess around with the config.ts path for content and media if you don't see your site content.

Using Tina

The user interface isn't too exciting, not that it needs to be. Creating a new page is straight forward, because Tina completed a migration I'm assuming it has collected all of my previous key fields (can be seen in config) and now they become inputs for a new page. For example, I have a field named type which I use to manage drafts and features. This helps me to work on pages over a period of time without making a mistake a making it live. The other is to push pages to the top that i want featured, this is usually my latest articles.

The user interface looks something like this:

Wen Cloud?

Tbh, I'm little hesitant to fully invest time into TinaCMS right now. I'm grateful that the migration was pretty smooth but generally didn't like having to migrate and spend time on this. I was using forestry cloud for a while and preferred the hosted cloud admin of forestry even with it's problems. For now, I'll continue to test out using TinaCMS with the local admin to update content and think about how I want to move forward.

I don't mind the idea how hosting the admin part of out my hosting in AWS but would need to manage secure access, which is something I wanted to avoid for this site. I like to keep this isolated where possible to some extent.

Is there another way?

What do I really want in a content editor. Over the years I've used enterprise level CMS and have hated them but they usually have a place. I've used open source ones which have been somewhere in between and usually more acceptable given the cost.

When I first started going down the path of static sites I was hooked, this is the way. Fast, affordable hosting of static content, but managing that content is a bit clunky.

I don't want a hosted CMS to manage my content. I'm a simple guy, I wonder if there is something in Visual Studio Code that can provide the editor experience I want coupled with a local development server for viewing changes...

I'd be keen to hear if you come across this already and have any suggestions???

Quick mention for further reading, this site is hosted on AWS for cheaps, like cents, I wrote an article about how I do that if you're interested: https://www.talkncloud.com/ghost-pro-migration-hugo-static-aws/

Image banner credits: Photo by Marcelo Novais on Unsplash

No fuss no muss TypeScript Lambda with CDK

Mick Jacobsson — Sat, 18 Mar 2023 04:25:29 +0000

One of the great things about AWS Lambda is the flexibility to develop functions in a bunch of different languages like Python, Node.js, Go and some others. CDK allows us to easily create and maintain these functions (among other things) but if you need to build any assets or use typescript for Lambda you'll need to handle this before you can deploy the function. This isn't uncommon and many folk would be doing this already in pipelines etc but there is a super easy way to do this in CDK that might just come in handy...

AWS CDK Lambda

Creating a new Lambda function in CDK is pretty straight forward, it only takes a few lines of code:

new lambda.Function(this, "Function", {
  runtime: lambda.Runtime.NODEJS_18_X,
  handler: "index.handler",
  code: lambda.Code.fromAsset(path.join(__dirname, "lambda-handler")),
});

Source: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda.Function.html

This of course will work perfect fine for any Lambda's that you've written that don't need any further build steps or you've already built.

But what if...

The other way with typescript

The nodeJsFunction CDK construct (available in the aws-lib) provides build capability with esbuild. I recommend checking out the documentation it goes into great detail covering a bunch of different scenarios.

If you follow the construct way of structuring and naming your files you can create a very simple typescript lambda with a few lines of code, like the previous example but this also builds:

lambdaFunction = new lambda.NodejsFunction(this, "demo", {
  functionName: "talkncloud-demo-function",
});

The above code snippet will build the typescript lambda if it's located in the same directory as your stack and follows the same naming convention with .id on the end. It is also expected that the handler is named handler, you've probably seen this in the other constructs.

Of course, you don't have to do it this way, you can structure your project however you want, you'll just need to update the props to your location (see my code example at the end).

Building

To use this method of building you'll need to either have esbuild installed or docker. If CDK can't find esbuild it will run a container with everything needed to run the build. Pretty sweet.

There are no extra steps required to run the build, this construct works in nicely with your existing workflow, simply run CDK synth:

cdk synth

CDK Synth will automatically run esbuild with any specific configuration you've supplied and generate the build files.

Testing

If you want to test your new typescript lambda locally you can easily do this with sam. To this you'll first need to be authenticated (aws login) and have docker running, for tips on easy authentication tooling check out my previous post. Oh, and you'll need sam.

sam local invoke demo -t cdk.out/MyStack.template.json

You can find more documentation on locally testing Lambda with sam including testing with events in the official docs.

Note: this assumes you've run synth first

Final thoughts

The nodeJsFunction is handy way for you to leverage esbuild that fits in nicely with CDK. The construct has been built in a way that provides a lot of flexible and consideration for different requirements such as mono repos, customization and package dependencies. If you're already using a different build process or you're looking for alternative to smooth out the process I recommend checking it out.

I've added some sample code to the talkncloud repo, if you'd like to check it out it will deploy a working lamdba function written in typescript using the construct mentioned in this post.

https://github.com/talkncloud/aws/tree/main/lambda-nodejs

Are you already using this construct, how's it going for you, any tips?

Credits: Article photo by Jametlene Reskp on Unsplash

Easy as SSO tooling with Granted AWS

Mick Jacobsson — Sun, 12 Mar 2023 06:05:17 +0000

No doubt about it, AWS SSO (or should I say IAM Identity Centre?) is a great addition to the overall access management and security in AWS. But, as you mature in the cloud with a touch of AWS Organizations and dash of well-architected framework you'll soon have many AWS accounts and managing all of those accounts kind of sucks.

I should mention, I really didn't want to change tooling again, I was using a bunch of different tools to manage CLI and web access and the thought of configuring a new tool or tools didn't sound like a fun time but I was also very much over the default tools from AWS.

What's the problem?

OK, let's set the scene using the example above, you have multiple AWS accounts in the same organization. Let's say you have development, non-production and production accounts, each one being a separate AWS account.

Web Console

Using a browser (choose your poison) jump over to your AWS SSO start url and select the account and role to access the management console. This should launch a new tab or window with the management console for that account, now do it again for another account...the previous sessions is no longer valid. Let's say you wanted to compare development to production in the management (pretty common), you can't easily do that (incognito / private).

Local CLI

This one is less of issue but you can use the aws cli to start sessions using your organization SSO just fine. It can just get out of control quickly when you have many accounts, you have to review your config or just remember the names of your profiles.

Previous workarounds

Before SSO there were (are) a few brilliant projects out there that really helped out:

AWS Extend Switch Roles Chrome browser extension

This is a neat tool that would label your browser session with a different colour and name which made it obvious which account you were in. I really enjoyed this part of the extension as well as being able to switch accounts easily. It didn't support AWS SSO sometime ago but there is mention of it now.

YAWSSO cli tool

A handy little tool to sync up your SSO session tokens with the old style AWS CLI tokens to make it work with other tools like AWS CDK.

You start to get the idea, this is just a couple of examples, I didn't mention any vault or azure login tools which come to mind. The state of account access and management is spread across many different projects and is a delicate ecosystem.

If I were to summarize what a solution might look it:

Handle multiple concurrent sessions in the web based management console
Easily identify which account and role you're using
A better way to manage CLI access for multiple accounts without having to remember all of the details
Ideally one tool to manage the lot without workarounds for syncing etc

The search

Like most problems I started with the typical search for AWS SSO CLI and console related tools to help out. One of the write-ups that stood out for me was from Corey Quinn - taking aws logins for granted (fun fact: the title of this article was almost identical without me even noticing, you win this round Corey). The article really hit home the problems I was having and suggested the use of Granted (github link) (Granted.dev has some nice info).

Now for me, Granted really solved all of the shortcomings in the AWS CLI and web management console particularly around multiple accounts but also added a few nice features I didn't know I needed until now...

Granted AWS Logins

When I think about Granted it does everything that I wanted and more, it's super easy to install, super easy to configure and super easy to use. I'm not going to go into much detail on the install and configuration, the Granted docs are excellent and say it better than I can.

I'm going to assume you already have the aws CLI configured, I won't really be talking about creating configs in details.

Installation on MacOS is a simple brew tap and install, this will add the developer repo and install the Granted tools. Once it's installed you'll need to run through the configuration.

Before you begin I strongly recommend installing Firefox for your browser as containers are better than Chrome's incognito. Do you self a favour and install Firefox.

Once you've installed Granted the configuration is pretty simple, you'll notice a program granted which can be run from the terminal. When you run this the first time a CLI wizard will guide you through the steps to auto-configure your granted setup using your existing aws configs and add the extension to your browser.

This typically takes less than a minute.

Configuration done....

To use granted you will use the assume command from the terminal, the assumecommand will provide a list of profiles that you would like to assume. If you don't have a valid token already it will run you through the login setups (similar to AWS SSO), if you do have a valid session it will refresh your token.

Here is a screen shot from the granted docs showing what happens when you run assume, notice the dropdown like menu, this is ordered by frequency (configurable):

If you want to start an AWS web management console session you just add a parameter assume -c. The screenshot from the granted docs below showing what Granted looks like in the browser:

Notice the different colour lines in the tabs, one is blue and one is orange, notice the role-b in orange with an icon. Helping you to identify which session you are in but also allow concurrent sessions, each running in it's own firefox container.

You wanted more?

Like I said, Granted does much more and most of the options are configurable to some extent:

Encrypted session tokens (Keyring for Mac)
Customise colours and icons
Sync your profiles to a repo to share with your team (we all want that)
Dot env variable export for those local development times
Don't have any SSO configs yet? Granted can generate them from your SSO session

And the list goes on...

Final thoughts

Stop what you're doing and setup Granted, not tomorrow, not next week, just do it now. I promise it is a simple as it sounds and you'll be wishing you had done it sooner. Everything just works and it greatly improves your workflow and easier to work with others using it also. Be sure to check to the Granted docs if you want specifics on what was discussed today. Shout out to Corey from Last Week In AWS for the initial write up which got me headed in the right direction.

Do you have an alternative that works for you? Maybe better than Granted? Please share, I'd love to hear from others about what works for them.

Credits: Article photo by Studio Blackthorns on Unsplash

Project templating cloud

Mick Jacobsson — Tue, 21 Feb 2023 21:49:07 +0000

So you've started using AWS CDK, you've shipped a few projects, you've shared your findings with your team and now they've started using CDK...then you start to notice the subtle differences in how developers have created their projects, dot files, style, folder structure and the usage of custom constructs.

How do you start a new project? "Easy as, I just copy and paste it from a previous project and change out the bits I need"

You start to see the problem, If we use a typical CDK project as an example, you might have AWS CDK constructs, your constructs and your team dot files plus any other custom configs needed, CI/CD etc.

How do you promote consistency for apps outside of just constructs that isn't too hard to maintain and easily shared?

What can we do?

I reached out to the devtools folk in AWS Community Builder, surely there are others who have faced the same problem? What kind of tools were others usings? Not too surprising, projen got a mention as did cdk init and older popular choices like yeoman

I next combined the options from above with others found online:

Name	Comments	Language
CDK init	Built-in scaffolding tool	typescript
Projen	Popular in the CDK community	typescript
Hygen	Newer tool, aiming to tackle project local templating (not limited to)	javascript
Yeoman	Has been around for a long time, stable, good understanding of the problem	javascript

Key considerations

When I talk about templating or scaffolding there are a few things that come to mind that would be considered, key considerations. It can be easy to suggest that templating is a simple file copy and the need for more developed tools might be unnecessary. Also worth noting that often templating, scaffolding, project generators are synonymous with each other. I'll try to highlight some of the key considerations but as you read you might start to think of more:

Composition

Something that I identified in the earlier example is code reuse, once your team have a working project that is agreed within the team, we just want to use that for all similar projects moving forward. We want to layer or compose many templates or files to produce a single project output.

Something I touched on earlier is code reuse, once you have a working project you want to identify the different layers of your template. The project generator that you end up using should help support that workflow. We want to compose many layers or templates that produce a single project output. An example for CDK might be an empty CDK project and then more complex patterns on top e.g. web app, web API etc. The final template is one you'd likely change the most and the base CDK template could be updated, those changes would be consistent across all projects that use the base layer.

Placeholders

When I refer to placeholders I refer to string replacement at a minimum but often this includes transformation e.g. to uppercase. Because our templates are generic you will often want to replace generic placeholders like a project name with the actual name. You may have seen examples of this such as handlebars e.g. {{ swapThis }}

Global vs Local

A global generator would live outside of the project, its likely installed as an NPM package and you can call it whenever you want to produce a whole project (can do partials, but I'll focus on complete). Local generators often reside within the project, they are maintained in the same repo, they will often be able to render components.

Package

Once a new template has been created we want to be able to distribute, maintain and version the templates. This will likely be through the use of existing package managers such as NPM.

Let's do some stuff

I'm going to look at Projen, Hygen and Yeoman. The built-in CDK scaffolder can be used but isn't strictly a project templating or generator tool. When you run cdk init for a typescript CDK app project you'll get something like this:

Cdk init will be OK for plenty of people, use what works for you and your team.

There isn't any other customization here for CDK init, job done, this is why we won't be looking at it any more detail and moving onto the others.

Projen

I recommend visiting the github page for projen and flicking through the documentation as I won't do it justice. Projen aims to:

"Define and maintain complex project configuration through code", sounds great!

If you've been using CDK for a while you've probably heard of or used projen, it's getting lots of traction and is popular in the community.

Let's take a look at the basics:

Install

npx projen

New project

Creating a project, as in working project output to be used by users is simple enough:

npx projen new awscdk-app-ts

There are multiple project types not limited to CDK, check out the full list of project types.

How projen works

In the example above we've create a CDK app, not a custom construct. The way projen handles configuration of your project is through the .projenrc.js file located in the root of the project.

The projen way is to handle all configuration of your project through code, you will notice that all of the dot files are now read-only. Instead if you want to modify the package.json for example you will need to use the .projenrc.js to define your properties:

const { awscdk } = require("projen");
const project = new awscdk.AwsCdkTypeScriptApp({
  cdkVersion: "2.1.0",
  defaultReleaseBranch: "main",
  name: "projen-example",

  deps: ["prettier"], // simple example of adding dependency
});
project.synth();

Once you've updated the projen file you can re-run projen to apply those changes:

npx projen

The API reference for projen is extensive and there is a fair bit of chatter on the internet available to get you going. But, if you like idea of boilerplate projects and the projen ways I recommend checking it out. Even if you don't it's worth keeping any eye on the project to see what's coming or if you can help out.

Hygen

I stumbled across hygen while looking for solutions to project generators and was pleasantly surprised. Like the previous example, checkout the project page to get the full write up, here is what hygen is about:

"Hygen is the simple, fast, and scalable code generator that lives in your project."

What I like about this one is that everything in the quote about is so true once you start to unpack everything hygen has to offer.

Install

The quickstart guide on github has a bunch of different ways to hit the ground running, I'll show you using npm:

npm install -g hygen

New Project

I'm going to step through this at a high-level showing project local mode, this is what hygen is all about but know that it isn't limited to local and docs show you how to create a global generator. When in project local mode your template lives a long side your project.

hygen init self

You'll notice when you do this you get a bunch of files and folders you've never seen before, this is the hygen templating at work. Your core project would live at the top level and hygen would read the _templates folder (if present) for available options.

How Hygen works

As mentioned above, the default is to run in project local mode, the files added to your local project instruct hygen how to run. Out of the box you get generator documentation and prompting which provides a UI for users to step through. Hygen handles file copying and manipulation using frontmatter:

---

to: src/stack-<%=name%>.ts

---

import { App, Stack, StackProps } from 'aws-cdk-lib';

import { Construct } from 'constructs';

export class <%=name%> extends Stack
{ constructor(scope: Construct, id: string, props: StackProps = {})
{ super(scope, id, props);
// define resources here...
}}

You'll notice the --- delimiter in the header, this is frontmatter. The basic example above is showing that this file will be copied into the root source directory and a variable for name has been used. The variable placeholders are using ejs, which is a popular templating framework and quite powerful.

Now that you've added a source template in hygen you can commit your local generator with your project and teams can initialize the project at any time using:

hygen generator new --name talkncloud

This is a basic example, but you can start to see how you might create CDK templates and share them with your team. I recommend checking out the hygen website, it's an active project and the documentation is quite good.

Yeoman

We've reached the end and I saved yeoman for last for many reasons, but the main one being yeoman is like the og of generators, it's been around for donkeys, well before CDK came about at least. So, what does yeoman try to achieve:

Yeoman helps you to kickstart new projects, prescribing best practices and tools to help you stay productive.

The yeoman website does a better job at detailing this than I'll do, check it out when you get time.

Install

npm install -g yo

New project

Once you have install yeoman you can run the cli using the yo command. You'll be presented with a few options:

yeoman-example git:(main) ✗ yo

? 'Allo mike! What would you like to do? (Use arrow keys)

Run a generator

──────────────

❯ Install a generator

Find some help

Get me out of here!

──────────────

Yeoman has a public repository for community provided generators (templates), from the cli you can search those. We'll be making our own basic generator and to do that we are going to use the yeoman generator-generator package...

Source: https://github.com/yeoman/generator-generator (this always cracks me up, hahaha)

npm install -g yo generator-generator

The generator above has provided a boilerplate generator we can use to create our first generator. The dummy file included is part of the app generator, I've added the CDK example from the previous examples, hopefully you can see how the generators/app/templates folder is where you'd place your template files for the appgenerator. App might be a CDK API GW Lambda Route53 pattern that you'd like to share with your team.

Note: If you'd like to use typescript there is an example project here to get you started.

How it works

There is quite a bit to yeoman under the hood, check out the getting started guide for authoring generators. You may have noticed in the generator above there are several methods:

prompting();

writing();

install();

As you would have guessed it, yeoman also provides UI for prompting users for options that you can use in your template. These can then be used to conditionally do whatever you want or perform string replacement / transformation using ejs templating. Then you can write out your files and perform installation of package dependencies, what this does is once someone uses your generators it will run npm install (or whatever package manager).

You can start to see that there has been a fair amount of thought put into yeoman in how generators need to work which is why this is often a popular choice, you may also find more out of the box integration options with other services.

Supporting these projects

The projects mentioned today are all open-source projects, you can help out these projects by:

Project	Contribute Code	Contribute Cash
Projen	contributing
Hygen	contributing
Yeoman	contributing	opencollective

Summary

I'm sure if you're reading this you've probably used various scaffolders in the past and maybe thought it would be good to extend that for your own use. The generators discussed are a few of the popular ones and there are others, not all project generators are the same and some will work better for your team than others. I do know that using project generators will help your team and organization to produce consistent projects with all of the sensible defaults you want to provide which is a great place to be.

The code samples mentioned in this article can be found in the talkncloud github repository, while these aren't complete CDK projects the purpose is to show the outputs from the various generators:

talkncloud github repo - project templating

Credits: Article banner photo by Nik on Unsplash

Migrating from Ghost Pro to Hugo static on AWS for 24 cents

Mick Jacobsson — Fri, 31 Dec 2021 05:28:33 +0000

Confession, I've been using a managed service provider for my blog instead of self hosting on AWS...feels good to get that out. I've been using Ghost Pro to host this site since I started over 12 months ago. You see when it comes to writing or starting anything really I'm a fan of just writing, don't sweat the details, sit down and start typing. Which is why I didn't self host, the plan was to get something going first to get my writing fix, pause, review and then migrate later to something else if I wanted to.

Ghost Pro

It's better to write something than not at all. So when I started I did a quick scan of the market for the best "blog" platform that suited my needs and Ghost Pro does that. It has heaps of integrations, looks great and the web CMS frontend is easy to use which means I can write and publish from anywhere.

One of the really cool things that I liked about Ghost Pro was that it's open source, so if going with a managed service isn't for you then you can self host. More information is available on the ghost repo.

Straight from the repo...

"The easiest way to get a production instance deployed is with our official Ghost(Pro) managed service. It takes about 2 minutes to launch a new site with worldwide CDN, backups, security and maintenance all done for you..."

You can see why I went with Ghost Pro in the first place, if you're just starting out and want to take that hands off approach I highly recommend going with Ghost Pro. Great platform and helps the community:

Ghost Pro

Why switch?

To be honest, I have no problems switching back to Ghost Pro tomorrow or continuing to work with customers using Ghost Pro, it really comes down to your requirements, timelines and skills. If you have no time, go ghost, if you lack the skills right now, go ghost, if you have lots of common integrations go ghost. I actually tested out ghost pro with ecommerce and wrote about when I started this blog, my swag for sale on this site was easy as with ghost (reminds me, I need to add some cloud shirt designs, lacking right now).

If everything is so great with Ghost Pro than why switch? It's a good question. There are a few reasons, one of the main personal reasons is that I like to host my services. I write about these services, designs and share code I should be hosting them and I want to host them. One of the other key reasons is the underlying architecture design that suits my needs, my blogs doesn't change too much other than the written articles. I don't have loads of fancy integrations right now. I am a big fan of static sites because of the simplicity, fewer resources, security and low overheads. So my end goal was to always switch over to a static site of some sort in the future.

What is a static site?

Firstly, what is a static website and how does it differ from a traditional website? A static site is exactly that, it can be a simple index.html flat file that is read from the client directly perhaps from S3 and displayed. There is no hosted compute server. A traditional website like wordpress would consist of a compute resource like EC2 and a database like postgres maybe using RDS. You can start to see the differences and benefits and we haven't discussed, scale, security or cost.

Game Plan

I only really had a few basic requirements to get started:

Some sort of migration from ghost to the new framework
Static site framework that supports markdown
Good page load speeds
Theme or templating, preferred free themes or decent marketplace
Something basic and automated that allows me to publish new content

When you start to break it down, this is a pretty tall order. When I was using Ghost Pro I'd simply go to the web admin interface, type away and publish content. I didn't need to think about any of this.

The Stack

I started off by researching what others had done, I had made a bunch of assumptions based on what I know on how the site was going to be hosted but I was unsure on the static website framework.

I ended up selecting hugo for the website framework because it's popular and it boasts about speed, who doesn't like a fast loading website? I went through a few different tests on my local setup to make sure it was a good fit. My only complaint so far is that I find the directory structure and templating syntax a little confusing compared to others I've used in the past.

Here is the full stack list:

Hugo static website framework
AWS S3
AWS CloudFront
AWS Route53
CDK
Github
Forestry.io

The Design

It's much easier to understand with a picture...

The design shows that readers will access the domain via the registration and records in Route53, this will direct them to our alias record CloudFront CDN and we'll add some certificates for security and so that visitors get that warm fuzzy feeling of security with a nice padlock in the browser and no errors. I'm using Lambda@Edge (cool stuff) here to fix up some of the path handling for the static site (more on this later) and then finally we hit the bucket with our static files.

There are a few other considerations here, like cloudfront OAI (origin access identity), we can force our users down the cloudfront path and remove the need for our bucket to be public which is great.

Pretty cool, how do the files get updated though?

Maintenance workflow

Most modern web applications need to be built now, by that I mean there is a build stage in the process. The site is often developed using a framework that speeds up development and code needs to be processed or built into an output that the browser understands e.g. our static files, like an index.html.

I don't want to be building on my local development setup and then dragging some files over to S3 and then potentially invalidating the cloudfront CDN so that my changes are live for viewers.

So, here is the workflow I've come up with for now:

So we can see from the workflow design above when admins (authors) make changes to the repo it will kick off the workflow. I've designed this one so that it only happens on a specific branch so that we don't waste unnecessary build time or deploy unwanted changes, so to start the workflow you'll need to raise a PR.

I've developed my own workflow for this one which basically installs and configures hugo, then runs the build, some basic testing and then finally the deploy. I'm using hugo's built-in deployment feature which works well with AWS S3, here is an example from the config file from hugo:

[[deployment.targets]]
# An arbitrary name for this target.
name = "mydeployment"
# The Go Cloud Development Kit URL to deploy to. Examples:
# GCS; see https://gocloud.dev/howto/blob/#gcs
# URL = "gs://<Bucket Name>"

# S3; see https://gocloud.dev/howto/blob/#s3
# For S3-compatible endpoints, see https://gocloud.dev/howto/blob/#s3-compatible
# URL = "s3://<Bucket Name>?region=<AWS region>"

# Azure Blob Storage; see https://gocloud.dev/howto/blob/#azure
# URL = "azblob://$web"

# You can use a "prefix=" query parameter to target a subfolder of the bucket:
# URL = "gs://<Bucket Name>?prefix=a/subfolder/"

# If you are using a CloudFront CDN, deploy will invalidate the cache as needed.
cloudFrontDistributionID = <ID>

You can see that you'll supply your bucket and cloudfront CDN, you can also have multiple environments e.g. prod, non-prod (targets). Simply running the hugo deploy will use this config with your AWS cli access to deploy and invalidate the CDN. Pretty neat.

So, now from here, all we need to do is make our changes in a normal source code repo, do a pull request and our workflow will take care of the rest. Once the actions have been completed the static website changes will be live. Easy peesy, fairly well automated.

Client-side development

As I commented earlier, I write mostly in markdown, it works for me. Actually, when I was using Ghost Pro I would mostly use the markdown editor for all my posts. But, now how do we make changes from the client, what does it look like?

The image above is a dummy markdown file I've written. I'm using Visual Studio Code and I've simply created a new plain text md file. Markdown has its own syntax which helps you style the document. I'll leave you to research that, it's straight forward and easy to hang of.

For the majority of the content, that is the words this will be fine, it gets a little harder when you need more complex styling. I've decided to trial forestry.io for a couple of reasons:

Would I miss the web based editor from Ghost and can this fill the gap?
Future projects and working with clients

Forestry.io basically aims to fill the gap as a CMS for static site generators, this isn't just for hugo. It hooks into your code repository and puts a web CMS frontend around it so that you can manage your site.

You can see here you get a simple WYSIWYG editor so that if you're not familiar with markdown you can jump straight. Even if you are familiar it can be time saving it other ways.

I might do a more detailed write up on forestry.io, I'm still trying it out, I have found some issues that I've overcome. It's not all it's cracked up to be and I don't know if I'll stick with it. I like the general idea and where it's headed though and it's certainly something I'll be keeping an eye on.

Lambda@edge

Argh, this one kinda of grinds my gears a little. I like Lambda@edge don't get me wrong, I think it's quite powerful but it would have been great if AWS handled this out of the box knowing that this is for static site hosting.

Anyway, so, if you have a standard website serving a very flat static structure of one level e.g. /index.html, /contact.html then everything is fine but if you have sub directories like /blog/awesome-sauce/index.html then it doesn't work. To work around this you need to handle the path matching with something like Lambda so that it can access the files correctly.

Thankfully AWS have done a pretty great write up on the problem with some example code that does work. Have a read, grab the code, change it as needed (if at all), problem solved, moving on...

Migrating the data

One of my key requirements was to be able to migrate the data from Ghost to Hugo, there isn't much point migrating to a static design if you have to rewrite all of the posts.

Ghost provides an export function from within the admin panel which outputs a json dump of the content without the images. To get the images you just need to contact support and they'll provide a link with your site configuration so that you can download all of the content.

The Hugo open source community has provided a tool to migrate from ghost to hugo, you simply use the cli tool to point it at your ghost export and it will convert it for you.

ghostToHugo is the converter, it didn't work for me, I found that if you had a bookmark with a nil description it would cause the tool to fail. I've fixed the bug and committed it back to the repo so it works if you have the same problem as I did. You can check out my commit here.

The output of the data in Hugo format was mostly OK, I had some issues with some of the images that needed fixing but this was easy done with a bulk find and replace.

Styling hugo

One of the things I liked about Hugo was that there were more open source created themes than others (to me anyway). This is great and helps when you're starting out to see what others have done with practical examples.

I tried a bunch of different themes, I didn't want to get to far into customization if I didn't have to but in the end I did. I landed on geek-hugo by themefisher it is open source and looked like a good starting point for me.

The majority of my time went into customizing the theme. The main changes I made were to convert it do dark mode, widen the formatting and add in some custom styling changes that I liked, human readable times etc, bump some of the margins around. It was straight forward enough to update. If you are starting out using this theme let me know and I can help out.

Performance

Using Google Lighthouse you can run tests on your test which give you a wealth of information. If we look at general scores we can see that the site is hitting high 80's across the board which isn't bad at all, always room for improvement.

Costs

OK, so this will probably be a big draw card for many. When I was using Ghost Pro I was paying $36 USD per month, this was for the fully managed service that gave me full customization.

AWS Cost estimates for this stack:

Estimates

Service	Comments	Cost (USD)
S3	1GB	$0.03
Cloudfront	1GB, 100k reqs	$0.21
Lambda	100k reqs, 128MB, 3 sec	$0.00

Total: $0.24 USD per month

I've included some Cloudfront costs to cover readers that might hit the site from non-edge locations but just know that with Cloudfront you get 1TB and 10M requests in the free tier.

Note: None of these included domain registration. If you want to bring a custom domain remember to throw another $10-12 bucks per year on your final figures. Forestry.io and github are both free for this use case.

Per my previous comments, the money for Ghost helps the open source community so this isn't a direct comparison per say. It's fine to spend a little extra for good but you can use these numbers to compare with closed source enterprise offerings.

Final thoughts

This pattern for static hosting is pretty standard on AWS, using this pattern reduces a lot of the maintenance and scaling considerations thanks to AWS and static frameworks. If you're looking for one of the lowest cost solutions this would certainly be up there, you can also host something like this on Github pages for free which is another option. I enjoy having control and flexibility over the architecture and once it's in AWS I can continue to expand into other services as needed. Ghost Pro is one of the best options if you want to be completely hands off and need a more end to end solution, a perfect option for non technical people and helps support the community.

Have a think about your current website or customer websites and see if static fits the needs, it is most definitely the right way to go if you can swing it.

I haven't provided a CDK project this time around, I'll spend a few minutes to get something out of the next day or two that I can share that should help people get going with AWS static hosting and hugo.

AWS Lambda Graviton2, cheap and slow

Mick Jacobsson — Sat, 16 Oct 2021 09:57:11 +0000

I was pretty pumped when I saw the announcement for AWS Lambda Graviton2 support for a couple of reasons, performance and cost. AWS have provided Graviton2 support for EC2 for a while now so it's good to see this flow into Lambda especially if it's going to save us a buck and it's faster, whats not to like?

The official AWS blog has done a pretty good write up detailing how to use the new architecture type, strategies for migrating by running both your x86 and arm functions and they've used an open-source project to optimize the workload. Probably the most thorough write up we've seen for a while from AWS, which is great!

Just a side note, I'll be talking about performance, cost and usability, there are other benefits to running Graviton2. You can read about some of the other benefits on the AWS Graviton2 page.

Video: If you want to out check devs in the shed (shout out to the local AWS Devs in the shed, Paul Kukiel and Matt Coles), this is first info I saw about Lambda Graviton2 that popped up in my feed: https://www.youtube.com/watch?v=adAvn_NxLU8

The Test Environment

I had a bit of a look around before tackling this one and found some previous writeups which used some well known tools. Shout out to Tai Nguygen Bui, I found his write up to be quite detailed and dips into API GW performance as well. In the end I selected https://artillery.io/ and a simple AWS API GW setup...and of course, everything is developed using CDK.

All of the testing has been completed using ap-southeast-2 (Sydney) region, any pricing is in USD from us-east-1 to make it easier.

We can see from the image above the setup is simple, we have a client using the artillery.io toolkit which is from an external network outside of AWS, this requests the API-GW using the various paths which then invoke the Lambda function and return the results.

The Lambda functions have been developed using Python and each function is the same as far as code. I added containers because I was curious about cold starts with arm and this would be the best way to find out.

Note: for the containers I updated the boto client as seen in the Dockerfile, this is the same on x86 and arm

The Code and configuration

If you'd like to play along you can replicate the environment by provisioning the stack with CDK in your account:

https://github.com/talkncloud/aws/tree/main/lambda-graviton2

If you want to run the artillery.io tests you'll need to copy the perf.sample.yaml over to perf.yaml. You will also need to update the x-api-key with your API key and obviously the endpoint url as well.

You can use environment variables you prefer, see the commented out section below. Also note the region.

Sample perf.yaml

config:
  target: "https://TALKNCLOUD.execute-api.ap-southeast-2.amazonaws.com/kaboom"
  phases:
    - duration: 900
      arrivalRate: 30
  defaults:
    headers:
      # x-api-key: "{{ $processEnvironment.SERVICE_API_KEY }}"
      x-api-key: "SOMEKEYGENERATED"
scenarios:
  - flow:
      - get:
          url: "/x86"
      - get:
          url: "/arm"
      - get:
          url: "/x86container"
      - get:
          url: "/armcontainer"

What this will do is throw 27,000 requests at the API-GW which will in turn invoke each one of our lambda functions over a 15 minute window. It's a pretty nice little tool, the end result will show if we had any errors like non HTTP 200's.

The actual performance test itself is simple enough, the AWS blog digs into prime numbers. I wanted something different. I didn't find anything simple enough for what I wanted, but I did see a nice little snippet that runs some simple calcuations. I've added some some string concatenation to it and picked an end for the loop. It's simple enough to produce a bit of load for us to talk about.

listAppend = []
for item in range(10):
    someString = "i like"
    someString = someString + " pie"
    listAppend.append(someString)

    # number calculation
    # Source / Credit: https://github.com/alexdedyura/cpu-benchmark
    for x in range(1,1000):
      3.141592 * 2**x
    for x in range(1,10000):
      float(x) / 3.141592
    for x in range(1,10000):
      float(3.141592) / x

There are a few other bits and pieces that return a message in the payload but again, less important, feel free to dig into that if you like.

What did I find?

When I first developed this testing I was only looking at a simple hello world using the x86 and arm. As you can imagine the response would be so quick it wouldn't be that interesting. I won't be discussing anything outside of Lambda e.g. the time from the client start to finish or the API GW performance. I'm only interested in how long lambda took and what does that mean for billing.

First of the summary results from artillery

Test table summary (after cold start)

Item	Result
Scenarios launched	27,000
Scenarios completed	27,000
Requests completed	108,000
Mean response/sec	119.61
Response time (msec) min	158
Response time (msec) max	4551
Response time (msec) median	564
Response time (msec) p95	631
Response time (msec) p99	691
Codes (HTTP) 200	108,000

As above, this is after the cold start. What I mean to say is I split the testing to make it easier for me to look at warm invokes only e.g. no initialization. The table above is showing that we made over 100k successful requests.

CloudWatch function comparison

The chart below is showing the function duration for all functions, this is the 15 minute of testing with 27,000 hits per function.

Note: I performed a cold start first to split the charts out, which I'll show next.

Here is the cold start, this is me basically running the test and then canceling it. The Lambda function hadn't been invoked for days prior to these results.

For both of these charts I've set the statistic to Max and Period to 1 Second. The average of the same period less the cold start shows another useful story:

Whats interesting about these charts?

First thing, the obvious was the performance, I'm just not seeing the performance I was expecting given all the hype of Gravtion2 support for Lambda. The second was the cold start, wowzers, would you look at that arm container cold-start, it's a good 7 seconds slower than x86.

I thought maybe the arm container was larger in size compared to x86, this might help explain something. The arm container is larger, 1MB. Not enough to explain it.

X-ray, digging a little deeper

I'm a big fan of AWS x-ray, it's so simple to enable and provides some nice easy to use visibility of our setup. I took a closer look at the arm container startup using x-ray to confirm:

I grabbed this by sorting by response time in x-ray and selecting the highest one. We can see the initialization time of 8.7 seconds.

For comparison, here is the next highest x86 container:

Longer cold starts aren't the end the world, it was just something interesting I found along the way and unexpected. If we take a look at what the arm container is doing once its warmed up, it ticks along as per normal:

You can see the arm container trace, warmed up, not requiring initialization. Good stuff.

Duration stats table

Using CloudWatch insights I was able to query the function data to show the duration in milliseconds for each function, billed versus actual. Again, this shouldn't be too surprising after seeing the cloudwatch charts above.

The surprising numbers being that x86 was quicker all round, not surprising after seeing the cloudwatch charts.

Function	BilledDurationMS (Total)	DurationInMS (Total)
ArmContainer	15216876	15180466.08
x86Container	13304215	13234641.51
Arm	14985048	14971569.68
x86	12938295	12924734.27

Switching to Graviton2 with CDK

I wanted to talk briefly on how you go about implementing Graviton2 with CDK, the AWS blog has a better write up if you want to use the console but I'll touch on CDK only.

Note: You'll need to be on the latest version of CDK to do this.

When you develop a function in CDK there is a new property for functions which is architecture, if you don't specify it defaults to x86, cool cool, no problems. But, if you want to choose an arch you are presented with an array, why would you need an array of architecture????

// Lambda arm (graviton2)
const lambdaArm = new lambda.Function(this, "lambda-arm", {
  functionName: "talkncloud-lambda-arm",
  description: "performance testing lambda to compare x86 to arm",
  runtime: lambda.Runtime.PYTHON_3_8,
  architectures: [lambda.Architecture.ARM_64],
  code: lambda.Code.fromAsset("./src/lambda/perf"),
  handler: "lambda.handler",
  tracing: lambda.Tracing.ACTIVE,
});

It's not entirely clear, the CDK team have already updated the documentation https://docs.aws.amazon.com/cdk/api/latest/docs/aws-lambda-readme.html#architecture which shows why this might be needed for layers but not really the core function. Once more, if you do this manually in the AWS console you are presented with a radio button, x86 or arm not both.

Note: this is as of CDK version 1.125.0

If you're interested in reading the CDK github changelog, others are also asking this question. I suspect it will be fixed in a future release:

https://github.com/aws/aws-cdk/commit/b3ba35e9b8b157303a29350031885eff0c73b05b

Other than updating CDK and specifying the architecture thats all thats needed to deploy an arm/Graviton2 lambda function using CDK.

Containers and multi-arch

One thing that I didn't really think of when I started going down the arm path was building containers without having an arm processor. I'm using cloud9 for some of my container work and it doesn't run on Graviton2, when you provision using CDK it abstracts alot of the heavy lifting for you and it failed during the container build for arm due to the arch mismatch.

This just wasn't something that I thought of initially, it's very obvious now, but just something to think about if you need to use containers and Lambda, you'll need to eat some engineering time to get it running.

Docker buidx seems to be the answer, so at the end of the day the tools behind the scenes are changing which should make it easier. It runs an emulator so that you can build the container for different supported architectures. I followed this AWS blog to setup my environment. Just scroll down to "Creating a multi-arch image builder" and you should be good to go.

Now, if you run provision the CDK stack with containers it will build successfully.

Wrapping up

One thing that I haven't discussed is that Lambda with Graviton2 is cheaper, it's actually priced differently.

us-east-1 128MB price per 1ms:

Arch	Price (USD)
x86	$0.0000000021
arm	$0.0000000017

When I read the initial information coming from AWS and checked out the blogs, I was under the impression that it was cheaper because it was faster. Faster meant less milliseconds duration and less billing. But, no, as you can see from the AWS pricing its just cheaper in general.

I was expecting this to be more clear cut than it was, I wanted Graviton2 to be the clear winner hands down but it just wasn't the case. When you factor in a little more dev time in multi-arch containers (if needed or keeping options open) and potentially having issues with library support there is a little more to consider than what you might think.

This is just a simple case of your mileage may vary, if you're hitting the free tier and other features of Graviton2 don't float your boat you'll probably do nothing. If you're paying serious dollars for Lambda right now then I'd be doing side by side comparison to make sure you get what you expect, every app is different.

I'm open to suggestions and feedback on the testing, this didn't touch on an scaling or other scenarios so there is a bit more that could be done. Any other tests you can think of? If you've switched to Graviton2 with Lambda, how has it been?

Core 2 M5 IoT Stack AWS EduKit Unboxing

Mick Jacobsson — Thu, 22 Jul 2021 15:11:34 +0000

I have no idea how I missed this one, I only just caught this on my network someone (can't recall who) was promoting the last days to register for the hackster healthy spaces challenge.

I didn't end up registering for the free hardware but I went ahead and purchased one and registered for the event. I'm going to run through unboxing the Core 2 M5 Stack and first impressions.
Where cloud?

OK, you're probably wondering where the cloud part comes in and if you know me, more specially where does the AWS part come in. Well, this is the M5Stack Core2 ESP32 IoT Development Education Kit for AWS which as you've probably gathered is geared towards using the device with AWS IoT, it's also branded with AWS logos.
Why this kit?

Apart from just being keen on AWS in general, this device looked pretty cool on the surface. In the past I've dabbled in various micro controllers and typically what I've found has been more challenging is once you start adding a bunch of sensors the bread board gets pretty busy and it's usually not so plug n play. If you want battery than that will be another challenge. This device has a bunch of built in sensors (and LiPO) built-in that you can start using with little effort, helps for prototyping quickly.

Unboxing

The box itself is cool, branded in the M5 stack logo on the cardboard and tape. I like this subtle detail. The overall weight of the unit is very light, the approximate weight is around 100 grams.

Upon opening the box you'll be greeted with a pretty decent chunk of bubble wrap. The actual contents of the box are pretty small. I should note that I purchased an additional cable that I wanted and an additional sensor. I'd be curious to hear if the box is smaller if you just get the Core 2 on its own?

The actual Core 2 packaging reminds me of a matchbox, sliding the contents outwards. I really liked the AWS branding on the packaging but in general don't expect Apple quality packaging. There is a sticker that needs to be removed before you can slide it out which is a little tricky or you'll need a knife or something.

As you can see from the size, it's quite a small footprint, it easily fits in the palm of my hand.

Whats in the box

Firstly, there are no instructions in the box, there is no QR code or anything on the box guiding you to what you're supposed to do next. The box is limited to the following items:

1. Core 2 M5 Stack device
2. USB-C to USB-A cable
3. Allen key / hex wrench

Naturally, by including a tool I'm going to open this thing...

What's inside the box inside the Core 2

Once you remove the four hex screws you get a better look inside the Core 2, you'll notice the battery and the small ESP32 board. Separating the cover is a little tricky, you kind of need to pry it open and be careful not to bend the pins but it's not to bad.

Tip: Take note of the different length screws, it's a tad annoying that they are different lengths.

The specs

It's probably easy to just show you the diagram from the supplier as it lists out the specs pretty nicely.

Here are some key take aways:

* ESP32 16MB
* USB-C interface for connection, power etc
* Capacitive colour touch screen
* Microphone
* Speaker
* Vibration motor (kind of like your smart phone when it vibrates)
* RGB lights either side of the unit
* Led light
* Micro SD card slot
* Battery - 500mAh lithium
* 2.4Ghz Wifi

The M5 store has the most details if you want to dig further into it
https://shop.m5stack.com/collections/m5-core/products/m5stack-core2-esp32-iot-development-kit-for-aws-iot-edukit

The construction of the casing is pretty impressive, there are no sharp corners or manufacturing defects on my unit. The screen is protected by a plastic film that you can peel back and dispose of. There is a nice subtle M5 imprinted on the side of the unit which is nice detail. Apart from the unit being AWS orange there is no AWS logo or wording on the case itself.

Bonus component

I did purchase an additional sensor, a microphone. I was curious to see what the quality of additional sensors was like and for my hackster project I need to use a microphone. I was unsure what the quality of the built-in mic would be so it was a just in case purchase (deadlines)

The mic sensor is pretty sweet. Again, the construction of the casing for the sensor is great and the cable is plugs in snugly. The sensor comes in its only little case. Which reminds me of gameboy game cases for some reason, ha!

Boot up

There is a small power button that you press to power up the beast, it only takes a moment, you've got to be quick if you want to snap a pic of the AWS logo but it's glorious!

The lack of instructions becomes less of an issue as the touch screen welcomes you and has a bit of demo of the various sensors included with the device.

The welcome program will have you swipe through and interact with the device to activate the vibration sensor, use the microphone, toggle controls and others like the gyro. Don't expect the same touch responsiveness as your phone, I think it's pretty good considering, I did find that when I loaded one of demo firmwares the controls weren't as good as the demo loaded from factory. I'm guessing there is an activate area or something that needs to be programmed. I haven't gotten that far.

The final screen is a url to the AWS EduKit website to begin going through the "hello-world" samples.

Ignore the help

So, I did the obvious which was ignore the instructions, how hard could it be? I continued to fumble my way though thinking there must be a way to connect this to the wifi or there must be a mobile app. Again, none of this was obvious.

Don't do what I did, read the instructions from AWS.

Mac OS Cable Connection

If you're running Apple devices like a macbook you might run into trouble with the USB-C connection. When you plug the M5 in it loads a driver and creates a serial connection which you use to interface with. Connecting through my dock / hub didn't work, I had to connect to directly. Booo...

If you're running a macbook usb-c only, make sure you have a cable e.g. USB-C to USB-C (not included). I can confirm the genuine Apple USB-C display, USB adapter worked if you have one of those.

A tale of two buttons

One thing I found kinda of funny was the power and reset buttons, there are two buttons. For some reason I thought there was only one button but you held it down to reset the device. This is the most common type of button I've seen on IoT devices. The M5 has two separate buttons, one for power and one for reset, they are identical but on different sides. Subtle.

Initial connections, IDE

Good news! The M5 uses PlatformIO which integrates with Visual Studio Code, so you get that familiar IDE. I did find the PlatformIO extension to be pretty chunky and generally not very intuitive though. In my opinion it's the separate menu systems inside the extension which add to the confusion for me, it doesn't take long to work it out, so this is minor.

Once you've connected the device and confirmed you actual have a serial connection available, loading firmware is pretty easy. I went ahead and loaded the getting started firmware, this firmware has a very basic spinning fan on the display that you can control.

To toggle the controls remotely you can use the cli or you can use the mobile app. Follow the AWS EduKit guide to find the app.

Tip: Restart the device once you've loaded the firmware and watch the terminal output on your computer. The QR code needed for claiming and registration is in the terminal output.

Recap key tips

* If you're running Apple Mac or USB-C only host, buy a USB-C <> USB-C cable


Restart the M5 once flashed, monitor the output for the QR code for claiming
Read the manual (shocker)
Note the screws if you take it apart as they are different lengths

Final Thoughts

As an owner of multiple generations of raspberry pi's, various hats, cases, feathers and other mc's i'm impressed by the build quality most. The case construction, fitting and detail that has gone into it is really impressive for the price (~$49.90 USD). Shipping was pretty quick at around 5-7 days to Australia.

I thought the on-boarding wasn't as smooth as other IoT devices that I've used in the past, which would be scan a QR code, download app, register and toggle various buttons through the app. There are a few more steps to jump through to get going.

The platformIO IDE looks pretty powerful, bit of a learning curve there, I would have liked a more basic interface. One thing to note is that the platformIO IDE caters to many different types of devices which is probably why its feature rich.

In summary, I'm happy with the purchase so far, I'm looking forward to having a play with the Core 2 M5 and AWS IoT. Hopefully I'll have some more info to share soon!

AWS SCS-C01 Security Specialty Certification Recap Jun 2021

Mick Jacobsson — Mon, 14 Jun 2021 11:18:31 +0000

I am fresh off the AWS Security Specialty exam for 2021 having successfully sat for the exam last week. What a doozy! I've 4 other AWS certifications and for me, personally I found this one to be the most challenging. I do practice security in my day to day to ensure I am building secure systems and enjoy discussing designs with security professionals but I'm not in a dedicated security team or SysOps role so I'm not living and breathing security everyday.

In this post I'll give you a run down on the exam, what you can expect on the day and what materials I used to prepare for the exam.

What is the Security Specialty Exam?

As usual the official AWS doco gives you the best info on the exam. It's basically an exam for security professionals (shocker), so, think security in transit, rest, shared responsibility model, users, design, access to resources etc.

AWS recommend 2 years hands on experience securing workloads in AWS

As above, I'm not in a day to day security role but treat security very seriously and I have more than the 2 years experience. It's a recommendation not a requirement, if you're ready you're ready, don't let that stop you.
Exam Booking

Covid has introduced a little more flexibility with exam testing and now you sit the exam in the comfort of your own home. All you need is a computer, internet, camera and microphone. Laptops are good for this. I went with pearson vue, this is the same provider I'd be using if I had to go into the physical testing office.

I already had an account as I've used them before. One thing I noticed was that there were so many time slots, you could basically register and sit the exam the same day from I could see and there slots every hour or less. This is really cool and incredibly flexible. This means I can book the exam around my work and personal life (you can sit an exam around mid-night if you want to).

Even though I've done these types of exams before, I complete the system test before the exam every time. It's just a simulation of the exam software before the actual test so that you know you're good to go.

Exam day remotely is no different from when you had to go into the physical testing office. It's the same application, same format, same flow, same flag, review etc.

The Exam (the bit you probably came for)

OK, let's get down to it, this is likely why you are here. I'll try to give you as much information as possible hopefully without overstepping. I'll break it up into sections as best I can. Obviously on the day your mileage may vary so prep as much as you can or think you need and go for it.

Security Services

This one seems obvious. Know the AWS services that are related to security, know the names and what they do. This is useful for any AWS exam, I find that many of the questions will rely on you knowing how the services fit together so understanding what they are is a good step in the right direction.

Tip: Get to know the architecture diagram symbols, create some designs. They don't have to be perfect.

Updating workloads

When you provision servers in AWS remember to think of the shared security model. What part are you responsible for in the model? If deploy a virtual server how might you keep it updated using AWS and how might you check that it is updated? When we talk about updating servers we need to think of scale an automation, you don't want to be manually patching 100 servers. Hopefully that gives you a hint.

Tip: There are great services available in AWS that inspect your systems and provide reports. These types of services are valuable in many situations.

Post incident response

You're going about your business and then all of sudden there has been a breach. A security incident. What would you do?

Where are the logs?
How can you isolate it?
What services may have detected the incident?
Did someone use access keys? How can you check, can you disable the keys and how?
How can you check the state policy before and after?

Encryption at rest and transit

This where knowing the services comes in handy, you might get asked about protecting data in transit but it's easy to fall into a trap if you don't how certain services can provide this.

Buckets
- Different types of encryption
Load balancers
- Different types of load balancers

Tip: Speaking of load balancers, consider high availability design, multiple EC2 servers etc

Network Design

As per the AWS best practice you are most likely already familiar with how to design secure workloads. You should be familiar with how to design private and public subnets, what might go into those and why.

You should be familiar with VPC's and when to use direct connect and/or VPN and why. Understand that there are other networking services as well like transit gateway and VPC peering.

If you have a bunch of networking floating around your AWS accounts how might you get a better a look at that. Can you turn on additional logging at the VPC level. If you turn on additional logging, where does it go? Can you search it, can you visualize it, can you customize it?

Security groups and Network Access Control Lists (NACL) will more than likely come up. You can deploy an EC2 without a security group and when you think about networks NACLS are always there. Know the difference between the two and how ordering might work.

Security Design

I wasn't expecting too much of this, well any to be honest. But, as I'm a serverless guy it kinda worked out well for me. Think about traditional architecture design like a 2 tier web application hosted on virtual servers in AWS. What would that look like if its a static web site hosted in s3 and what key components can you think of? What would you need to do to secure it, to provide access, to encrypt the communications.

Security automation and notification

Again, understanding the different services goes along way here. There is a key difference between being notified of a security event and automation. If you want to be notified via email or similar in AWS what would you use? It's more than likely going to be SNS. But, if you think of what services have information related to security can they be integrated into SNS, what if you want to act on that information what else could you use?

Logging

Knowing how to enable logging in AWS is a big part of security. How do you know whats happening in your account if its not logged? How do you audit access and activity.

You should be pretty familiar with CloudWatch and CloudTrail, how are they different? What information can you search in each, can you export it? Can you archive it?

Tip: There are other services that can be used to stream data into AWS and then visualized.
Tip: AWS Organizations plays are part here, how does logging look like with many accounts.

Users

Hopefully you're not one of those people who create an AWS account and use the root account for everything. If you are, stop. Why is that a bad idea and how might you protect the root account?

What are the different concepts inside IAM, what is a role or a group and how do they differ.

If you're an enterprise customer how might they want to provide access to AWS, they probably have Active Directory. Can they connect that somehow to IAM?

When it comes to policies get to know the basics, what are IAM policies, study them, make them, break them. Know the difference between IAM policies and bucket policies.

AWS knows all your secrets or do they?

In AWS there are heaps of different ways to store secrets if you really think about it. Hey, if you're Corey Quinn you might even store them in Route 53 as a cheap database (joke). But, in reality there are probably like two services in AWS where you can store secrets. Thats going to be Secret Manager and System Manager Parameter Store.

Just get to know them at a high level, one has a bit more functionality when it comes to secret storage. Honestly, I have used Secret Manager to store some creds before, I know Parameter Store can but I've just used Secret Manager as its purpose built and provides more management. It was news to me that people were storing secrets in Parameter Store other than just well parameters. But, anyway. It is cheaper, so theres that?

What did I use to prep

Everyone's study style is different, I myself am a hands on kinda of learner, I need to get in and build it, break in to understand it. So I like to use practice exams with decent explanations and also pair that with the console to get a better understanding if I need to.

I use Jon Bonso's Tutorial Dojo, I've recommended him before (no affiliation):

https://portal.tutorialsdojo.com/courses/aws-certified-security-specialty-practice-exams/

I will say that this particular one wasn't as good as the others I've studied with Jon. I usually like this resource as it provides a bunch of different practice exams in varying modes, review and timed. The responses are usually filled with helpful responses and links to AWS materials so that you can work out where you went wrong or find or more.

However, the security responses and the questions just seemed bit lacking for some of the questions. I also found that it was very repetitive, there were many KMS questions. Maybe previous exams have had an overload of KMS related questions or something.

Never the less, this is the sole resource I used.

My Tips

Write down the services and come up with your own explanation for them
Create architecture diagrams, draw.io is good
If you don't have access to practice exams, write your own questions
Get your hands dirty, you 100% need to use these services, get into the console and start building
Get some sleep, if it's the night before the exam don't stay up cramming.

Try not to sweat it, if at the end of the day you don't pass you can always retake the exam. This sucks because it's expensive and you've put this time in, but it's not for nothing you have just had first hand experience on the exam and now you can use that for the next time.

How do you learn?

I am interested in hearing from others about how they learn, what resources they use and what works for them. I am researching a project to help out in this space, so reach out if you can.

Happy days

Hopefully this guide has been useful for somebody out there studying for the exam. This is one of the more difficult exams, I found that the variety of the questions and the number of services in the questions to be generally challenging.

If you've studied security before or are familiar with the practices around security design, encryption, policy etc this will help to go along way but you'll still need to understand how AWS tackle it and what services are used.

AWS She Builds Day June 2021 - Loud and Clear

Mick Jacobsson — Thu, 10 Jun 2021 11:37:21 +0000

Every now and then I'd see or hear about AWS She Builds, there was a cool logo, bright colours and some new faces. Enough was enough it was time to find out once and for all what AWS She Builds was all about.

Full disclosure this is my experience as a male attending AWS She Builds Day June 2021 Virtual event.

I'll go into more details later, but if you're a woman there are a couple of stand out programs you should be aware of right now!

AWS She Builds Mentor Program

A 12 week mentor program that is open for applications, spots are limited and applications close June 30th 2021.

https://awsshebuildsmentoringprogram.splashthat.com/

AWS She Builds CloudU

CloudU is an AWS training and certification program with the aim of becoming a certified AWS Cloud Practitioner during the 8 week program. It looks like you have until the 28th of June 2021 to register (or its starts the 28th?).

Both of these programs will help to connect you with like minded women in the tech industry, you don't have to be an expert or experienced. You don't need to be a software engineer or developer. Take a look at the links above and register if it looks like something you might be into. If you're on the fence about it, do it anyway!

https://awscloudushebuildcpeapj.splashthat.com/

What is AWS She Builds Day?

You might have guessed already but, AWS She Builds Day is an event for women in tech to come together and discuss tech, network, leadership and more. This is an inclusive event, everyone is welcome to attend (I didn't sneak in). Expect to hear from a wide range female leaders from AWS and other companies around the globe with discussions varying from non-technical (leadership, career) to technical (yay building stuff!).

AWS She Builds Day 2021

I have no idea if these events are always virtual but this year She Builds was virtual, hosted out of Sydney Australia (go Australia!). Although the event this year was hosted in Australia the event features women from around the world. The virtual event was streamed live on Twitch via the AWS channel and included some engaging chat with prizes!!!. I've only used twitch a few times now, I signed up for another AWS event that was on twitch and I've gotta say I really like it like for these events. It works well and it's easy to engage with others.

https://www.twitch.tv/aws

Tip: If you are going to attend an AWS event with twitch I recommend registering for an twitch.tv account. This way you can participate in the conversation. You don't have to, you can view you just won't be able to ask questions etc.

Here is a link to the agenda this year to give you a better idea of the line up: https://aws-shebuilds-day-june.splashthat.com/

Highlights

Here are my highlights from the event this year, this isn't an exhaustive list, this is what I found stood out from my point of view. Be sure to watch the replay so you can find your own highlights from the event.

Equality in tech through salary transparency

Kesha Williams created Salary Overflow developed on AWS, "think of it as Stack Overflow for Salaries". Kesha created Salary Overflow to help bring transparency to tech salaries, the application is free to register. You can search for salaries by role name, location and drill further into the details. Kesha walks us through the goal of Salary Overflow, the architecture design and quick demo of the interface.

https://www.salaryoverflow.com/

I found Kesha quite inspiring as a builder, the presentation was clear and the q&a was great. It's clear that Kesha enjoys building but also really enjoys sharing that knowledge and encourages others "share what they learn".

Program Updates

As I mentioned above, there were several program updates. It's important to act fast on all of the programs as they appear to have end dates and some have limited spots. There are some great support programs on offer here, I strongly encourage everyone to apply. The mentor program will connect you with women at AWS and honestly, everyone at AWS that I've met has been great to work with so you won't be disappointed.

https://awsshebuildsmentoringprogram.splashthat.com/
https://awscloudushebuildcpeapj.splashthat.com/
https://www.thedreamcollective.com.au/programs/she-dares/

Technical Presentations

Anita Miller gives us an overview of Apache Kafka, the key components and why you might want to use it. Anita also explains how you can reduce administrative overhead by using AWS MSK and some of the benefits in doing so. I haven't spent much time looking into Kafka but found that Anita's overview was easy to understand and also provided practical examples of why you might want to use Kafka, MSK or Kinesis.

Ekaterina Gasparian (Kate, I wrote down Kate?) from Canva is using machine learning to help drive a better user experience. I am a big fan of Canva and found the use cases very interesting. Kate goes into details about some of the challenges about presenting content to users that is accurate and meaningful using AWS Personalize. I can see how this would reduce the amount of time it takes a user to create content but also helps to surface content the user is interested in. Some of this is aided by machine learning on AWS and more complex cases using Sage Maker. I'm butchering this, watch the stream and you'll get a better understanding of what Kate's is working on it's very cool!
Leadership

There is a Female Exec Leaders Panel discussion featuring six panel members hosted by Kim Bonilla (She Builds founder & President). There was discussion around mentorship that stuck with me and it was really around how mentors can come and go and that's OK. Further discussion about mentors and sponsors, when and why you might need them and how the relationship works which I found very insightful and has given me a bit to think about. I personally wasn't familiar with the sponsor arrangement and hearing about these leaders discuss the two really connected some dots for me. I'll give a special mention to Vanessa Alves from Brazil, such an energized passionate speaker (my notes actually say, Vanessa was awesome).

I haven't listed each panel member, there are too many, catch the full list of speakers on the agenda: https://aws-shebuilds-day-june.splashthat.com/

Sarah James has written a book "Are you Ready to Change the World? Thoughts on Technology Leadership for the Future". This is an excellent interview with well thought out questions and gives you some insight into Sarah and her experience as leader as well as her book. I was impressed by the session and picked up a copy myself, I'm looking forward to the read. I might follow up with a post on thoughts after I've finished.

No affiliation or links etc

https://www.amazon.com.au/Are-You-Ready-Change-World-ebook/dp/B08BK4XNFY

Juliette Murphy tells us all her secrets! No seriously, Juliette is the Co-Founder of FloodMapp and gives us her top 5 secrets to leadership. I won't give them away here but I will say that I found them very useful and there is something for everyone in the secrets that can be applied by everyone no matter the role or experience.

Finishing up

Alrighty, it's been great to get some insights into this community which really promotes women in tech, diversity and inclusivenes. This write up was a very quick summary of the stand outs to the AWS She Builds Day 2021. I'm just scratching the surface on the event, hopefully the replay links from the day will be out shortly so we can share with others.

Shout out to Kim Bonilla and the rest of the She Builds team for a great event! It was very well organized, awesome line up of speakers and great content. If you get a chance to attend the next She Builds event be sure to check it out, go to the live sessions if you can, that way you can engage with the speakers. You might even get your hands on some sweet merch and other prizes ;).

If you're just hearing about AWS She Builds and you want to find out more, there is a LinkedIn group that should put you in touch with plenty of women in tech:

https://www.linkedin.com/groups/13977813/

If I can be of help, feel free to reach out.

AWS official lambda layers with AWS CDK

Mick Jacobsson — Tue, 01 Jun 2021 10:04:18 +0000

This is my first official post since joining the AWS community builders and well it's not super exciting but none the less it was a source of frustration for me so it might be for others...lambda layers.

If you've been writing lambda functions for a while, maybe before layers, you might not be aware of layers. The basic idea of layers is to separate out your code so that you can better share and reuse your functions. For most people this might be libraries or modules, python is a good example where you might have several common libraries across many functions.

This sounds pretty basic, what gives?

It gets a little more complicated when you have different environments, AWS has a specific environment and for the most part it's not a problem. But, every now and then you'll come across a module that isn't compatible. Like I did today. AWS has created a few public lambda layers so now I'll walk through how to use them.

Finding the Lambda Layer AWS Resource Name (ARN)

If you head over to the AWS management console and go to Lambda and then click on an existing function. If you don't have a function just make a dummy one for now. Once you click on your function, head to the very bottom of the page and you'll see a section for layers.

Now you can select Add a layer

The very first screen that you go to next will let you choose from the very few AWS official layers, custom or by ARN. The options are pretty straight forward:

AWS created layers
Layers created by you / your org
Layers shared or accessible external e.g. community / public

In my case, I wanted numpy which is part of the AWSLambda-Python38-SciPy1x layer. Select the version (I've only ever seen 1 version at a time, anyone else noticed multiple versions?) go ahead and select add.

You should be redirected back to the console and now you can see your new lambda added to your lambda function with a new ARN:

Keep that ARN, we'll come back to that...

CDK to the rescue!

Now we are finally getting to the CDK, I'm going to assume you have a CDK project already with Lambda function and like me you're trying to add a Lambda layer from an ARN like the AWS official layer above.

If you don't have a CDK project or you're unsure you can check one of my previous posts, open source on github (look in the lib/lambda folder):

https://github.com/talkncloud/aws/tree/main/athena-appsync

If you do have some code, here is an example of adding the layer:

myLambdaFunction.addLayers(
    lambda.LayerVersion.fromLayerVersionArn(this, 'awsNumpyLayer', 'arn:aws:lambda:ap-southeast-2:817496625479:layer:AWSLambda-Python38-SciPy1x:29')
)

That's it! Now if you deploy your stack you'll have the SciPy AWS Official Lambda layer added to your function. If you're wondering if you need to do anything special in your function, no, just import the module as per normal and it will be accessible in your function.

Can we skip some steps?

Look, I mean, I agree, if you just want to use numpy this sounds like way too many steps. I did go searching for a list of AWS layers and didn't get very far. If you know of a list reach out, I'd be keen to have a look. When you start searching out there you'll find various ARN's for different layers but it seems like it's all over the place.

There is another reason for doing this rather than creating you're own, this particular package, numpy needs to be compiled using the AWS environment rather than just uploading it with your function. You can create your own layers with the AWS Linux Container (maybe another post?) but if you're in a bind and just need numpy this will get you out of a jam.

Wrapping up

Hopefully this will be of use to someone, personally it just wasn't obvious to me how to get the AWS official ARN and then use it in CDK. I was actually thinking i'd be able to reference the name (alias?) somehow in CDK but once you have the ARN it makes more sense. If you are using layers quite heavily I'd be interested to hear from anyone on their workflow for building modules that are Lambda friendly.