DEV Community: Lars Jacobsson

Take StepFunctions' TestState API one step further with samp-cli

Lars Jacobsson — Wed, 06 Dec 2023 17:39:11 +0000

A lot of great features were released by the StepFunctions team just before and during re:invent 2023. One of them was the TestState API that lets developers test individual states in isolation from the rest of the state machine. Until now, we've had to run the state machine from the beginning to reach the state we're working on.

AWS offers two ways to integrate with the API;

Programmatically using the AWS-SDK - useful for unit/integration testing
Via Workflow Studio in the AWS console - useful for iterating over changes in Workflow Studio:

Both are simple operations that let you target an ASL snippet (the task you want to test) with an optional input along with an IAM role, which is typically the execution role of the state machine.

Whilst this is a great feature that allows us to add a whole new level of testability to our state machines, I think there's a gap in developer experience that can easily be filled.

The DX gap

Developers spend their time in code editors and terminals of their choice. Every time they have to open up a web browser, let it be to comment on a Jira issue, Google/ChatGPT something or navigate the AWS console, they get subject to a cognitive distraction. To me, DX is about removing this distraction.

The TestState API is great for iterating over changes in an individual step and lets you massage the input/output processing or conditional logic in a choice state.

Unless you're testing the very first state in the state machine, you'll need to get a sensible input payload. Without having run through the previous steps, this can be difficult and prone to errors through typos or guesswork.

You're likely to look up an execution that successfully reached the step you're working on, copy that input and use it in your test. That's a lot of clicks.

Use `samp-cli` to bridge the gap

https://github.com/ljacobsson/samp-cli#readme is a CLI tool aimed at bridging DX gaps for developers using AWS SAM.

Version 1.0.66 introduces support for the TestState API.

Installation

$ npm i -g samp-cli

Usage

There are three StepFunction's specific commands;

samp sfn init - inits a new state machine in a SAM project
samp sfn sync - establishes a live upstream sync between your local ASL and the cloud
samp sfn test-state - lets you integrate with the TestState API

For the following example I'll use the sync and test-state sub commands.

Consider the following example state machine:

It takes an input of a payment id { "id": "..." }, fetches the payment from DynamoDB, checks the payment status in a Choice state which sends the execution down the appropriate flow.

However, upon running the state machine, each execution ends up in the Unknown error state:

We can assume that something is wrong with the Choice logic, so let's test that one in isolation. As we have an execution which we expected to succeed, we'll want to grab the input from that.

To achieve this I run samp sfn test-state in the same folder as my SAM template and samconfig.(toml|yaml). The tool will attempt to parse your AWS profile, region and stack name from your samconfig. You can override these (see samp sfn --help).

In a different terminal I run samp sfn sync in order to get my changes to the ASL instantly reflected in my test account.

The following video demonstrates the steps I took to test, identify, fix and verify the bug I had in the Choice state - all in less than a minute and without leaving VSCode.

Note that reusing input from recent executions only works with standard workflows. For express you will need to use inputs from a JSON file or from manual input.

Introducing samp-cli for local Lambda debugging of SAM and CDK stacks

Lars Jacobsson — Mon, 24 Jul 2023 20:05:39 +0000

"Ok, how do I run this thing locally", is the first thing most developers from a non-serverless background ask when they're tasked to work on a serverless stack. It's a valid question which seasoned serverless developers have worked around in various ways by:

Using local emulators like sam local, serverless-offline, localstack, etc.
Wrapping Lambda handlers in unit or integration tests.

While these approaches sort of work, they're not ideal:

Local emulators are not always accurate and can be a pain to setup and maintain. Also, they don't work well with other AWS services like S3, SQS, etc.
Unit tests are great for testing individual functions, but they don't test the entire stack and you'll need to keep test payloads up to date.

What you really want is to be able to run Lambda functions on your local machine under the same context and IAM permissions as in the cloud, but have them triggered by real event sources, such as API Gateway, SQS, EventBridge, etc.

I became inspired to do something about this while listening to Sebastian Bille talk about sst.dev at a recent AWS User Group meetup in Stockholm. SST includes local debugging out of the box and Sebastian's demo looked great.

With my FOMO nerve tightly pinched, I felt determined to deliver a similar experience to SAM/CDK users.

My goals for this were:

No need to install any additional dev dependencies (given that you already have samp-cli installed).
No need to change your existing SAM template or CDK code.
No need to change your existing Lambda code.
It should work for all runtimes supported by Lambda. (so far it supports JavaScript/TypeScript, .NET and Python as of v1.0.40. Java, Go and Ruby is coming)

Get debugging

In this example I will debug a CDK stack since it's slightly more complicated than a SAM. The same approach works for SAM templates as well.

I will use dynamodb-streams-to-eventbridge-outbox-pattern by David Boyne as an example.

Install samp-cli

First, install the latest version of samp-cli

$ npm install -g samp-cli

Deploy CDK stack to AWS if you haven't already

$ cdk deploy

Configure launch.config

Note that automatic launch configuration is currently only supported for vscode.

From your project root, run:

$ samp local --debug

For SAM stacks, this step is fast and easy, as it finds all information it needs in the samconfig.toml file, but for CDK you have to give it some additional information.

This one-time step per project guides you through a short wizard to create (or append to) a launch.json file in your .vscode folder.

Start debugging

Now you're ready to start debugging, but before you press F5 it's a good idea to understand what's happening under the hood. This will help you understand why YOU SHOULD NEVER RUN THIS ON PRODUCTION.

The first thing to understand is that the tool replaces your function code in AWS with a relay proxy for the duration of the debug session. Once you stop debugging, it will restore to the last deployed code artifact.

It also updates your function configuration to set MemorySize to 128mb and Timeout to 60 seconds (it can be increased to 15 minutes by adding environment variable SAMP_TIMEOUT=900 in launch.json). The reason for this is to allow you to sit for longer on breakpoints without the function timing out. Also, the relay proxy doesn't require much memory, so setting it to 128mb gets a bit cheaper.

Both the local machine and the relay proxy lambda function establish an MQTT connection to AWS IoT Core. When the relay function gets triggered, it publishes the event payload, context object and environment variables to the MQTT topic. A local event router receives the message and routes it to the correct function on the developer's machine. Finally, the response it returned to the cloud over MQTT.

Video demonstrating setting breakpoints in each of the function in the above architecture as well as hot-reloading code changes:

The following sequence diagram visualises the flow of interactions (high resolution image here):

Stop debugging

When debugging is stopped, a clean-up script is run. This restores each function according to the last deploy in CloudFormation. Once this is done, functions are invoked as normal in AWS Lambda.

If this should fail for whatever reason, like if your AWS credentials have expired, you can re-run it using samp local --force-restore. If that fails, just redeploy your stack.

Limitations

CDK can be set up in many different ways and there will be approaches out there that won't work with this tool. It's been tested with the TypeScript/CDK patterns in Serverless Land. I'd like to hear about your setup though, so please take a minute to raise an issue
Functions using Lambda Layers could cause issues. If the layer is in the same stack, this workaround might work. If you include third party layers, like Lambda Powertools, just make sure you have it included in your devDependencies.
If your function accesses resources in a VPC, you'll need to establish a tunnel to the VPC resources. See the AWS docs. You can also mock these calls by checking the presence of environment variable LOCAL_DEBUG.

Cost

The cost of running this should be small, but please make sure you understand the pricing for Lambda and IoT Core.

In short:

A 128mb Lambda invocation is charged at $0.000126 per minute
$1 per 1 million AWS IoT messages

About `samp-cli`

Samp-cli was previously knows as sam-patterns-cli. It was born in a response to the announcement of serverlessland.com/patterns back in the spring of 2021. Since then, it's grown into a versatile productivity tool for SAM users, hence the rename. Here's a list of what it supports at the time of writing:

Conclusion

Every way to improve the developer experience for Lambda is a win for all the serverless developers out there. Getting fast feedback on the code we write has always been important and has not changed with the serverless movement.

I hope this will speed up development for you as much as it has for me. If not, please let me know any improvements you'd like to see by raising an issue.

Supercharge your StepFunctions productivity with local file system sync in Workflow Studio

Lars Jacobsson — Mon, 02 Jan 2023 22:10:10 +0000

Note: This article is now outdated, but I'll leave it public anyway. As of re:Invent 2023, Workflow Studio with local file system sync is available natively from within Application Composer. I hope that this article served as inspiration to AWS when deciding to build this.

One of my favourite AWS services is Step Functions. It's a serverless visual workflow orchestration service which, once you are up to speed with it, helps you build extremely scalable and resilient applications.

At Mathem, we are heavy users of it as an integral part of our serverless architecture. At the time of writing we have over 70 state machines running in production.

With heavy usage comes ideas on how you want the perfect developer experience to be.

The release of Workflow Studio in 2021 was a huge leap in the right direction. It lets developers quickly and intuitively design visual workflows using drag and drop instead of writing ASL in YAML, JSON or CDK.

This post assumes that you have some pre-existing experience of building state machines in Step Functions, preferably using Workflow Studio.

The problem

Despite how great Workflow Studio is, there is a gap in the full end-to-end developer experience. Once the workflow is designed, the developer has to go through the following manual steps:

Export the workflow as YAML or JSON
Import the file in their project
Replace placeholders in the YAML with DefinitionSubstitutions. In the vast majority of development scenarios, the workflow isn't perfect after the first iteration - it normally needs several iterations. If iterating requires manual steps, then it'll get very costly, not to mention boring, for the developer. For each change, the developer needs to:
Convert their ASL to JSON (if it isn't already)
Import it back into Workflow Studio
Make changes
GOTO 1

Below is a state machine describing the above steps. The steps with a sad smiley are the ones we want to eliminate.

The solution

My ideal solution, alike my #1 AWS wishlist item, would be an AWS supported version of Workflow Studio as a VSCode extension with a bidirectional sync between the designer and the ASL definition. Until we have that I will support this workaround.

Getting started

First, you need to install a Chrome extension from github.com/ljacobsson/sfn-workflow-studio-sync. The instructions are in the readme.

Link Workflow Studio with your local ASL file

Next, you'll need to navigate to Workflow Studio. You can also open an existing state machine and choose Edit and click through to Workflow Studio.

If the extension was successfully installed, you'll see a new button at the top right corner:

Clicking that will bring up a file picker dialog. This is using the File System Access API to create a temporary read/write access between the webpage and the selected file for the duration of the page's lifetime.

Keep your ASL file in one window and Workflow Studio in the other and watch the time saving as it happens:

This solves the export/import manual tasks in the developer experience flow from above, but we want to automate the linking of resources in the SAM template and the ASL definition. This is of course only applicable if you are a SAM user.

Link your SAM template with Workflow Studio

After the ASL file sync has been established, you'll notice two new buttons in the top right corner of the studio:

The 'Force sync'-button forces the sync to happen. In some cases it's needed, for example after changing Choice conditions.

The other button lets us link a SAM template to Workflow Studio in order to do the definition substitution mapping.

Some resource types, for example Lambda, offer a drop down in the GUI to select a hard-coded reference. Using a hard coded reference is normally a bad idea. In order to create a substitution, select 'Enter' at the top of the dropdown list.

This will bring up a dropdown representing the resource types in your SAM template along with their available attributes:

Selecting an option will update both your SAM and ASL simultaneously:

Most resource types, however, require you to complete JSON to set up your integrations. Here you simply type in the substitution variable you want to use:

You can add more than one substitution variable per JSON configuration:

Summary

This extension is heavily dependent on the DOM of the Workflow Studion, which I have no control over. Chances are that AWS will release a breaking change at any time. If that happens, please submit an issue and I'll do my best to fix it.

AWS prioritises new features based on customer feedback, so if you like this, please submit a feature request to AWS for official support <3

Follow me on Twitter (@lajacobsson) for updates on this and other serverless developer experience tools I maintain.

Building a low code URL shortener using API Gateway, StepFunctions and DynamoDB

Lars Jacobsson — Tue, 25 Oct 2022 13:37:00 +0000

This post describes how you can implement a simple URL shortener using native low code AWS services. Some previous knowledge of API Gateway and StepFunctions is assumed.

Motivation

At Mathem, we use a home built URL shortener in SMS communication with our customers. Until now, this has been hosted on an on-prem legacy environment which is soon to be turned off, so we had to move it to our serverless architecture in AWS one way or another.

There are plenty of serverless URL shorteners publically available on GitHub. However, I wanted to explore if there's a way to achieve this without having to write any Lambda function code at all.

I'm a huge fan of Lambda, but it does come with (for this use case, arguably insignificant) cold starts and the responsibility of keeping the runtime version up to date.

The StepFunctions team has made a couple of releases during 2022 that has enabled state machines to do more than just orchestrating Lambda functions invocations, and this solution would not have been possible a few months ago prior to the new intrinsic functions were released.

The full solution is available on GitHub

Creating a short URL

State machine design

The state machine consists of three states, Initialise, Create hash and Store URL as well as some logic to handle duplicate hashes.

Let's go through each of them from top down:

Initialise

Initialise:
  Type: Pass
  OutputPath: $
  Parameters: 
    Splitter: "-"
    Attempts: 0
  Next: Create hash

This is a Pass state that initialises the execution. It passes on two parameters; Splitter, which is used to split the UUID in the next step as well as Attempts which is used to avoid an infinite loop if all hashes are already taken.

Create hash

To get a short, but unique hash to hide the long URL behind we'll make use of three new intrinsic functions:

This is also a Pass state and looks like this:

Create hash:
  Type: Pass
  OutputPath: $
  Parameters:
    Hash.$: States.ArrayGetItem(States.StringSplit(States.UUID(), $.Splitter), 1)
    Attempts.$: States.MathAdd($.Attempts, 1)
  Next: Store URL

Note how the output from the first state, $.Splitter is used here. Ideally we'd like to just use States.StringSplit(States.UUID(), "-"), but the StringSplit function expects a valid JSON path as the second argument.

The UUID is formatted like this: ca4c1140-dcc1-40cd-ad05-7b4aa23df4a8. Splitting it on the dash ('-') character gives us this array:

["ca4c1140", "dcc1", "40cd", "ad05", "7b4aa23df4a8"]

It's always divided in lower case, alphanumeric sequences of 8, 4, 4, 4, and 12 characters.

Next, we have to decide how long a hash we want and it comes down to how many different combinations (n³⁶) of URLs we need.
4 characters: 1,679,616 permutations
8 characters: 2,821109907×10¹² permutations
12 characters: 4,738381338×10¹⁸ permutations

We are fine with 4 for our use case, so we'll access it using index 1: States.ArrayGetItem(splitArray, 1).

Note that we are limited to lower case characters. To make short hashes with a mix of casing, we'd need a Lambda function in the mix.

Store URL

This state takes the output and stores it in DynamoDB using a native service integration

Store URL:
  Type: Task
  Resource: arn:aws:states:::aws-sdk:dynamodb:putItem
  ResultPath: null
  Parameters:
    TableName: ${UrlTable}
    ConditionExpression: attribute_not_exists(Id)
    Item:
      Id:
        S.$: $.Hash
      Url:
        S.$: $$.Execution.Input.Url
      HitCount: 
        N: "0"
  Catch:
    - ErrorEquals: 
        - DynamoDb.ConditionalCheckFailedException
      Next: Continue trying?
      ResultPath: null
  End: true
Continue trying?:
  Type: Choice
  Choices:
    - Variable: $.Attempts
      NumericLessThan: 10
      Next: Create hash
  Default: Fail
Fail:
  Type: Fail

Note the ConditionExpression and the error handling in the Catch clause. This handles scenarios of duplicate hashes and will simply generate a new one until it finds an available one. As a safety guard it will bail out after 10 attempts. In a production environment you'd want an alarm on when that happens as it's an indication that the number of available permutations are running out.

Accessing a short URL

This state machine is much simpler and only contains a single state that does two things; increments a hit counter and returns the long URL.

The ASL looks like this and uses an SDK integration to DynamoDB:

StartAt: Do redirect
States:
  Do redirect:
    Type: Task
    Resource: arn:aws:states:::aws-sdk:dynamodb:updateItem
    Parameters:
      TableName: ${UrlTable}
      ConditionExpression: attribute_exists(Id)
      ReturnValues: ALL_NEW
      UpdateExpression: SET HitCount = HitCount + :incr
      ExpressionAttributeValues:
        :incr:
          N: "1"
      Key:
        Id:
          S.$: $.hash
    ResultSelector:
      Url.$: $.Attributes.Url.S
    End: true

Hooking the state machines up with API Gateway

At first I wanted to use a HttpApi to enjoy lower latency and less cost, but it proved really hard to get the request and response mapping working. The main issue was that the output from the state machine comes as stringified JSON, and when using HttpApi, the $util.parseJSON() function wasn't available. Shoutout to all Community Builders, and in particular @jimmydahlqvist who got engaged in the problem <3

After much frustration I swapped to use a RestApi, which made my life easier. I will not go into details here, but let's zoom in on the request and response mapping. The full OpenAPI

Create URL: (POST /)

responses:
  200:
    statusCode: 200
    responseTemplates:
      application/json: 
        Fn::Sub: "#set($response = $input.path('$'))\n { \"ShortUrl\": \"https://${DomainName}/$util.parseJson($response.output).Hash\" }"
requestTemplates:
  application/json: 
    Fn::Sub: "#set($data = $input.json('$')) { \"input\": \"$util.escapeJavaScript($data)\", \"stateMachineArn\": \"${CreateUrl}\" }"

Redirect to URL (GET /{id})

responses:
  200:
    statusCode: 301
    responseTemplates:
      text/html: "#set($response = $input.path('$'))\n#set($context.responseOverride.header.Location = $util.parseJson($response.output).Url)"
requestTemplates:
  application/json: 
    Fn::Sub: "#set($data = $util.escapeJavaScript($input.params('id'))) { \"input\": \"{ \\\"hash\\\": \\\"$data\\\" }\", \"stateMachineArn\": \"${RedirectToUrl}\" }"

Too see the above mappings in context, visit the OpenAPI spec here

Conclusion

This article showed how we can use StepFunctions new intrinsic functions together with its native SDK service integrations to create a fully functional, yet simple, URL shortener. It certainly comes with some limitations that Lambda can solve and if you hit them, feel free to extend the workflow with a function.

If you have any improvements, such as converting to HttpApi or introducing better hashing, please submit a pull request

Building this project I spent 5% on creating the state machines and 95% on the API Gateway mappings. I'm hoping to see an improved SAM support for connecting API Gateway and synchronous StepFunctions Express state machines. Please upvote this issue if you agree.

The serverless architecture of a talking doorbell

Lars Jacobsson — Sun, 09 Jan 2022 21:33:16 +0000

In this post I'll describe how I used some of AWS' serverless offering to build a doorbell that leverages AI and speech synthesis to describe the person or people outside the door.

Here's a video demo. Keep on reading to learn how it's built.

Note: age approximation needs to be worked on

Motivation

My family has gone through quite a few cheap wireless doorbells over recent years. They've all had issues such as breaking randomly and not warning when the battery is low.

We've considered buying a Google Nest Doorbell, but they are quite expensive, so I decided to build my own doorbell that takes the concept to the next level.

The speech synthesis isn't just a fun feature, it could also be useful for visually impaired people.

Tech stack

Hardware

Raspberry Pi 3B+
Raspberry Pi camera module
2 meter Raspbery Pi camera flex cable
Small speaker with 3.5mm AUX interface
Shelly Button 1 (could really be any MQTT enabled button)

AWS services

API Gateway
S3 - for image and speech storage
EventBridge - to act on items created in S3
Lambda - compute
Simple Email Service - to send email alerts
Rekognition - picture analysis
Polly - speech synthesis
Iot - MQTT broker to send messages to the Raspberry Pi
StepFunctions - to orchestrate the above

High level architecture

Putting the hardware together

The first challenge was to get the camera installed ouside the door. My door has a roof over it and right next to it a window. I 3D printed a small camera mount and squeezed the ribbon cable between the window and the window frame. Since I put it out it's been both humid and cold (-15C) and the camera is still alive.

I also printed a wall mount for the button and modded it slightly to add my family's surname.

Next I configured a Mosquitto MQTT broker on the Raspberry Pi and configured the Shelly Button to send its events to it.

Raspberry Pi services

The Raspberry Pi is running three services:

camera.py - listens to MQTT messages from the doorbell button, takes a picture and uploads to S3 via a presigned URL it fetches from a Lambda function hosted behind API Gateway*
speech.py - listens to MQTT messages from the AWS IoT endpoint which contain a pre-signed S3 URL of an audio file describing the guest
battery.py - Reports battery status to an API Gateway endpoint*
I'm using IP restriction on the API Gateway endpoints to only allow traffic from my home network. IPs can be spoofed, so an enhancement here could be to use a Lambda authorizer with a client secret.

My Python skills include a lot of googling and copy/pasting, so I will not go into details on these scripts.

Cloud infra

When a new image lands in the S3 bucket a number of independent things need to happen;

The image needs to be analysed for faces, emotion and other labels. This is done in two independent Amazon Rekognition jobs
If there are faces found, generate a human friendly string based on the features and labels in the image
Send this information to a number of destination - in this case email and MQTT back to the RaspberryPi
In the case of MQTT also generate a speech synthesis

Whilst all this could be achieved in a single Lambda functions, I wanted to acheive a clean orchestration with independently retryable tasks, for which StepFunctions is the obvious choice in AWS.

I'm using the native S3 to EventBridge integration to trigger the state machine with the following event pattern:

S3Event:
  Type: EventBridgeRule
  Properties:
    EventBusName: default
    InputPath: $.detail.object
    Pattern:
      source:
        - aws.s3
      detail-type:
        - Object Created
      detail:
        bucket:
          name:
            - !Ref S3Bucket
        object:
          key:
            - prefix: image/

This rule triggers the state machine when a new image with a prefix of image/ is uploaded to the bucket.

I'm not normally a fan of drag-and-drop, but the StepFunctions Workflow Studio is actually really nice to get that initial ASL generated.

The state machine looks like this and below I'll go through and describe each task.

State machine tasks

1. Parallel image analysis

I want to describe the person at the door by both facial features such as age, gender, beard, emotion, etc and by other labels, like clothing or other things they might be carrying.
Amazon Rekognition offers two separate endpoints for this;

Both jobs are run on the same image independently from eachother, hence I can use a parallel state.

Until recently this would have to be run in a Lambda Function, but since the release of StepFunctions AWS-SDK integration we can now integrate directly with most services without glue code in Lambda.

The output of the parallel state is an array of both tasks outputs.

2. Has faces?

This is a choice state that checks if the face detection gave any results. There are three branches of this:

No faces were found - use generic message.
Exactly one face was found - generate description and enrich with results from label detection.
More than one faces were found. Since there's no way to link the label detection to the faces I skip the label enrichment and just describe the people by their faces.

The choice state gives each branch exactly one thing to do without if statements. In the case of #1, there's no need to even invoke a Lambda function.

For each face / Build face description string

This is a map state that iterates through each face from the face detection and invokes a function Build face description string. Each function returns a string like "a happy man aged about 39". The output of the map state is an array such as ["a surprised man aged about 25", "a happy woman aged about 32"].
Source

Combine descriptions

This is a simple function that joins the strings from the map state into one string and modifies it to make it gramatically ok.
Source

Build single face description

When exactly one face is detected we can enrich it with the output form the object label detection. The output of this function can be something like "a surprised man aged about 25 with a baseball cap and a jacket"
Source

Generic message

This is a pass state that outputs the generic message "There is someone at the door"

Send message to notification channels

In the first version I'm happy with receiving an email when someone is at the door as well as the audio description. This can easily be extended with Slack notifications, etc.

In the 'send email'-branch we need a pre-signed S3 URL for the image before firing off the email which looks like this:
Source

If there are more than one person in the picture, then each person will be described, but without the label enrichment:

Synthesise speech and send to MQTT

This step calls a Lambda function that does the following;

Uses the Amazon Polly SDK to synthesize speech
Uploads audio to S3
Generates a pre-signed URL to the audio
Publish URL to an MQTT topic [Source]https://github.com/ljacobsson/ai-doorbell/blob/main/src/SynthesizeSpeech.js

The first approach was to use the StepFunction's SDK integration and call StartSpeechSynthesis without a Lambda function and then act on the audio that it uploaded to S3. However, I saw delays of ~15 seconds before the audio was available, so I decided to pack it all into one function for performance reasons.

REST APIs

There are two API Gateway endpoints:

GET /url - called by the raspberry Pi to get a pre-signed URL with which it uploads the captured image
PUT /battery - The Shelly button reports on its battery level via a MQTT topic on every button press. It PUTs the current level and it it goes below 20% it'll send an email to the user so they can take it of fthe wall for a couple of hours to charge.

Areas of improvement

Sometimes there's quite a long delay between the ding-dong and the speech. This brings a risk that the description of the person might be read out after the door has been opened which could lead to an uncomfortable situation depending on how the AI interpreted the person. So far, whilst testing, it hasn't said anything rude or offensive.
Train a model to recognise people I know. My fellow AWS Community Builder Pubudu Jayawardana has also built a doorbell that does just that which he describes here
Convert the state machine to an express workflow. This will be faster and cheaper. However, I keep it as standard for now to get the visual debugging option in the console. We don't get enough people visiting to make the pricing benefit of express noticable.
It currently connects to AWS IoT without [AWS Greengrass}(https://aws.amazon.com/greengrass/). If and when this doorbell goes on sale I'll need a better fleet management, so I should run the python scripts under Greengrass.

Summary

This doorbell has been deployed at my home for a week now and is the first of my IoT hobby projects that we've actually have found useful.

The first working version of it took two hours to build and didn't use Polly for speech synthesis. Instead it used pyttsx3 which gave the user the voice response a bit faster faster, but with a very unpleasant tone:

The mechanical bell is something I might bring back in a future version.

You can find the project on GitHub:

ljacobsson / ai-doorbell

AI doorbell that uses speech synthesis to describe the person at the door

An approach to loosely coupled CloudWatch alarms and contextual alerts

Lars Jacobsson — Tue, 14 Sep 2021 21:50:27 +0000

We have for the past four years used a third party for monitoring and alerting of our large scale serverless ecommerce platform.

For a number of reasons, we have recently decided to leave this provider in favour of CloudWatch.

This post will take you through how we used my two favourite AWS services, EventBridge and StepFunctions, to set up error anomaly detection for 100% of our Lambda functions along with context rich Slack alerts in less than two days. This model supports any resource type and alarm type, but for brevity I will focus on error anomaly alarms for Lambda.

Some understanding of both EventBridge and StepFunctions is assumed.

One thing we lacked with the third party solution was a coupling with CloudFormation stacks. Monitors were created by hand and if something was removed on the AWS side we were left with an orphaned monitor.

Being obsessed with automation, we needed a fast way to onboard all core resources to being monitored by CloudWatch. As a first approach we build cfn-alarms, a CLI tool to generate CloudFormation alarm and alerting boilerplate resources based on the resources in a template, but to roll that out on hundreds of stacks proved inefficient. However, we still wanted to tie the alarms with the lifecycle of the resources they monitor so that when a Lambda function is deleted, the associated alarms are deleted with it.

Loosely coupled alarms

At first we looked at what we require our teams to monitor when they deploy features to our platform. We came up with a quite small list of services including Lambda errors, SQS queue depth, API 5XX/4XX rate, etc.

When new resources are created or deleted, CloudTrail emits events to EventBridge's default bus with the configuration of the resource. To create or delete alarms we can simply create rules to match these events and route them to Lambda functions that programmatically spin them up or down:

  LambdaCreation:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./src
      Handler: lambda/creation.handler
      Events:
        LambdaCreationEvent:
          Type: EventBridgeRule
          Properties:
            InputPath: $.detail.requestParameters
            EventBusName: default
            Pattern:
              source:
                - aws.lambda
              detail-type:
                - AWS API Call via CloudTrail
              detail:
                eventName:
                  - prefix: CreateFunction
    [...]

This rule will forward the CloudTrail event's requestParameters to the function handler, which looks like this (truncated for brevity. Refer to the SDK docs for syntax):

exports.handler = async (event) => {
  const functionName = event.functionName;
  const tags = event.tags;

  const threshold = tags["alarm:lambda:errors:anomaly:threshold"] || 2;

  await cloudWatch
    .putAnomalyDetector({
        ... truncated ...
    })
    .promise();

  await cloudWatch
    .putMetricAlarm({
      AlarmName: `auto:${functionName}:lambda:errors:anomaly`,
      AlarmDescription: `Error anomaly detected`,
      Tags: Object.keys(tags).map((p) => {
        return { Key: p, Value: tags[p] };
      }),
      ... truncated ...
    })
    .promise();
};

Note how we allow for customisation of certain alarm configurations by using a tagging strategy of alarm:<service>:<metric>:<evaluation type>:<variable>. This naming is reflected in the AlarmName property for a semantic coupling. Also note how we relay the tags from the monitored resource onto the alarm resource. This will later be used in the alerting phase when monitors go in and out of alarm.

The alarm lifecycle event flow is very simple and looks like this:

This approach also allows us to build custom tooling to onboard or reconfigure the entire platform in one go without the need of deploying all stacks.

Alerting

Getting alerting right is difficult and is a balance act of not skipping valuable alerts, but at the same time keeping it brief and relevant to avoid alarm fatigue.

At Mathem we use Slack across all development teams and we have a requirement to tag all resources with the team name owning the service. We wanted to leverage this to automatically direct alerts to team specific Slack channels. This is why we relay the resource tags on the alarm resource in the code example above.

When a CloudWatch alarm changes state there's an event put on EventBridge. A state can be one of OK, ALARM and INSUFFICIENT_DATA. We are interested in alerting on ALARM and communicating recovery on OK. The aim is to be as brief as possible whilst giving the developers quick access to extended context, such as logs and CloudWatch metrics related to the alarm. For this we'll use a StepFunctions state machine triggered by an EventBridge rule:

source:
  - aws.cloudwatch
detail-type:
  - CloudWatch Alarm State Change
detail:
  state:
    value:
      - ALARM
      - OK
resources:
  - prefix: !Sub >-
      arn:aws:cloudwatch:${AWS::Region}:${AWS::AccountId}:alarm:auto:

Note the prefix matching on the alarm ARN. Including auto: at the end ensures only automatically created alarms are consumed by the state machine.

The state machine takes different actions depending on the alarm state and resource type the alarm is concerning.

The first state simply fetches the tags from the alarm resource. The tags we are interested in are team, aws:cloudformation:stack-name and aws:cloudformation:logical-id. The team tag decides where to send the alert and the aws:cloudformation:... tags are used to make the alert message more human readable.

Next, we check if it's an alarm or a recovery. If it's an alarm we'll send a message to a Slack channel following #alerts-<teamname>-<environment>. If the channel doesn't exist, our Slack bot creates it for us.

The alert is short and to the point. It also provides the developer buttons that instantly takes them to either the alarm page or the failing resource's page in the AWS console. This saves us from wasting time manually clicking our way to the root cause.

After the message is sent we pass the output to a parallel state. In one branch we store the message id, or timestamp, we get back from Slack to a DynamoDB table. We'll retrieve this when the alarm has recovered to update the original message.

The other branch in the parallel state allow us to extend the alarm notification with additional context depending on the resource type we're notifying about. At this point in time we have only implemented extra context for Lambda error alarms for which we fetch the most recent error log and post it as a thread reply along with a link to the log group:

This can be extended with for example X-Ray data or whatever might be useful without flooding the channel.

When an alarm reach an OK state we update the original alarm instead of posting a new message.

Summary

This post covered an approach using EventBridge and Lambda to onboard a large set of resources to be monitored by CloudWatch Alarms and StepFunctions to create contextual Slack alerts that can easily be extended to other notification channels.

You can find a reference project for this post here. The state of the project is early days and be mindful of CloudWatch Alarm costs before deploying.

Accelerate your serverless development with sam-patterns-cli

Lars Jacobsson — Tue, 06 Apr 2021 21:17:39 +0000

This post targets users of the AWS Serverless Application Model (SAM)

AWS recently released the Serverless Patterns Collection which serves as a central directory of common best practice serverless patterns that, when they are combined, can make up complex applications.

At Mathem we have a home built templating engine that contains patterns that are common to our SAM stacks which developers can inject directly into their templates. Many of these are specific to our use cases and not easy to open source.

Contributing to our solution is fairly easy, but is still enough of a learning curve and a context switching exercise for the collection to remain more basic than we’d desire.

Our initial thought when we saw the Serverless Patterns Collection announcement was to build a command line interface around it to make the patterns easily accessible and usable directly from the code editor. From experience we've learned that for such collection to be alive and grow there has to be minimal effort involved in contributing to it. The goal is that a pattern gets written once, then shared and reused.

Introducing sam-patterns-cli

Installation:
npm install -g sam-patterns-cli

This is still a very young project under development - version 0.0.9 at the time of writing, but is nevertheless ready to use.

It’s by default linked to the Serverless Patterns Collection’s GitHub repository, but more sources, public or private, can be added.

The tool comes with 4 commands;

source - Lets the user add their own repositories. This could be a private and company specific repository or some other collection, like Jeremy Daly’s Serverless Reference Architectures. See the readme for instructions on how to link it
explore - Lets the user navigate available patterns and visualise then using the power of cfn-diagram
import - Lets the user import a pattern straight into their CloudFormation/SAM template without manually copy/pasting
share - When working on a template, a developer might identify some resources making up a serverless pattern. This lets them extract that pattern and share it with other users of the tool by pushing it to a patterns collection on GitHub

In the following example we'll build an application that receives order updates from EventBridge, processes the order in a Lambda function and stores it in an S3 bucket. There will be no focus on the function code itself - just the SAM template.

We launch sam-patterns import and get presented with a list of available patterns. Searching for ‘eventbridge lambda’ gives us a few options. These patterns are named according to the flow order of them. In our case the ideal pattern would be ‘eventbridge-lambda-s3’, but the closest one is ‘eventbridge-lambda’, so we’ll choose that.

At the time of writing there was no pattern where a Lambda function writes to an S3 bucket, so we’ll go ahead and create one. The template now looks like this:

AWSTemplateFormatVersion: 2010-09-09
Transform:
 - AWS::Serverless-2016-10-31
Resources:
 OrderConsumer:
   Type: AWS::Serverless::Function
   Properties:
     CodeUri: src/
     Handler: OrderConsumer.handler
     Runtime: nodejs14.x
     Timeout: 3
     Policies:
       - S3WritePolicy:
         BucketName: !Ref OrderBucket
     Environment:
       Variables:
         OrderBucket: !Ref OrderBucket
     Events:
       Trigger:
         Type: EventBridgeRule
         Properties:
           EventBusName: custom-bus
           Pattern:
             source:
               - order-service
             detail-type:
               - order-updated
 OrderBucket:
   Type: AWS::S3::Bucket

We have now made a modification to an existing pattern that makes up a new pattern; EventBridge->Lambda->S3.

To avoid having to write the same code again or to help someone else in your team or in the world to quickly create the same pattern you can now share it back to GitHub. If the pattern is of a generic nature we encourage you to submit a pull request to aws-samples/serverless-patterns, but to quickly share it with your team you can submit it to a repository of your own.

To do this, we’ve created a repository, github.com/mhlabs/sam-patterns-collection that will serve as our patterns repository. Next we’ll add it as a source for sam-patterns-cli. For this we use sam-patterns source.

We’ve now linked our repository and we’re ready to use sam-patterns share to share the pattern with the world.

It’s likely that the developer reusing this pattern will work on a stack that handles something else than orders, so in this step we’ll make the shared template dynamically editable by the end user.

That’s it! The new pattern is ready to be used by ourselves or by someone else needing the same functionality.

Let’s say that someone is building a payment processor service that receives payments from EventBridge, processes them and stores the records in S3. They’ll be up and running in seconds:

The user gets prompted to fill in the dynamic values; item name, Lambda runtime, eventbus name, pattern source and detail-type and that’s all they need to do. Now, this was a simple example for the sake of brevity. More complex patterns can easily be created and shared.

Summary

This example took us through how to import a serverless pattern, modify it into a new pattern, share it and, finally, reuse it.

To understand how the dynamic values work, please head over to the pattern we created in this example and study the Metadata section of the template.

At this point in time only SAM template code can be imported and shared. The aim is to support backing files, such as Lambda function code and OpenAPI definitions in the future.

This project is open source and can be found here. Contributions, bug reports and feature requests make us happy :-)

Maximize your EventBridge productivity with evb-cli

Lars Jacobsson — Tue, 29 Dec 2020 07:46:56 +0000

The aim of this article is to demonstrate how evb-cli can be used to boost your productivity when working with Amazon EventBridge. I will present use cases and how you can use the tool to solve them.

I assume that you are familiar with or a user of EventBridge. If not, there are some great videos and blog posts available to get started.

This is part one of two covering this tool. Some commands require a back-end deployed to the target account and for the sake of digestability I will cover them in a separate post at a later date.

Background
Prerequisites
Installation
Schema Registry basics
Commands
- evb pattern - generate event patterns from schemas in the Schema Registry
- evb input - generate InputTransformer based on schemas in the Schema Registry
- evb code-binding - generate code bindings for you Lambda consumers
- evb browse - browse where and how events from a given source/detail-type are consumed
- evb diagram - Visualise your EventBridge architecture
- evb extract-sam-event - convert SAM notation to AWS::Events::Rule for more complex use cases
- evb test-event - test event pattern against an event payload

Commands covered in part two:

evb replay - replay archived events against select target(s)
evb replay-dead-letter - replay dead letter events
evb local - local debugging

Background

EventBridge was released in July 2019, but it took about six months before we started using it heavily at MatHem. Up until then we were leveraging SNS and SQS for our events, but as we started playing with EventBridge we quickly noticed how it improved the decoupling of our services and decreased friction between teams.

Prior to EventBridge, when someone from team A wanted to subscribe to messages using filtering from an SNS topic owned by team B, they had to submit a pull request to team B to add the message attribute they wanted to filter on. We found this being backwards and inefficient.

With EventBridge, the producing team doesn't need to know its consumers. The consumers have access to perform filtering on any part of the payload, but they still need to know the contract of the data they can filter on. The release of the EventBridge Schema Registry at re:Invent 2019 meant that the decoupling got even more optimized.

As we started composing patterns we found that although we had removed team friction we still spent a lot of time on getting the patterns right. Also, writing complex patterns is error prone and requires multiple deploys to test and get right, so we began automating the bridge between the Schema Registry and the pattern composition.

The first version of evb-cli had that sole purpose - to generate event patterns to be pasted into the template. As we got really fast at that, we quickly found further bottlenecks, so we added commands for composing InputTransformers, architecture visualization, code bindings, etc. The sections below describe each command in depth.

Prerequisites

Much of the functionality is based on content in the EventBridge Schema Registry. Make sure you have enabled schema discovery on your event buses.

To get the most of this tool you need to have the aws-cli preconfigured with an IAM or SSO user with permissions covering events:Get*, events:List*, schemas:Get*, schemas:List* and events:List*.

You will also need NPM package manager and NodeJS 12+ installed on your system.

Installation

npm install -g @mhlabs/evb-cli

Schema Registry basics

To make the most of this tool it's important to understand the basics and the limitations of the EventBridge Schema Registry.

There are three types of registries;

AWS event schema registry - the schemas of the events the AWS services produce. These are the events that live on the default event bus
Discovered schema registry - on a custom eventbus you can enable schema discovery. These schemas for these events end up here. They can be from an SaaS-provider or your own custom events
Custom schema registry - here you can upload your own schemas

The discoverer ingests a sampling of the real events and analyses the values. An annoying thing here is that it can't handle null very well and seem to create new versions from 2 events where a value is null in one and non-null in the other. This means when we run commands like evb pattern it only knows the structure of the last event it ingested. If that contained many null values it simply won't know about them.

Another important thing to know is that what constitutes a schema is the combination of source and detail-type. For example, the following event will get an entry in the registry under the name my-source@my-detail-type

{
  "source": "my-source",
  "detail-type": "my-detail-type
  ...
}

It's therefore a good practice to stick to the same detail structure for all events sharing that combination.

Commands

The commands of the CLI have come to life organically as we identified the need for them.

`evb pattern`

Purpose: Quickly and accurately build event patterns to match events from the EventBridge Schema Registry

Example use case: You are tasked with sending a Slack message to a channel each time a CodePipeline execution in any US region fails.

The events from CodePipeline look like this (taken from here):

{
  "version": "0",
  "id": "CWE-event-id",
  "detail-type": "CodePipeline Pipeline Execution State Change",
  "source": "aws.codepipeline",
  "account": "123456789012",
  "time": "2017-04-22T03:31:47Z",
  "region": "us-east-1",
  "resources": [
    "arn:aws:codepipeline:us-east-1:123456789012:pipeline:myPipeline"
  ],
  "detail": {
    "pipeline": "myPipeline",
    "version": "1",
    "state": "STARTED",
    "execution-id": "01234567-0123-0123-0123-012345678901"
  }
}

The pattern you're after looks like this:

source:
  - "aws.codepipeline"
detail-type:
  - "CodePipeline Pipeline Execution State Change"
region:
  - prefix: "us-"
detail:
  state:
    - "FAILED"

Even though this is a very simple pattern it still requires googling the event structure if you want to compose it manually.

Using evb pattern you can solve this in seconds:

Note that I'm passing -t template.yaml to the command. This will prompt you with the option to attach the generated pattern to existing resources in your template. These can either be of type AWS::Serverless::Function or AWS::Events::Rule. If you skip the -t flag, the event will be written to the console.

Template injection works for both YAML and JSON, but note that any YAML comments will be stripped during serialization.

`evb input`

Purpose: Quickly and accurately build InputTransformer CloudFormation objects.

Example use case: Building on the use case above we now want to invoke a Lambda function when events from CodePipeline match our pattern. To make the function code simpler, we only want to forward the properties we actually need from the event payload. Let's say we want to let the Slack users know the name of the pipeline and the region it was running in.

To achieve this we need to compose an InputTransformer block for the Target under the AWS::Events::Rule resource that maps JSON paths form the original payload into a new template:

InputTransformer:
  InputPathsMap:
    region: "$.region"
    pipeline: "$.detail.pipeline"
  InputTemplate: "{\"region\": <region>, \"pipeline\": <pipeline>}"

This is tedious copy/paste work and, at least for us, the InputTemplate is often a straight off representation of the InputPathsMap. A nice feature would be if the CloudFormation team made InputTemplate optional and default it to mirror the InputPathsMap if omitted.

evb input is very similar to evb-pattern, but the output is different. Also, at the time of writing it doesn't yet support template injection, so a copy/paste from the console is required:

Note that I pass in -f yaml. This is telling the tool to output the result in YAML. Default is JSON.

`evb code-binding`

Purpose: Generate code bindings in your preferred language for your Lambda function to use.

Example use case: Given the above InputTemplate we now want to receive this as a strongly typed object. For this example I will use C#, but a fill list of supported languages can be found in quicktype's documentation

In the following video I generate two types of bindings; one for the InputTemplate and one for the entire original event.

To get the optimized code binding for my InputTemplate (FromTemplate.cs), I run evb code-binding and point it at my template. At first I get prompted which language I want to generate for. Next it asks me in which registry the schema can be found. Since we're working with AWS produced events, it's under aws.events. If it's from a custom event bus, then you'd select discovered-schemas. Lastly, I select which Rule and Target combination I want to generate for. Here we only have one choice, so I select that one.

For the sake of demo I also generate a class for the entire original untransformed event (FromSchema.cs). I do this by skipping to provide the -t flag. The tool now takes me through prompts to select language, registry, source and detail-type. As you can see, the output is very verbose and the majority of properties will not be of any interest to the function.

`evb browse`

Purpose: Quickly find usages of events conforming to a given schema.

Example use cases:

I could be a producer of an event and I've sent broken data for a period of time and I need to alert downstream teams
I'm about to start working on a feature and I want to check if something similar already exists in the system

Here we can see our Lambda target that consumes events belonging to the aws.codepipeline@CodePipelinePipelineExecutionStateChange schema along with some other pre-existing consumers. I can view the event pattern and any input transformations.

Another way to browse events is to visualize them using evb diagram as described below.

`evb diagram`

Purpose: Visualize how events are flowing within an event bus

Example use cases:

To have a discussion around during project planning meetings
To quickly get a mental view of how things hang together

In my next post I will go through setting up the evb-local back-end. With that deployed on the target account you are able to see the events as JSON in real-time as they happen.

`evb extract-sam-event`

Purpose: To convert a SAM EventBridgeRule to an AWS::Event::Rule to get more control over the rule.

Example use case: Often when you need an EventBridge to Lambda integration it's tempting to start with the EventBridgeRule SAM shorthand. Under the hood, the SAM macro inflates this into a full AWS::Events::Rule and an AWS::Lambda::Permission, but using this disables you from using the powerful InputTransformer. I've opened an issue about this, so please upvote.

I've found myself refactoring a SAM EventBridgeRule into the real thing enough times to automate it with a simple command. Again, note that if you have comments in your YAML, then these will be stripped using this command.

`evb test-event`

Purpose: Tests an event payload against rules on an event bus.

Example use cases:

You have been given an example payload to consume and you want to test your rule.
You are a producer of events and want to see which rules match it.
Integration tests

Conclusion

In this post I've gone through how we optimize many of the tasks developers working with EventBridge encounter - from building patterns, input transformers, code bindings to gaining insights into the system by browsing schema usage or visualizing the architecture.

In my next post I will focus on local debugging and replaying of archived events.

Turning hand written shopping lists into online carts using Amazon Textract

Lars Jacobsson — Wed, 23 Dec 2020 13:41:40 +0000

MatHem is Sweden's leading independent online grocery shop, with a distribution network reaching over 50% of Swedish households and a strong brand built over the past ten years.

Table of content

Motivation
Hardware
Device software
- Authentication
- Components setup
- Capture and upload image
Backend architecture
- Choosing AI service
Designing the case
End result

Motivation

Using your fingers to type on a keyboard or phone when searching for things to buy is the assumed most efficient way to shop online. Along came voice assistants a few years ago as a good alternative to the screen.

I think there's a good point in minimizing pulling the phone up from our pockets and in my family we have a habit of using old fashioned hand written shopping lists that, when it's time to place the order, we type into the search bar at Mathem. This means we're writing it twice - once with a pen and once with a keyboard which is way too inefficient.

To tackle this problem I decided to bridge the gap between the physical list and the online cart by building a scanning device that uses hand writing recognition along with existing Mathem APIs. I wanted to have this done as a weekend project, and knowing that the 3D modelling and printing would be the most time consuming parts, I decided to time cap the development time to a minimum and write as little code as possible.

Since MatHem is already runs on a 100% serverless architecture, the obvious path was to hook this into that and rely on AWS's serverless offerings for AI/ML.

Hardware components

I decided to use components I had lying around at home;

Raspberry Pi 3 Model B+ running latest Raspberry Pi OS
Bump sensor. Used as button. There are probably better buttons out there, but this was accessible
Raspberry Pi Camera Module

The case was printed using a Creality Ender Pro 3

Wiring the bump sensor is easily done. It has three pins - one for ground, one for power and one for GPIO - take note of the pin you use. In the code below I use GPIO 12, but it could be any GPIO.

Device software

Firstly I had to decide where to do the inference - on the edge or in the cloud. There are solutions for this that can run on the Pi, but that would tie this feature to the hardware. What if I want to move it to a less performant hardware or integrate it in our mobile apps?

The only benefit I could see for running it on the Raspberry Pi would be that it'd probably be faster since it would be fewer round trips to the cloud.

With extensibility in mind I decided to make the edge device dumb and literally just capture the image and upload to the cloud and handle the AI there.

Authentication

At Mathem we use Cognito User Pools for authentication. Since the UI for this this device is just a bump sensor we need to perform a headless login. I perform this the same way as you do headless authentication to your WiFi network, but with a custom file in the boot partition, mh-credentials.json:

{
    "username": "me@email.com",
    "password": "mypassword"
}

make sure to change your Pi's default password and make sure that it's not publicly accessible over the internet

In /home/pi/camera we place a python script, camera.py:

import RPi.GPIO as GPIO
import time
import json
from picamera import PiCamera
from pycognito import Cognito
import requests

userPoolId = 'eu-west-1_abcdefgh'
appClientId = '1unghcf5krf83i76325abcdefg'

credentials = None
with open('/boot/mh-credentials.json') as credentials_file:
    credentials = json.load(credentials_file)

user = Cognito(userPoolId, appClientId, username=credentials['username'])

user.authenticate(password=credentials['password'])

This reads the credentials file and uses pycognito to authenticate the username and password.

Components setup

Next, we initialize the camera and the bump sensor

camera = PiCamera()
camera.rotation = 180  # The camera is fitted on the device upside down, so we need to rotate it
GPIO.setmode(GPIO.BCM) # This means I'm using GPIO numbering instead of board numbering

GPIO.setup(12, GPIO.IN, pull_up_down=GPIO.PUD_UP) # Use GPIO 12
GPIO.add_event_detect(12, GPIO.RISING, callback=capture, bouncetime=3000) # add event listener to bump sensor
while True:
    time.sleep(10)  # Wait for input

The above snippet assigns capture() as a callback to the GPIO event.

Capture and upload image

def capture(channel):    
    global user

    # Refresh token if expired
    user = Cognito(userPoolId, appClientId, id_token=user.id_token,refresh_token=user.refresh_token, access_token=user.access_token)

    # Set authorization header
    headers = {'authorization': user.id_token, 'content-type': 'image/jpg'}

    # Request presigned URL
    s3url = requests.get('https://api.mathem.io/cam-cart/s3/signedurl', headers=headers)
    contentDict = json.loads(s3url.content)
    signedUrl = contentDict["url"]

    # Take picture
    camera.capture('/tmp/capture.jpg')
    print('Captured image. Uploading...')

    # Upload
    with open('/tmp/capture.jpg', 'rb') as f:
        http_response = requests.put(signedUrl, data=f.read())
        print('Done!')

Note that the S3 key will have the format <userid>/<timestamp>.jpg. More on that later.

To make this run on startup, add the following to /etc/rc.local:

sudo python3 /home/pi/camera/camera.py

Backend architecture

Already having a 100% serverless architecture made it simple to hook this workflow into the mix.

The prerequisite for uploading an image in a secure manner is to obtain a pre-signed URL that allows this. Having already authenticated the device, we can now call an API Gateway endpoint that authorizes the user before invoking the following code. For nodejs we use Jeremy Daly's lambda-api.

async function get(req, res) {
  const jwt = req.headers.authorization;
  const claims = jwtDecode(jwt);
  // adding custom claim `userid` to the file name
  const filename = `${claims['custom:userid']}/${new Date().getTime()}.jpg`;
  const url = await s3.getSignedUrlPromise('putObject', {
    Bucket: process.env.Bucket,
    Key: filename,
    Expires: 10 // valid for 10 seconds
  });
  return res.status(200).send({ url });
}

Next we need to act when new images get uploaded. We do this by triggering a lambda function upload-trigger.js when new objects get created.

UploadTrigger:
    Type: 'AWS::Serverless::Function'
    Properties:
        Handler: src/upload-trigger.handler
        Policies:
        - AmazonAPIGatewayInvokeFullAccess
        - Version: 2012-10-17
            Statement:
            - Sid: Statement1
                Effect: Allow
                Action:
                - 'textract:detectDocumentText'
                Resource: '*'
        - S3CrudPolicy:
            BucketName: !Sub ${AWS::AccountId}-${AWS::Region}-cam-cart
        Events:
        S3Event:
            Type: S3
            Properties:
            Bucket: !Ref Bucket
            Events: s3:ObjectCreated:*

Choosing AI service

AWS offers two AI services that can recognize hand writing; Rekognition and Textract. At first I wasn't aware that Textract had this capabliity, but it turns out it was announced only about a month prior to the time of writing.

Using the Rekognition API worked ok, but it wasn't great once the hand writing turned scribbly. Also, as you'll see later on, the camera is fitted a bit too close to the paper, so no matter how I adjust the lens, it will always be out of focus on that distance.

Once I realized Textract had similar capabilities I did some side-by-side comparison and saw that using Textract takes this project a step away from being a fun proof-of-concept and closer to an actually useful device.

Here follows an example of using the same blurry image in Rekogintion vs. Textract. Since MatHem is currently only offered in Swedish, so are the items on the list:

Rekognition

Pricing
$0.001 per image after free tier

Textract

Pricing
$0.0015 per document (image) after free tier

Here we can see that Textract nailed all four words while Rekognition really struggled. Now, our product search engine allows for typos, so the end result still often made sense with Rekogintion's results, but Textract was still superior.

Also - Rekognition just gives a list of words and their coordinates in the image while Textract categorizes the matches into blocks, so for example "bar of soap" would be one search term using Textract, but three separate ones using Textract.

Pricing is similar and although Textract is charged a bit higher, for our volumes it's neglectable.

With all that in account and the fact that Textract is the better service name, the choice was simple.

upload-trigger.js

When an image is uploaded, we trigger the following Lambda function:

exports.handler = async function (event) {
  const s3Record = event.Records[0].s3;

  const detection = await textract
    .detectDocumentText({
      Document: {
        S3Object: { Bucket: s3Record.bucket.name, Name: s3Record.object.key }
      }
    })
    .promise();

  // Get user id based on the S3 object prefix
  const userId = s3Record.object.key.split('/')[0]; 

  for (const block of detection.Blocks.filter((p) => p.BlockType === 'LINE')) {
    if (block.Confidence > 80) {
        // call internal APIs to search product and add to cart
    }
  }
};

When calling the Textract API you can choose if you want to do it synchronously or in a asynchronous fire-and-forget manner. The latter means that you pass it an SNS topic ARN to which it sends a notification when the job is done.

Here I went for the synchronous call since these images are small and processed fairly quickly (~2s) and I feel it favours the end-to-end duration.

That's it for the code. The detected text is passed to internal APIs and the front end is updated using web sockets, but that's out of scope for this article.

Designing the case

The initial proof of concept wasn't presented in a very usability inviting way, so I had to wrap all the wiring up in a sturdy case:

The end result is this wall mounted beauty:

I'm a noob when it comes to 3D modelling, but it's really easy to get started with Tinkercad.com.

The case consists of three items;

A case with screw holes for attaching the Raspberry Pi and bump sensor.
An arm holding the camera. The arm is hollow to allow us to hide the flex cable. This arm would benefit from being longer to get less blurry images, but I was constrained to the 10cm length of the stock cable.
A wall mount that the inner construction slides in to.

Inner case and arm

Printed and all wired up it looks like this:

These took ~12 hours to print

Outer casing / wall mount

This took an additional 8-9 hours to print.

The inner casing slides perfectly into the mount and the back of the mount secures the loose cables in place.

End result

Here I have mounted it on the wall in the kitchen next to my Echo Show on which I've used Firefox to browse and log in to mathem.se for the sake of this demo. I also coated the device with a blue paper to add some friction to when I push the shopping list in.

Please get in touch if you have ideas for improvement or other applications for this use case.

A brief summary of our AWS productivity tools

Lars Jacobsson — Sat, 12 Sep 2020 22:29:37 +0000

Table of content

Background - What brought us here?
cfn-diagram - CloudFormation visualizer
evb-cli - EventBridge pattern generator and debugging suite
cfn-resource-actions - VS Code extension that lets you interact with deployed resources from the template
sam-policies-cli - CLI UI for browsing and injecting SAM Policy Templates
iam-policies-cli - CLI UI for building complex IAM policies
cwlogs-cli - Tool to quickly launch CloudWatch Logs Insights with multiple log groups pre-selected

A background

When we in January 2017 made the decision to rewrite Sweden's leading online grocery store, MatHem.se, from scratch into a serverless microservices architecture we found ourselves in an extremely promising yet premature tech segment with a lack of tooling due to a yet quite small community.

This led us to adopting a mindset of 'when tasks get repetitive, find a tool for it or write one yourself'.

Much of our tooling solves bespoke in-house procedural bottlenecks, but when we find solutions to tasks that can benefit the community we make sure to publish it in the open.

This post is a summary of the AWS/serverless related productivity tools we have authored.

cfn-diagram

NPM: https://www.npmjs.com/package/@mhlabs/cfn-diagram
GitHub: https://github.com/mhlabs/cfn-diagram

Installation
npm i -g @mhlabs/cfn-diagram

Features
Parses CloudFormation/SAM JSON or YAML and renders a diagram in either draw.io or vis.js network format.

Examples

Reason for existing
Our application is made up of hundreds of microservices, each defined with its own CloudFormation/SAM template. No engineer knows about every stack. Our engineering team has been going through a rapid growth phase which has required onboarding of new developers to be efficient.

To ease the getting to know a stack, we had an initiative that every repository was required to have an up-to-date diagram over the service's resources in the readme file. No matter how good the intentions were, those sort of 'musts' will never be followed and these diagram will be something that 'might be true' and therefore never trusted.

With cfn-diagram any developer new to a stack can quickly pull up a visualisation of how it all hangs together. It's also useful to render in planning meetings instead of drawing on the whiteboard

evb-cli

NPM: https://www.npmjs.com/package/@mhlabs/evb-cli
GitHub: https://github.com/mhlabs/evb-cli

Installation
npm i -g @mhlabs/evb-cli

Features
Pattern generator and debugging tool for EventBridge with features including:

Event pattern builder
Input transformation builder
Event rules browser
Generate a diagram over the events flow in a region
Local debugging of events where actual events are sent to the console over websockets. These can be forwarded into sam-local for some advanced debugging.

Examples

Pattern generation

Events flow diagram generator

Local debugging

Reason for existing
Historically we have been heavily invested in pub/sub using SNS/SQS, but we were always annoyed with the tight coupling this created between services and teams, so when EventBridge was announced we got very excited.

One of the most powerful features of EventBridge is its content based filtering where you only let through events matching a part of the payload that interests the consumer of the event. Together with EventBridge's Schema Registry an engineer can build event patterns without knowing anything about the producing service.

These event patterns often get quite complex and we found ourselves spending a lot of time first writing, then deploying, then debugging them. Typos were a common mistake both when composing event patterns and input transformations.

Our developers also reported that they find it cumbersome to debug their rules as well as finding it difficult to see the whole picture of how all events hang together, so from that feedback we added local debugging features

cfn-resource-actions

Marketplace: https://marketplace.visualstudio.com/items?itemName=ljacobsson.cfn-resource-actions
GitHub: https://github.com/mhlabs/cfn-resource-actions

Installation
ext install ljacobsson.cfn-resource-actions

Features
Interact with your deployed CloudFormation/SAM templates directly from the template. Turns your template in to an interface into the AWS console

Tail lambda log
Query DynamoDB tables
Copy Physical Id of any resource
Consume EventBridge events directly in vs code
Send to and Poll SQS
Visualise template
etc

Examples

Invoke lambda + tail its logs

CTRL+click from template to Lambda handler

Query DynamoDB

Visualise stack requires Draw.io Integration extension

Reason for existing
Context switching is a huge cost in productivity loss. Leaving VS Code to go and do stuff in the AWS console is a distraction we want to avoid

sam-policies-cli

NPM: https://www.npmjs.com/package/@mhlabs/sam-policies-cli
GitHub: https://github.com/mhlabs/sam-policies-cli

Installation
npm i -g @mhlabs/sam-policies-cli

Features
CLI UI to quickly inject SAM policy templates into your AWS::Serverless::Function resources

Example

Reason for existing
We strive to follow the Principle of Least Privilege and finding the correct policy templates requires googling

iam-policies-cli

NPM: https://www.npmjs.com/package/@mhlabs/iam-policies-cli
GitHub: https://github.com/mhlabs/iam-policies-cli

Installation
npm i -g @mhlabs/iam-policies-cli

Features
UI to quickly build simple to complex IAM policies based on resources in a CloudFormation template

Example

Reason for existing
We strive to follow the Principle of Least Privilege and composing granular IAM policies is time consuming and prone to human errors

cwlogs-cli

NPM: https://www.npmjs.com/package/@mhlabs/cwlogs-cli
GitHub: https://github.com/mhlabs/cwlogs-cli

Installation
npm i -g @mhlabs/cwlogs-cli

Features
Create groups of log groups from resource tags or log group prefixes and launch CloudWatch Logs Insights with those groups predefined

Example

Reason for existing
CloudWatch Insights Logs is a great way to search your logs. Often you want to search across many log groups to, for example, tracing a correlation id across multiple services. CloudWatch Logs Insights lets you search across up to 20 log groups in one query, but adding log groups from the dropdown is a bit cumbersome.

DEV Community: Lars Jacobsson

Take StepFunctions' TestState API one step further with samp-cli

The DX gap

Use samp-cli to bridge the gap

Installation

Usage

Introducing samp-cli for local Lambda debugging of SAM and CDK stacks

Get debugging

Install samp-cli

Deploy CDK stack to AWS if you haven't already

Configure launch.config

Start debugging

Stop debugging

Limitations

Cost

About samp-cli

Conclusion

Supercharge your StepFunctions productivity with local file system sync in Workflow Studio

Note: This article is now outdated, but I'll leave it public anyway. As of re:Invent 2023, Workflow Studio with local file system sync is available natively from within Application Composer. I hope that this article served as inspiration to AWS when deciding to build this.

The problem

The solution

Getting started

Link Workflow Studio with your local ASL file

Link your SAM template with Workflow Studio

Summary

Building a low code URL shortener using API Gateway, StepFunctions and DynamoDB

Motivation

Creating a short URL

State machine design

Initialise

Create hash

Store URL

Accessing a short URL

Hooking the state machines up with API Gateway

Create URL: (POST /)

Redirect to URL (GET /{id})

Conclusion

The serverless architecture of a talking doorbell

Motivation

Tech stack

Hardware

AWS services

High level architecture

Putting the hardware together

Raspberry Pi services

Cloud infra

State machine tasks

1. Parallel image analysis

2. Has faces?

For each face / Build face description string

Combine descriptions

Build single face description

Generic message

Send message to notification channels

Synthesise speech and send to MQTT

REST APIs

Areas of improvement

Summary

ljacobsson / ai-doorbell

AI doorbell that uses speech synthesis to describe the person at the door

An approach to loosely coupled CloudWatch alarms and contextual alerts

Loosely coupled alarms

Alerting

Summary

Accelerate your serverless development with sam-patterns-cli

Introducing sam-patterns-cli

Summary

Maximize your EventBridge productivity with evb-cli

Table of contents

Background

Prerequisites

Installation

Schema Registry basics

Commands

evb pattern

evb input

evb code-binding

evb browse

evb diagram

evb extract-sam-event

Use `samp-cli` to bridge the gap

About `samp-cli`

`evb pattern`

`evb input`

`evb code-binding`

`evb browse`

`evb diagram`

`evb extract-sam-event`

`evb test-event`