DEV Community: Jimmy Dahlqvist

How to re:Invent, episode 1 - Vegas, Travel, and Accommodations

Jimmy Dahlqvist — Thu, 31 Jul 2025 13:51:11 +0000

The time has come for an updated version of my popular re:Invent guide. In 2024, I wrote a very long post and tried to cover everything. This year, I will take a multi-post approach and share my tips and tricks on specific topics in each one.

I have attended re:Invent every year since 2016 and have collected a fair amount of tips and tricks on how to survive not only re:Invent but also Vegas as a city. If you read my 2024 guide, you will recognize some parts but hopefully also discover some new hidden gems.

In this episode, I will delve into Vegas as a city, travel, and accommodations.

AWS re:Invent means you should prepare to walk a lot, get very little sleep, gain a ton of learning, and meet some new friends!

Travel

So, you're coming to re:Invent—that's awesome! You will have an amazing time.

When booking your travel, there are some simple "rules" I have learned over the years that make things much smoother. First of all, I don't recommend getting to Vegas later than the Saturday before re:Invent. Especially if you travel internationally, my experience is that I need the Sunday outside in natural light to fight jet lag. The rest of the re:Invent week will mostly be spent inside with artificial lights. So, a full day of sun and natural light helps my body adjust to the new time zone. Badge pickup also opens on Sunday, and picking up your badge early is a good thing.

I normally travel back home again on the Friday afternoon or the Saturday of the re:Invent week. There are sessions planned until lunch on Friday, so getting a flight in the afternoon is a good option.

Other things to know are that re:Invent takes place the week after Thanksgiving, meaning it's a busy travel weekend. Expect longer wait times at airport security and immigration, especially if you arrive as early as the Thursday the week before. But the entire weekend is busy, so plan for that. Tight connections are probably not a good thing. I rather spend some extra time at the airport than feeling the stress in the immigration line.

Getting from/to the airport

Basically, you have two ways to get from the airport to your hotel: Taxi or Rideshare (Uber/Lyft). Don't assume that Uber/Lyft will always be the cheapest; they often are, but during busy hours, the price goes up, and the wait can easily be an hour or even more. There is a fixed rate between the airport and the hotels on The Strip (Las Vegas Blvd). Sometimes the drivers insist you use the meter, but it will always be more expensive than the fixed price, so don't fall for that trick. This is the first scam to look out for. Normally, I do use Rideshare, but I always check the price and the wait time. One good thing about Taxis is that they can drop you off at the entrance; Rideshares are often referred to the side entrance.

For more information about fixed rates, visit Nevada Taxicab Authority

re:Invent Campus

Before we start digging into accommodations and where to stay, it's good to understand how the re:Invent campus is built. It stretches all the way from Mandalay Bay in the south to Wynn/Encore in the north. This is a huge distance; we're talking about 75 minutes walking or 15 minutes by car, depending on traffic. It is across the entire Strip.

Maps of the campus are available on the re:Invent webpage.

The campus consists of two main categories of hotels: venue and sleeping hotel. A venue is where you will find sessions and content; most venues also serve as sleeping hotels. Then the sleeping hotel is just for accommodations; there are no sessions or content. For all Venues and sleeping hotels, check out Maps of the campus.

Accommodations

So, where should you stay? That is not an easy question to answer. The main re:Invent venue is The Venetian, where the Expo is located. Close to The Venetian is Caesars Forum and Wynn/Encore, which will host a good portion of the sessions. In the south, Mandalay Bay hosts a lot of sessions.

There are many good hotels in Vegas, and where you stay depends on your budget. To be honest, you will not spend much time in your room, so for me, a comfortable bed and a clean room are all I ever ask for. So, I normally like to stay somewhere in the middle. My favorite hotels include Paris and Horseshoe, but also New York New York and MGM in the south end. Venetian is a great hotel but costs a lot more. I feel that Horseshoe is an underestimated hotel; it's very close to Caesars Forum, and you can walk down Linq Ln, which is a great shortcut.

I need to throw in one of my friends' favorite hotels, Casino Royale. I have never stayed here, but the prices are good, and it's close to the venues.

The hotel prices have increased a lot during 2024 and 2025, and the average price on The Strip is now $203. AWS offers deals on several hotels on the campus from the re:Invent portal; to access that, you need to have a ticket in place.

During the last couple of years, I have said that you should not assume that the prices via AWS are always the best; you need to do some research, and most years I have managed to book at a cheaper price outside of the re:Invent portal. For 2025, it does look like the AWS prices are the best.

Here is an extra tip for you if you plan to come back again next year. All hotels in Vegas offer free cancellations, all money back. So, by booking already in February, you can score some sweet deals.

Viva Las Vegas

The first thing to know about Vegas is that it has become much more expensive. See the budget section below to make things a bit cheaper.

Vegas Weather

The winter weather in Vegas is very unpredictable. I have seen everything from snow to very warm weather. In the morning, it can go as low as 5-10°C (40-50°F), and in the afternoon, it can easily reach 25°C (75°F) in the sun. Layers, light jacket, hoodies are your friend. It rarely rains, but when it does, it RAINS. If you have room in your suitcase, packing an umbrella or a rain poncho is a good idea.

Places to Eat

Even though there are meals being served, breakfast and lunch, at re:Invent, you still need to find some good food for dinner and if you arrive in Vegas early. Just to mention, during re:Invent, there will be a lot of free parties where you can also get dinner. But here are my favorite places to eat.

Breakfast

My absolutely favorite breakfast spot is Mon Ami Gabi by Paris. They serve some amazing Eggs Benedict, but their waffles are also really good. Now you can also get both brunch, lunch, and dinner here if you like.

My second breakfast spot is Eggslut inside Cosmopolitan; their egg sandwich is to die for!

Dinner

For dinner, I could mention so many good places, but these are four places that I truly enjoy. You could eat here without totally breaking the bank.

Gordon Ramsay Pub and Grill Inside Caesars Palace, you will find one of Gordon's many restaurants in Vegas. Here you can get some classic pub food but also signature dishes like Beef Wellington.

Yard House On the Linq Promenade, you will find Yard House, a good chain restaurant with great food and beer for reasonable prices.

Virgil's Real BBQ Staying on the Linq Promenade, you will find this BBQ place. Is it the best BBQ in Vegas? No, it's not, but it's still really good and located on the Strip. Off the strip, you will find Rollin Smoke BBQ.

Peppermill and Fireside Lounge A classic restaurant that serves really good steaks.

As an honorable mention, I need to throw in Gordon Ramsay Hell's Kitchen. Now this is really good food and a true experience, but it's super expensive!!

Budget Tips

Cheap Eats
Food in Vegas can be really expensive; there are so many nice restaurants. However, places with cheap and good food are rare. So, these are my favorite spots. All prices mentioned are at the time when I'm writing this post.

Burger
Inside the Miracle Mile Shops in Planet Hollywood, you will find Ketchup Premium Burger Bar. Here you can get a burger and fries for $6.

Vegetarian
Also inside the Miracle Mile Shops in Planet Hollywood, you will find Tacotarian, an all plant-based Taco place. You can get Tacos for $3.99.

Chicken
There are two locations for Dave's Hot Chicken, inside the Miracle Mile Shops in Planet Hollywood and by Grand Bazar shops by Horseshoe. Here you can get a chicken deal between $6-8. Don't be afraid of the Hot Chicken name; you can pick your heat level.

Morning Coffee Hack
The lines at Starbucks can be crazy long in the morning when everyone is trying to grab that first cup of joe. Instead of waiting in line, go to the sportsbook or the center bar of your hotel; most hotels have this. Early in the morning, they will be serving affordable, freshly brewed coffee without any lines. Best survival hack ever for a coffee maniac like me.

CVS, Target, ABC
Buying your snacks at the hotel convenience store is crazy expensive. See the Scams section. Instead, go to CVS, Target, or any of the ABC stores. Here you can get snacks and drinks for a fraction of the price. I normally go once and stock up for the entire week.

Five Things to Do in and Around Vegas

Vegas is the party capital of the world, and there are plenty of entertainment and things to do. During re:Invent week, you will be fully busy with re:Invent, so don't try to add a lot of other things to your schedule. If you arrive a few days before and want to mix things up a bit, this is what I would recommend.

Hoover Dam and Grand Canyon

Rent a car and drive to Hoover Dam and Grand Canyon. I have on several occasions done exactly this. Eagle Point in Grand Canyon West is a good location. This is around a 2.5-hour drive.

Las Vegas Bus Tours
The hop-on hop-off bus tour is a great way to see many of the things. It travels up and down The Strip, and you can get on and off at several locations. A great way to see the Las Vegas sign in the south and ride all the way up to classic casinos like the Golden Nugget on Fremont Street. Check out the website for tickets and maps.

The Sphere

The Sphere opened up in 2023, and there are some good shows, like Postcard from Earth, Wizard of Oz, and some more. I have seen Postcard from Earth and think it was entertaining and fun.

Meow Wolf Las Vegas
I have been to other Meow Wolf installations but never the one in Vegas. Meow Wolf is an interactive art installation, and the one in Vegas features the greatest supermarket experience in the world, created by over 300 artists. Prepare to have your perception of reality altered. Omega Mart is where reality checks out! Visit the website for prices and opening hours.

Neon Museum and Art Districts
The Neon Museum is very popular and is a nonprofit open-air exhibition of over 200 old neon signs salvaged from old casinos and businesses. Many of the signs originally crafted by the Young Electric Sign Company are now preserved in the Neon Boneyard. There are guided tours that bring Las Vegas history to light through storytelling. Visit the website for prices and opening hours.

Five Free Things to Do in Vegas

Now, let's dive into some free things to do in Vegas.

Bellagio Fountain
The Fountains of Bellagio are a choreographed fountain spectacle, with water jets reaching up to 140 meters (460 feet) that are synchronized to music ranging from classical to pop. It opened in 1998 and remains one of the most iconic and frequently photographed attractions on the Las Vegas Strip.

Bellagio Conservatory & Botanical Gardens
Located just inside Bellagio’s lobby, the Conservatory & Botanical Gardens features themed floral and plant displays that change seasonally. Often highlighting events like spring, summer, fall, winter, and Christmas. Open to the public at no charge, and offers a serene, artistic escape from the bustle of the Strip. Conservatory & Botanical Gardens

Las Vegas Sign

The Las Vegas sign is a bit overrated, but as a first-timer, it for sure should be on your list.

Walk The Strip

Just go for a walk down The Strip, take photos of all of the hotels. Walk into M&M Store, Hershey's, Coca-Cola, and several others.

Fremont Street Experience
Fremont Street is the older part of Vegas, and this is where you find classic casinos like the Golden Nugget. Fremont is well worth a visit. Go in the evening, buy a beer, listen to some music, hang around, and eat some cheap food. The prices on Fremont are significantly lower than on The Strip.

Don't Get Scammed

When visiting Las Vegas and re:Invent for the first time, or any time for that matter, you don't want to go home and feel that you got scammed. In this section, I'll try to list some of the most common scams in Vegas; avoid them!

Bicycle Ride
There is a new scam floating around, especially around The Sphere. There are people offering bicycle taxis to and from your hotel. Often without any listed price, and when arriving at your destination, they demand a large sum for the ride, upwards of $100 per person is not unheard of. Avoid them at all costs!

Hotel Convenience Stores

NEVER ever buy things from the hotel convenience stores; this is seriously overpriced. You can have to pay upwards of $25 for a bag of candy or snacks. The prices are also dynamic, meaning that things are more expensive during certain hours of the day. See the budget section for where to buy your snacks.

Hotel Minibar
If the convenience store wasn't bad enough, the minibar is a crazy scam. A bottle of water can be upwards of $25. Also, the minibar comes equipped with sensors, so if you remove an item, you will be automatically charged.

Show Girls and Characters

All over The Strip, you will find all kinds of characters, from Disney to Transformers, and show girls with big feather attire. All of them want you to take a picture with them, afterwards they will require a hefty tip, $50 easy. Now it is a tip, and they can't legally charge you, but at the same time, you probably don't want to make a big scene. IF you would like to take a picture with them as a memory, make sure to agree on the "price" before snapping the image!

Monk Scam

This is not unique to Vegas. All over, you will run into these fake monks that try to give you a bracelet. Just don't take it. If you do, they will start asking that you make a donation. If you then try to hand the bracelet back, they will argue that since you touched it, it has lost its "magic". Just avoid them, and put your hands up if they try to put the bracelet on you.

Fake CD Scam

Also not unique to Vegas, is people who try to give you a "free CD". They claim to be struggling artists and even sign the CD for you. Of course, they also want a donation for the CD. If you take it and hand them some cash, the CD is 100% blank.

Fremont Street Stands

The street stands on Fremont Street are crazy overpriced. If you want to buy from them, make sure to haggle a lot!

Tips, Tips, Tips Everywhere

Vegas runs on tips; many of the hospitality workers rely on tips. I personally don't like the tipping culture, but when in Vegas, make sure you tip the cocktail waitress, housekeeping, taxi drivers, and other hospitality staff. If you go to a re:Invent party with free drinks, tip the waitress. First of all, you will get great service, but second, as this is a free event, they don't make tips, so be kind and leave a tip. Make sure to bring cash for easy tipping.

Five Vegas Rookie Mistakes to Avoid

When in Vegas as a rookie, there are things you should know that maybe aren't classed as scams, rather mistakes to avoid.

Not Trying the $20 Trick
The “$20 trick” is a popular Las Vegas trick for getting a complimentary room upgrade at check-in. When you hand your ID and credit card to the front desk agent, you discreetly include a folded $20 bill between them and politely ask if there are any complimentary upgrades available. While it’s not guaranteed to work, many times it works. Especially during less busy times, making it a legendary—and sometimes effective—Vegas travel hack for better views, larger rooms, or extra perks.

Not Using In-Room Safe
One of the biggest mistakes would be to not secure your room door or use the in-room safe. You don't want anyone to get into your room, but if they do, you don't want them to easily steal your laptop. Use the safe, people!

Hotel Minibar

This almost classifies as a scam! Be aware of the minibar in the room. The drinks and snacks are crazy expensive, and many of the minibars have sensors that detect if you remove an item and automatically charge you! There are normally some vague signs about it, just don't touch the items in the minibar, and absolutely don't remove something to keep the drink you bought at Target cold.

ATM Fees

Make sure you bring cash! The fees charged by the ATM machines in the hotels can be upwards of $10 per withdrawal!

Pot Builder Slot Machine

If you plan to play some slot machines, understand that the "Pot builder" machine, these are machines that show some form of graphics like gold in a pot, fireworks about to go off, pigs that get bigger and bigger. If the pot fills up or the fireworks go off, you win the bonus. This graphic is just a graphic and does not indicate that you are "close to winning".

Early Packing Tips

I will do a full episode around packing tips when we get closer to re:Invent. But I wanted to add some small short advice already now, mainly related to Vegas and not the event itself.

Shoes
Great walking shoes, trust me, you will be walking. On average, I walk between 25-30K steps per day. Also, bring more than one pair.

Chapstick and Moisturizer
Vegas is in the desert, trust me! Your lips and skin will thank you!

Earplugs
Hotels and Casinos are loud. RePlay is loud. Your ears will thank you!

Travel Adapter and Power Bank
If you live outside of the US, don't forget your travel adapter! For all attendees, a power bank is good to have, to keep your phone charged all day.

Getting Around

First of all, don't underestimate the distance between hotels. They might look close, but the time it takes to get from one hotel to another can be very long. Also, just walking from the entrance of the hotel to your room can be an easy 15 minutes; these hotels are huge.

The transportation options you have are:

Walking

I like walking! I tend to walk everywhere. Often this can be the quickest route, depending on traffic. BUT!! Distance, don't underestimate it. Walking from Venetian to Mandalay Bay is an easy one-hour walk. So, before opting for this, make sure to check Google / Apple maps to see how long it will take.

Free Trams

There are some free trams running between certain hotels on the strip, for example, between Bellagio and MGM Park, or between Excalibur and Mandalay Bay.

Monorail

There is a monorail that runs between MGM Grand and Sahara; you can buy single ride, day, and week passes. During re:Invent, AWS often offers free rides between MGM Grand and The Linq station during certain hours.

Final Words

This was the first episode in "How to re:Invent." Stay tuned for more episodes on how to plan sessions, reserved seating, re:Invent in general, and final packing advice!

See you all in Vegas! Send me a message on LinkedIn if you like to meet up!

Don't forget to follow me on LinkedIn and X for more content, and read the rest of my Blogs.

As Werner says! Now Go Build!

Extending My Blog with Translations by Amazon Nova

Jimmy Dahlqvist — Thu, 17 Jul 2025 18:20:31 +0000

Are you reading this post in English? Or maybe Spanish? Or why not Italian?

In my previous post about automating tasks for my blog, I added automated proofreading using Amazon Nova. While writing in English allows me to connect with a large technical audience, there's an even larger global community that could benefit from my content if it were available in their native languages.

I've previously extended my blog with voice generation using Amazon Polly. I did that so people struggling with reading, such as those with dyslexia, could learn from my content without struggling to read, and instead listen to it.

This time, I'm breaking down language barriers and making my technical content accessible to developers and cloud enthusiasts around the world through automated translation using Amazon Nova.

Vision

The technical community is incredibly diverse, and while English is mostly used in the tech world, far from everyone is a native English speaker. I'm not, and far from everyone is 100% comfortable consuming content in English. They prefer to read it in their native language. Complex technical concepts can be challenging enough without the additional barrier of language comprehension.

In my post Serverless statistics solution with Lambda@Edge, I explain how I built a serverless analytics system for my blog, where I could see where my readers are from, and more.

Looking at my analytics, I can see a lot of readers from Germany, Austria, France, Italy, Spain (and other Spanish-speaking countries), and Brazil and Portugal. I therefore wanted to create a solution where people could choose if they like to read in English or in a different language.

For this first implementation of a translation service, I chose to support five languages that represent a large portion of my readers: German, Spanish, French, Italian, and Portuguese. I might add additional languages such as Japanese in a second phase.

What was important is that the solution would:

Maintain technical accuracy: Technical terms and AWS service names need to be translated correctly.

Preserve markdown structure: All formatting, code blocks, links, and images must remain intact.

Keep my writing style: The translation should feel natural while maintaining the technical depth.

Scale efficiently: Support multiple languages without multiplying my workload.

Integrate seamlessly: Fit into my existing serverless and event-driven pipeline.

Amazon Nova for Translation

Having successfully used Amazon Nova Pro for proofreading, I was sure it would be able to handle my translation tasks as well. Nova Pro's multimodal capabilities and understanding of context make it well-suited for translating technical content. You can read more about Nova, different models, and BedRock in my post automated proofreading using Amazon Nova.

Solution Architecture

Building on my existing event-driven architecture, I extended the pipeline to include the translation. When I created the pipeline, I built it in a very modular way, so it could easily be extended. This approach really pays off now as I can plug in extensions seamlessly into the same flow with very little effort.

The translations run in parallel with the generation of voice and quiz in case of running a pull request. However, I decided to separate it from the standard saga flow that I use for the voice and quiz. The reason for that is that I would like to be able to run the translations not only on pull requests, but also invoke translations of older posts.

So in case of the pull request context, the flow starts when proofreading is done, by an event from GitHub Actions. But I can also invoke and start the flow by sending an event with a branch name and the blog to translate, to the information service that then starts the entire flow. One addition in that case is that in one last step, I will also open a new pull request with the translations for the old post.

The translations are stored in language-specific directories (e.g., /de/, /es/, /fr/), which enables me to organize everything during the build process. I use 11ty to generate static HTML from my posts written in markdown.

Technical Deep Dive

The translation is implemented as its own service and is based on Step Functions as the main service and orchestrator. The translation starts either with me creating a new branch and starting the flow for an older post, or it's automatically started when a new pull request is opened, for a new post. Creating a translation flow like this.

Translation Step Function

The core of the translation service is this Step Function, which orchestrates the entire flow. If you have followed me for a while, you know that I normally try to use Step Functions SDK integrations as much as possible, but for the translation flow, I need to rely heavily on my second favorite service, Lambda.

The workflow consists of several key states:

CollectInfo: Normalizes input data and prepares for translations, reads languages from the event or uses default.
CheckoutMarkdownFile: Downloads the source markdown file from GitHub.
Iterate Languages: Runs translations for the target languages, limited to 1 parallel translation at the moment.
CreateCommit: Commits all translated files back to the repository, opens a new pull request if needed.

Looking at the definition for the Step Function, I'm using the support for JSONata query language, which is one of the best additions to Step Functions together with variables. The CollectInfo state will store information in variables that are available for all future states. Iterating over languages, I'm using a Map state with a limit set to 1.

Comment: Translate post content using Bedrock.
QueryLanguage: JSONata
StartAt: CollectInfo
States:
  CollectInfo:
    Type: Pass
    Assign:
      Languages: "{% $exists($states.input.detail.Data.Translate.Languages) ? $states.input.detail.Data.Translate.Languages : ['de','es','fr','it','pt'] %}"
      GitInfo:
        CommitSha: "{% $exists($states.input.detail.Data.GitInfo.CommitSha) ? $states.input.detail.Data.GitInfo.CommitSha : $states.input.detail.Data.PullRequestInfo.PullRequestCommitSha %}"
        Branch: "{% $exists($states.input.detail.Data.GitInfo.Branch) ? $states.input.detail.Data.GitInfo.Branch : $states.input.detail.Data.PullRequestInfo.PullRequestBranch %}"
      MarkdownInfo:
        FilePath: "{% $states.input.detail.Data.MarkdownFile.path %}"
        FileSlug: "{% $states.input.detail.Data.MarkdownFile.fileSlug %}"
    Next: CheckoutMarkdownFile
  CheckoutMarkdownFile:
    Type: Task
    Resource: arn:aws:states:::lambda:invoke
    Arguments:
      FunctionName: ${CheckoutMarkdownFileFunctionArn}
      Payload:
        GitInfo: "{% $GitInfo %}"
        MarkdownInfo: "{% $MarkdownInfo %}"
    Next: Iterate Languages
  Iterate Languages:
    Type: Map
    ItemProcessor:
      ProcessorConfig:
        Mode: INLINE
      StartAt: TranslateMarkdownFile
      States:
        TranslateMarkdownFile:
          Type: Task
          Resource: arn:aws:states:::lambda:invoke
          Arguments:
            FunctionName: ${TranslateBedrockFunctionArn}
            Payload:
              Key: "{%$MarkdownInfo.FileSlug & '/' & $MarkdownInfo.FilePath %}"
              S3Bucket: ${ETLBucket}
              Language: "{% $states.input %}"
          End: true
    Next: CreateCommit
    MaxConcurrency: 1
    Items: "{% $Languages %}"
  CreateCommit:
    Type: Task
    Resource: arn:aws:states:::lambda:invoke
    Arguments:
      FunctionName: ${CreateCommitFunctionArn}
      Payload:
        GitInfo: "{% $GitInfo %}"
        MarkdownInfo: "{% $MarkdownInfo %}"
    End: true

Lambda Functions

Looking at the Lambda functions, there are three main functions in play in the solution. The function that will use Octokit to checkout the code, and the function that will create the new commit. Both of these have been explained in previous posts on this topic, and are left out this time.

Translation Function

This is the heart and soul of the entire solution and will interact with Bedrock to translate the post. The translated post will be stored in a language-specific directory.

import json
import os
import boto3
from botocore.exceptions import ClientError

def handler(event, context):
    try:
        bucket = event.get("S3Bucket")
        key = event.get("Key")
        lang = event.get("Language", "error")

        if lang not in ["de", "es", "fr", "it", "pt"]:
            raise ValueError(
                f"Unsupported language '{lang}'. Supported languages are: de, es, fr, it, pt."
            )

        s3_client = boto3.client("s3")
        bedrock_client = boto3.client("bedrock-runtime", region_name="eu-west-1")

        response = s3_client.get_object(Bucket=bucket, Key=key)
        file_content = response["Body"].read().decode("utf-8")

        prompt = f"""You are a professional technical editor and translator, expert in AWS and Cloud computing. Please carefully translate the following markdown blog post to {lang}. 
        Instructions:
        - Maintain the original markdown formatting
        - Keep the tone and style consistent
        - Do not change the meaning or structure of the content
        - Return only the translated markdown text without any additional commentary
        - Do not translate any code blocks or inline code
        - Ensure proper grammar, spelling, and punctuation
        - If the content is already in {lang}, just ignore it and return the content as is
        - Do not surround the translated version with ```
{% endraw %}
markdown

        Here is the markdown content to translate:

        {file_content}"""

        # Prepare the request for Nova Pro model
        request_body = {
            "messages": [{"role": "user", "content": [{"text": prompt}]}],
            "inferenceConfig": {
                "temperature": 0.1,  # Low temperature for consistent translations
                "topP": 0.9,
                "maxTokens": 10240,  # Adjust based on expected output length
            },
        }

        bedrock_response = bedrock_client.invoke_model(
            modelId="eu.amazon.nova-pro-v1:0",
            body=json.dumps(request_body),
            contentType="application/json",
        )

        response_body = json.loads(bedrock_response["body"].read())
        translated_content = response_body["output"]["message"]["content"][0]["text"]

        new_key = f"{lang}/{key}"

        s3_client.put_object(
            Bucket=bucket,
            Key=new_key,
            Body=translated_content.encode("utf-8"),
            ContentType="text/markdown",
        )

        return {
            "statusCode": 200,
            "body": "Translation done",
        }

    except Exception as e:
        print(f"Unexpected error: {str(e)}")
        return {
            "statusCode": 500,
            "body": json.dumps({"error": "Internal Server Error", "message": str(e)}),
        }
{% raw %}

Translation Quality and Prompt Engineering

During the implementation of the proofreading solution, I learned how important the prompt engineering part was for the end result. The AI still does just what you tell it to do, and if your prompt is not specific enough, the quality of the result can vary. Using the learnings from proofreading, my prompt included some specific requirements.

In the end, the following prompt gave me some great results:


text
You are a professional technical editor and translator, expert in AWS and Cloud computing. 
Please carefully translate the following markdown blog post to {lang}. 
Instructions:
- Maintain the original markdown formatting
- Keep the tone and style consistent
- Do not change the meaning or structure of the content
- Return only the translated markdown text without any additional commentary
- Do not translate any code blocks or inline code
- Ensure proper grammar, spelling, and punctuation
- If the content is already in {lang}, just ignore it and return the content as is
- Do not surround the translated version with

 ```markdown

Here is the markdown content to translate:

11ty Changes

I'm using 11ty as the static HTML generator, all blogs are written in markdown and then converted to HTML. To be able to support different languages, I needed to make some changes to my 11ty build and layout. First of all, I added a new field to the front matter that indicates what language the post is in. Then I needed to change how my post collection was fetching blog posts.

I added a filter so I only fetch the posts in English so I don't get all posts in the collection.

export default (coll) => {
  const posts = coll
    .getFilteredByGlob("src/posts/*.md")
    .filter((p) => (p.data.lang || "en") === "en");

  return posts.reverse();
};

Next, I needed to locate all translations for a post. I used the file slug as the key to create this collection.

config.addCollection("postsBySlug", (collection) => {
  const posts = collection.getFilteredByGlob("src/posts/**/*.md");
  const postsBySlug = {};

  posts.forEach((post) => {
    const pathParts = post.inputPath.split("/");
    const filename = pathParts[pathParts.length - 1]; 
    const baseSlug = filename.replace(".md", "");

    if (!postsBySlug[baseSlug]) {
      postsBySlug[baseSlug] = [];
    }
    postsBySlug[baseSlug].push(post);
  });

  return postsBySlug;
});

With this in place, I could create and include a language indicator that would add flags indicating translations to the post grid. Where each flag is clickable and would take you directly to the translated post.

{% set item = post or talk %}
{% set pathParts = item.inputPath.split('/') %}
{% set filename = pathParts[pathParts.length - 1] %}
{% set baseSlug = filename.replace('.md', '') %}
{% set currentLang = item.data.lang or 'en' %}

{# Only show language indicator for posts (not talks or links) #}
{% if item.inputPath.includes('/posts/') %}
{% set availablePosts = collections.postsBySlug[baseSlug] %}
{% if availablePosts.length > 1 %}
<div class="mx-auto flex items-center flex-wrap">
    <div class="justify-start items-center flex-grow w-full md:w-auto md:flex">
        <p class="text-gray-700 mb-1 font-medium" title="Available languages">
            <i class="fas fa-globe text-teal-600 fill-current"></i> 
            {% for translatedPost in availablePosts %}
                {% set postLang = translatedPost.data.lang or 'en' %}
                <a href="{{ translatedPost.url }}" class="text-sm ml-1 hover:scale-110 transition-transform duration-200 inline-block" title="{{ postLang | getLanguageName }}">{{ postLang | getLanguageFlag }}</a>
            {% endfor %}
        </p>
    </div>
</div>
{% endif %}
{% endif %}

Resulting in:

Finally, I added a drop-down menu to the actual post layout together with a disclaimer that the post was translated by an AI.

With these changes, you can select what language you like to read each post in.

Conclusion

Adding automated translation to my blog felt like the next step in making my content available for everyone. By leveraging Amazon Nova Pro and my existing serverless architecture, I've created a scalable translation solution without adding operational complexity. Whether you're building a technical blog, documentation, or any content platform, automated translation could expand your reach.

Why didn't I translate the entire site? This was a route I first explored, but I felt that the main benefit would be in having the posts translated. It also gives me the freedom to decide if I like to translate all older posts or not, and I can add translations afterward. One day I might translate everything, but right now the focus was on posts individually.

Final Words

Don't forget to follow me on LinkedIn and for more content, read the rest of my Blogs.

As Werner says! Now Go Build!

This post was originally written in English and automatically translated into multiple languages using the very system described here. The translation process took less than 5 minutes and required zero manual intervention.

Extending My Blog with Proofreading by Amazon Nova

Jimmy Dahlqvist — Sun, 06 Jul 2025 12:47:13 +0000

Blogging has over the years really become something I enjoy doing. Often, I build something and then write a post about the solution and what I learned. More than once, it has been about serverless and event-driven architecture. However, there is one thing I don't like: as a non-native English speaker, I tend to make both spelling and grammatical errors, and even if I proofread several times, I always miss something.

I've previously extended my blog with voice generation using Amazon Polly and gamified learning with automated quizzes.

This time, it's time for one more addition to this, which will also help me and save time. That is an automated proofreading service using Amazon Nova.

My Problem

When I write my blog posts, I focus on the solution and the architecture, presenting and showcasing that in a good way. I have much less focus on spelling, even though I do have a spelling extension installed in VSCode. Then, proofreading my posts takes a lot of time, and to be honest, I'm not always that thorough. So, what I needed was an automated solution that could:

Maintain my writing style: I don't want the proofreading to change my voice or technical explanations.
Preserve markdown formatting: My blog posts are written in markdown with code blocks, links, and images.
Integrate seamlessly: The solution should fit into my existing solution with voice and quiz generation.
Be cost-effective: Run only on certain Pull-requests, and be serverless and event-driven just like the voice and quiz solution.

Introducing Amazon Nova

Amazon Nova is Amazon's own LLMs that were announced at re:Invent 2024. Before Nova, there were several Titan models, and to be honest, Titan wasn't that good. Nova, on the other hand, is really good and is a good, cost-efficient option. Nova models are available through Amazon Bedrock, like many other models, which provides an API for accessing various large language models.

What is Amazon Bedrock?

Amazon Bedrock is a fully managed service that makes access to a lot of foundation models easy. They are all available through a single API and it allows me to build great generative AI solutions in an easy way. Bedrock offers us:

Model choice: Access to models from Amazon, Anthropic, Meta, and several more, with new models constantly being added.
Easy integration: Serverless and managed, with simple API calls without managing infrastructure.
Fine-tuning capabilities: Customize models with your own data.
Security and data privacy: Our data is not used to train any models.
Bedrock Agents: Agents that can be used to build applications that combine LLMs and APIs to create powerful solutions.

Amazon Nova Models

Amazon Nova offers, at this moment, four different understanding/reasoning models, 2 creative models, and 1 speech model.

The understanding models can handle different inputs like text, images, video, documents, and code. They all output the response in text. These models are:

Nova Micro: Text only, low latency and fast responses with a low cost.
Nova Lite: Low cost multimodal model that is super fast processing text, images, and video.
Nova Pro: Very capable multimodal model, for a good price, that offers a great balance of accuracy, speed, and cost for a wide range of tasks.
Nova Premier: The most capable multimodal model for complex tasks, not available in that many regions.

I have selected Nova Pro for my task as, for proofreading blog posts, it provides a great balance of accuracy and cost-effectiveness. It handles understanding context, maintaining writing style, and making small corrections without over-editing my content.

The creative models are:

Amazon Nova Canvas: A high-quality image generation model.
Amazon Nova Reel: A video generation model.

Amazon Nova Canvas and Amazon Nova Reel are, right now, not available in that many regions. You have to use us-east-1 (N. Virginia), eu-west-1 (Ireland), or ap-northeast-1 (Tokyo).

The speech model is:

Amazon Nova Sonic: That delivers real-time, human-like voice conversations.

Amazon Nova Sonic is, right now, available in us-east-1 (N. Virginia), eu-north-1 (Stockholm), and ap-northeast-1 (Tokyo) and supports English (US, UK) and Spanish.

Not having models available in all regions or even the same regions can become a problem when building our applications.

Cross-Region Inference

One thing we need to bring up is the cross-region inference setup in Bedrock, which is done via inference models. What this does is that our calls automatically get routed to the region with the most available resources, ensuring that our calls can be fulfilled. When using the Nova models outside of us-east-1 (N. Virginia) and AWS GovCloud (US-West), we need to use an inference profile when calling the Bedrock API. If we don't, we will get hit by an error saying the model is not available in the region in a serverless way.

As an example, if we call the eu.amazon.nova-pro-v1:0 from eu-west-1, where my solution runs, our inference call will stay within the EU region and inference can be done from one of the EU regions: eu-central-1, eu-north-1, eu-west-1, eu-west-3.

Solution Architecture

In the post gamified learning with automated quizzes, I introduce the architecture. It's built on Amazon EventBridge, divided into multiple services, and runs as a Saga Pattern.

As explained in the post above, I perform two major things: I generate the quiz for the gamified learning and I use Polly to generate voice that reads my blog post. This is a mix of GitHub actions that will build the page and upload to a staging bucket in S3. After upload to the staging bucket is complete, GitHub actions will post an event onto an EventBridge event-bus and this is where my AWS-based part takes over.

The AWS-based CI/CD pipeline is event-driven and serverless. The primary services used are StepFunctions, Lambda, and EventBridge. The flow is based on a saga pattern where the domain services hand over by posting domain events on the event-bus, which will move the saga to the next step.

As my voice generation is done from the rendered HTML files, for most accuracy, I just couldn't plug in the proofreading as a first step in this Saga Pattern. The solution became to add GitHub actions into the saga. One day I might move away from GitHub Actions to something else, but as of now, it will be part of the entire solution.

So, I will add the following part, that now runs when my Pull-Request is opened:

In this part, I fetch information from GitHub, and call Amazon Nova via Amazon Bedrock to do my proofreading. When completed, the service adds a new commit to the PR with the updated markdown file.

Running the Voice and Quiz Parts

Now, I need a way to integrate and start the Voice and Quiz steps, that I before was running when the PR was created. Instead, I now have a GitHub Action that will detect the proofreading commit and will start that flow. Giving a solution with two distinct parts.

I have now created two distinct parts in the flow. All major logic is running on AWS and GitHub Action builds the staging blog and invokes the different parts, by sending different events to EventBridge.

Technical Deep Dive

Let's take a technical deep dive into the solution and the actual proofreading service.

StepFunction

The heart of everything is an orchestrator implemented with StepFunctions. It will check out the files, perform proofreading, and create the new commit.

The work is basically carried out by three Lambda functions. The core Lambda function is the one performing the actual proofreading. The functions checking out the PR and creating a new commit are both using Octokit. To see how these work, check my previous posts. I will focus on the actual proofreader.

Proofreading Lambda Function

The function will initialize everything, create the prompt, and call Nova and Bedrock, and store the proofread version in S3. The original post is also kept and the file is just renamed.

import json
import os
import boto3
from botocore.exceptions import ClientError

def handler(event, context):
    print("Received event:", json.dumps(event, indent=2))

    try:
        bucket = event.get("S3Bucket")
        key = event.get("Key")

        if not bucket or not key:
            raise ValueError("Both 'bucket' and 'key' must be provided in the event")

        s3_client = boto3.client("s3")
        bedrock_client = boto3.client("bedrock-runtime", region_name="eu-west-1")

        response = s3_client.get_object(Bucket=bucket, Key=key)
        file_content = response["Body"].read().decode("utf-8")

        prompt = f"""You are a professional technical editor and proofreader, expert in AWS and Cloud computing. Please carefully proofread the following markdown blog post for spelling and grammatical errors. 
        Instructions:
        - Fix any spelling mistakes
        - Correct grammatical errors
        - Maintain the original markdown formatting
        - Keep the tone and style consistent
        - Do not change the meaning or structure of the content
        - Return only the corrected markdown text without any additional commentary
        - Do not surround the proofread version with ```
{% endraw %}
markdown

        Here is the markdown content to proofread:
        {file_content}"""

        request_body = {
            "messages": [{"role": "user", "content": [{"text": prompt}]}],
            "inferenceConfig": {
                "temperature": 0.1,
                "topP": 0.9,
                "maxTokens": 10240,
            },
        }

        bedrock_response = bedrock_client.invoke_model(
            modelId="eu.amazon.nova-pro-v1:0",
            body=json.dumps(request_body),
            contentType="application/json",
        )

        response_body = json.loads(bedrock_response["body"].read())
        proofread_content = response_body["output"]["message"]["content"][0]["text"]

        if key.endswith(".md"):
            base_name = key[:-3]
        else:
            base_name = key

        original_backup_key = f"{base_name}-original.md"

        s3_client.copy_object(
            Bucket=bucket,
            CopySource={"Bucket": bucket, "Key": key},
            Key=original_backup_key,
        )

        s3_client.put_object(
            Bucket=bucket,
            Key=key,
            Body=proofread_content.encode("utf-8"),
            ContentType="text/markdown",
        )

        return {
            "statusCode": 200,
            "body": json.dumps(
                {
                    "message": "Proofreading completed successfully",
                    "proofread_file": f"s3://{bucket}/{key}",
                    "backup_file": f"s3://{bucket}/{original_backup_key}",
                    "original_length": len(file_content),
                    "proofread_length": len(proofread_content),
                }
            ),
        }

    except ClientError as e:
        error_code = e.response["Error"]["Code"]
        error_message = e.response["Error"]["Message"]
        print(f"AWS Client Error ({error_code}): {error_message}")

        return {
            "statusCode": 500,
            "body": json.dumps(
                {
                    "error": "AWS Client Error",
                    "error_code": error_code,
                    "message": error_message,
                }
            ),
        }

    except ValueError as e:
        print(f"Validation Error: {str(e)}")
        return {
            "statusCode": 400,
            "body": json.dumps({"error": "Validation Error", "message": str(e)}),
        }

    except Exception as e:
        print(f"Unexpected error: {str(e)}")
        return {
            "statusCode": 500,
            "body": json.dumps({"error": "Internal Server Error", "message": str(e)}),
        }
{% raw %}

Prompt Engineering

One important part that should not be forgotten is prompt engineering. The prompt needs to be clear and include detailed instructions on how the LLM should act. I needed to iterate the prompt several times to get the result I needed. My prompt includes specific instructions to:

Fix spelling and grammatical errors
Maintain the original markdown formatting
Preserve technical terminology
Keep the writing style and tone consistent
Return only the corrected content without commentary

In the end, the following prompt gave the result I wanted.


text
You are a professional technical editor and proofreader, expert in AWS and Cloud computing. Please carefully proofread the following markdown blog post for spelling and grammatical errors. 
Instructions:
- Fix any spelling mistakes
- Correct grammatical errors
- Maintain the original markdown formatting
- Keep the tone and style consistent
- Do not change the meaning or structure of the content
- Return only the corrected markdown text without any additional commentary
- Do not surround the proofread version with

 ```markdown

Here is the markdown content to proofread:

Starting Voice and Quiz Part

The second part of the flow, generating voice and quiz, runs when a new commit with the commit message "Proofread by Amazon Nova" is added to the pull-request. The action starts when the synchronize event occurs. The first part checks the commit and gets the latest commit message. The steps after that will then check the message.

name: Start Polly Voice and Quiz Flow

on:
  pull_request:
    types: [synchronize]
    paths:
      - "**/posts/*.md"

jobs:
  check-commit:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: ./
    outputs:
      message: ${{ steps.commit-msg.outputs.message }}
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
          ref: ${{ github.event.pull_request.head.sha }}

      - name: Get last commit message
        id: commit-msg
        run: |
          message=$(git log -1 --pretty=format:"%s" HEAD)
          echo "Last commit message: ${message}"
          echo "message=${message}" >> $GITHUB_OUTPUT

  build:
    needs: check-commit
    if: contains(needs.check-commit.outputs.message, 'Proofread by Amazon Nova')
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install and Build
        run: |
          npm ci
          npm run build

  deploy:
    needs: build
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    steps:
      - name: Upload build blog to S3
        working-directory: ./
        run: |
          aws s3 sync --delete public/. s3://<My-Staging-Bucket>
          aws events put-events --entries '....'

Conclusion

Extending my blog with automated proofreading using Amazon Nova has saved me a lot of time. I now do a quick proofreading myself, then delegate the big part to Amazon Nova. The extension fits great into the event-driven design I already had in place.

So, with Bedrock and Amazon Nova, together with a serverless approach, I created and extended my solution that is:

Reliable: Consistent proofreading quality for every post
Cost-effective: Pay-per-use model with minimal operational costs

Most importantly, automated proofreading allows me to focus on what I enjoy the most, creating technical content that helps others learn about cloud technologies.

Going forward, I have several other ideas that will enhance my blog using Amazon Nova and AI. Stay tuned for more fun solutions!

Final Words

Don't forget to follow me on LinkedIn and for more content, read the rest of my Blogs.

As Werner says! Now Go Build!

PEP and PDP for Secure Authorization with AVP and ABAC

Jimmy Dahlqvist — Wed, 07 May 2025 09:21:16 +0000

In the first part, PEP and PDP for Secure Authorization with Cognito, I introduced the concept of Policy Enforcement Points (PEPs) and Policy Decision Points (PDPs) using Lambda Authorizer in API Gateway as PEP and a Lambda based service as PDP. The authorization decision was based on the roles users held, represented by Cognito groups, with the permissions mapped in a DynamoDB table.

In the second part, PEP and PDP for Secure Authorization with AVP, we replaced the DynamoDB permission mapping with Amazon Verified Permissions (AVP) and implemented a Role-Based Access Control (RBAC) system using Cedar policies. This gave us a much more powerful way to handle authorization, especially for multi-tenant SaaS applications.

Now, in this third part, we'll take our authorization system to the next level by adding Attribute-Based Access Control (ABAC) alongside RBAC using AVP. This combination provides even more dynamic and context-aware authorization decisions, which is exactly what we need for complex multi-tenant applications.

Understanding Attribute-Based Access Control (ABAC)

So, what exactly is ABAC? At its core, ABAC is an authorization strategy that makes decisions based on attributes associated with users, resources, actions, and the environment. Unlike RBAC, which grants permissions solely based on roles, ABAC evaluates various attributes to make more fine grained authorization decisions.

Think of it this way - RBAC is like having different keys (roles) that open different doors (resources). ABAC, on the other hand, is like having a smart lock that considers not just who you are, but also where you're coming from, what time it is, what you're trying to do, and even what's going on around you before deciding whether to let you in.

As an example some common attributes used in ABAC could be:

User attributes: Department, location, clearance level, seniority

Resource attributes: Classification, owner, sensitivity level

Action attributes: Time of day, encryption status, access method

Environmental attributes: Device type, network location, security level

For example, an ABAC policy might state that Data scientists can access sensitive data, but only from corporate networks, during business hours, and if they have completed security training in the last 6 months.

Why Combine RBAC and ABAC?

But if ABAC is so powerful, why don't we just use it instead of RBAC? I would say that RBAC is really simple to understand and implement, which makes it great for basic authorization needs. However, when permissions need to be more granular or context-dependent, it can become very hard and complex to manage.

On the other hand, ABAC offers amazing flexibility but it can be very complex to implement from scratch and harder to reason about. By combining both approaches, we get the best of both worlds!

In our unicorn racing example, we can use roles to define broad access patterns (Admin, Trainer, Rider), then we'll use an attribute dataAccess to restrict access to specific datasets within those roles.

Implementing ABAC in Amazon Verified Permissions

AVP with its Cedar policy language is perfectly suited for implementing ABAC. Cedar policies can evaluate conditions based on attributes from the principal (user), resource, action, and context.

Token Mapping in AVP - A Critical Concept

An important part of implementing ABAC is understanding how token claims map to your Cedar policy schema. AVP handles this differently depending on whether you're using ID tokens or Access tokens.

ID Token: Claims are mapped to attributes of the principal entity representing the user

Access Token: Claims are mapped to the context.token object in policy evaluation

This difference is super important when writing Cedar policies because it affects how you access attributes in your policy conditions!

For example, with an ID token a policy would look something like this

permit (principal, action, resource)
when { principal.department == "Engineering" };

But with an access token, we need to use the context object and not the principal

permit (principal, action, resource)
when { context.token["department"] == "Engineering" };

In our implementation, we'll use the access token approach since I think it provides a cleaner separation between identity (who the user is) and access permissions and attributes (what they can do and what properties they have).

Why Access Tokens for Authorization

While ID tokens are primarily meant for authentication (proving identity), access tokens are designed specifically for authorization (determining permissions). I see several advantages using access tokens:

Separation of concerns: Authentication details remain in ID tokens, while authorization details live in access tokens

Reduced token size: ID tokens won't be bloated with authorization attributes

Independent lifetimes: Access token expiration can be shorter than ID token expiration

Easier token revocation: You can revoke access without affecting authentication

In our AVP-based PDP, we'll use access tokens and map custom claims to the context for better authorization decisions.

Implementing AVP with RBAC and ABAC

Now that we understand the concepts, let's get our hands dirty and implement our enhanced authorization system!

Architecture

Our architecture remains similar but with a few changes. Where we add a new Lambda function to be able to enrich our access token with custom claims.

This will give us an updated call flow like this.

The changes we need to do would be.

First of all we need to add dataAccess as a custom claim in our access token, to do that we must configure a Pre-token Generation Lambda function, that will be invoked by Cognito.

Second we need to update our schema and add the dataAccess attribute on each action we like the policy to be able to evaluate according to ABAC.

Last we need to update our Cedar policies to be context aware and evaluate both role (RBAC) and attributes (ABAC)

Adding Custom Attributes to Access Tokens

To implement ABAC, we need a way to add custom attributes to our access tokens. We'll use Cognito's Pre-Token Generation Lambda trigger for this purpose, and we need to use version 2 or 3 of the event. With version 1 we can only modify the ID Token.

Below is a implementation that adds the dataAccess attribute from user attributes to the access token:

def handler(event, context):
    user_attributes = event["request"]["userAttributes"]
    claims_to_add_to_access_token = {}

    if "custom:dataAccess" in user_attributes:
        data_access_values = [
            value.strip() for value in user_attributes["custom:dataAccess"].split(",")
        ]
        claims_to_add_to_access_token["custom:dataAccess"] = data_access_values

    response = {
        "claimsAndScopeOverrideDetails": {
            "accessTokenGeneration": {
                "claimsToAddOrOverride": claims_to_add_to_access_token,
            }
        }
    }

    event["response"] = response
    return event

This function checks if the user has a custom:dataAccess attribute. If found, it splits the comma-separated values into an array and adds them to the access token. This way, our AVP policies can check if the user has access to specific data categories.

To have Cognito call our function we need to configure the Lambda Triggers of the User Pool.

  PreTokenGenerationFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/PreTokenGeneration
      Handler: index.handler
      Runtime: python3.12
      Architectures:
        - x86_64
      MemorySize: 128
      Description: A Lambda function that adds custom attributes to the JWT access token
      Policies:
        - AWSLambdaBasicExecutionRole

  UserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UsernameConfiguration:
        CaseSensitive: false
      AutoVerifiedAttributes:
        - email
      UserPoolName: !Sub ${ApplicationName}-user-pool
      Schema:
        - Name: email
          AttributeDataType: String
          Mutable: false
          Required: true
        - Name: name
          AttributeDataType: String
          Mutable: true
          Required: true
        - Name: dataAccess
          AttributeDataType: String
          Mutable: true
          Required: false
      LambdaConfig:
        PreTokenGenerationConfig:
          LambdaArn: !GetAtt PreTokenGenerationFunction.Arn
          LambdaVersion: V2_0

Cedar Policy Schema with Context Attributes

Our Cedar schema defines the structure of principals, resources, actions, and their relationships. For ABAC, we need to update the schema to include context attributes:

{
  "${AVPNameSpace}": {
    "actions": {
      "get /rider": {
        "appliesTo": {
          "context": {
            "type": "Record",
            "attributes": {
              "dataAccess": {
                "type": "String",
                "required": true
              }
            }
          },
          "principalTypes": ["CognitoUser"],
          "resourceTypes": ["Application"]
        }
      }
      // Other actions...
    }
  }
}

This schema enables us to use the dataAccess attribute in our authorization policies.

Policies that Combine RBAC and ABAC

Our Cedar policies now combine RBAC (role-based) and ABAC (attribute-based) logic:

permit(
  principal in ${AVPNameSpace}::CognitoUserGroup::"${UserPoolId}|Admin",
  action in [
    ${AVPNameSpace}::Action::"get /rider",
    ${AVPNameSpace}::Action::"get /riders",
    ${AVPNameSpace}::Action::"get /trainer",
    // more actions...
  ],
  resource
)
when {
  // ABAC condition using attributes
  context.dataAccess == "" ||
  (context.token has "custom:dataAccess" &&
   context.token["custom:dataAccess"].contains(context.dataAccess))
};

This policy allows users in the Admin group to access endpoints (RBAC part), but only if one of these conditions is true (ABAC part):

No specific data access check is needed (empty context.dataAccess), OR
The user has the required data access attribute in their token

Let's break down the condition a bit for better understanding.

If context.dataAccess is empty, we don't enforce data access restrictions
Otherwise, we check if the user's token has the custom:dataAccess claim that contains the required value

Context-Aware Authorization Logic

Our PDP Lambda function now handles context-based authorization. It extracts relevant attributes from requests and includes them in the AVP authorization call. The below function has been simplified and some parts has been removed, check the source code for a full version.

def validate_permission(event):
    jwt_token = event["jwt_token"]
    resource = event["resource"]
    action = event["action"]
    resource_tags = {}

    if "resource_tags" in event:
        resource_tags = event["resource_tags"]

    parsed_token = decode_token(jwt_token)

    try:
        action_id = f"{action.lower()} {resource.lower()}"
        user_principal = parsed_token["sub"]

        # Call AVP with context for the authorization decision
        context = {
            "dataAccess": resource_tags.get("Data", "")
        }

        auth_response = avp_client.is_authorized_with_token(
            accessToken=jwt_token,
            policyStoreId=policy_store_id,
            action={"actionType": action_type, "actionId": action_id},
            resource={"entityType": resource_type, "entityId": resource_id},
            context=context
        )

        # Cache the result with the context hash
        store_auth_cache(user_principal, action_id, context_hash, auth_response)

        # Generate the response
        response_body = generate_access(
            user_principal,
            "Allow" if auth_response["decision"].upper() == "ALLOW" else "Deny",
            action,
            resource
        )

        return {
            "statusCode": 200,
            "body": json.dumps(response_body),
            "headers": {"Content-Type": "application/json"},
        }

    except Exception as e:
        print(f"Error validating permissions: {e}")

    response_body = generate_access(parsed_token["sub"], "Deny", action, resource)

    return {
        "statusCode": 200,
        "body": json.dumps(response_body),
        "headers": {"Content-Type": "application/json"},
    }

With this implementation, our PDP now evaluates both role-based rules (RBAC) and attribute-based conditions (ABAC).

Context-Aware Caching

One of the challenges with adding ABAC is that authorization decisions now depend not just on who you are and what you're trying to do, but also on context. This means our caching strategy needs to be updated.

We'll include context information in our cache key by creating a hash of the context attributes:

def generate_context_hash(context_dict):
    sorted_items = sorted(context_dict.items())
    context_str = json.dumps(sorted_items)
    return hashlib.md5(context_str.encode()).hexdigest()

def get_auth_cache(principal, action, context_hash):
    try:
        response = dynamodb_client.get_item(
            TableName=table_name, 
            Key={
                "PK": {"S": principal}, 
                "SK": {"S": f"{action}#{context_hash}"}
            }
        )

        item = response.get("Item")
        if not item:
            return None

        ttl = int(item.get("TTL")["N"])
        if ttl and ttl < int(time.time() * 1000):  # TTL is in milliseconds
            return None

        return item.get("Effect")["S"]

    except Exception as e:
        print(f"Error getting auth cache: {e}")
        return None

def store_auth_cache(principal, action, context_hash, auth_response):
    try:
        ttl = int((datetime.now(timezone.utc) + timedelta(hours=12)).timestamp() * 1000)

        effect = "Allow" if auth_response.get("decision", "").upper() == "ALLOW" else "Deny"

        dynamodb_client.put_item(
            TableName=table_name,
            Item={
                "PK": {"S": principal},
                "SK": {"S": f"{action}#{context_hash}"},
                "TTL": {"N": str(ttl)},
                "Effect": {"S": effect},
            },
        )

    except Exception as e:
        print(f"Error storing auth cache: {e}")

By including a hash of the context in our cache key, we ensure that authorization decisions are correctly cached and retrieved based on both the user's role and the relevant context attributes.

Modifying the PEP to Pass Context

Our PEP (Lambda Authorizer) also needs a small update to pass context information to the PDP, this is a very basic implementation, and in a production system you would for sure have something more sophisticated, like looking at resources Tags on AWS resources.

def get_resource_tags_for_path(path):
    if (
        path.startswith("/unicorn")
        or path.startswith("/rider")
        or path.startswith("/trainer")
    ):
        return {"Data": "Unicorn"}
    elif path.startswith("/race"):
        return {"Data": "Races"}
    else:
        return {}

# Inside the handler function
resource_tags = get_resource_tags_for_path(path)

data = {
    "jwt_token": token,
    "resource": path,
    "action": method,
    "resource_tags": resource_tags,
}

This function determines what data category a resource belongs to based on its path, and passes that information to the PDP for context-aware authorization.

Testing the ABAC Implementation

Now that we have our implementation in place, it's time to test it! Here's how we can test our combined RBAC + ABAC solution.

First we need to create or update a user in different Cognito groups (Admin, Trainer, Rider). We also need to assign different dataAccess attributes to these users (e.g., "Unicorn", "Races").

Next navigate to the webpage deployed with the CloudFront distribution and inspect the JWT tokens, cookies.

If we copy the access token and decode that, I use jwt.io, we can see that my user has the claim custom:dataAccess that our PEP and PDP will use later for permissions.

Now we can use a tool like Bruno or Postman to include the access token in the calls to the API. Of course, depending on how you have configured the access you should get either an Allow or Deny back from the PDP resulting in different results from the API. Below is some example views where I use Bruno to call the API with and without access.

Conclusion

By combining RBAC and ABAC with Amazon Verified Permissions, we've created a powerful, flexible authorization system that can handle complex access control requirements. This approach provides the simplicity of role-based access while enabling the fine-grained control of attribute-based decisions.

This implementation approach can be used in an multi-tenant SaaS applications where data segregation, based on tenant, and context-aware access control are critical. With AVP handling the policy evaluation and our serverless architecture for enforcement, we have a scalable, maintainable solution for even the most demanding authorization needs.

Source Code

The entire setup, with detailed deployment instructions, and all the code can be found on Serverless Handbook PEP and PDP

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

PEP and PDP for Secure Authorization with AVP

Jimmy Dahlqvist — Fri, 21 Feb 2025 06:38:53 +0000

When running a PEP and PDP setup in a multi-tenant SaaS solution we for sure need something more powerful than a DynamoDB table with permissions. Therefor in this part we will look at using Amazon Verified Permissions (AVP) as the foundation in our PDP. Which would serve as an great choice to do Authorization in SaaS.

What is Amazon Verified Permissions?

Amazon Verified Permissions (AVP) is a fully managed service that simplifies the implementation of fine-grained access control in our applications. AVP acts as a central Policy Decision Point (PDP), where it evaluates policies and makes authorization decisions based on the attributes of the request, user roles, and similar. It doesn't only support role-based access control (RBAC), but AVP also supports attribute-based access control (ABAC) and other dynamic policies, allowing for more flexible and granular authorization decisions.

With AVP, our Lambda function (acting as the PDP) no longer has to manually parse and evaluate policies, and we don’t need to store permission mapping in a DynamoDB table. Instead, we define and manage our policies directly in AVP, which then makes the authorization decision for us.

Alternatives Open Source solution to AVP

While AVP is an excellent choice for applications specially if they run in AWS, it’s not the only solution for implementing fine-grained authorization. One alternative is Oso, an open-source authorization framework designed to provide flexible, policy-based access control. Oso allows us to define authorization logic in a declarative language and integrates with our applications across various environments. Choosing between AVP and Oso depends on our specific use case, infrastructure etc.

Caching authorization decisions

Now that we have looked at using AVP as a central part of our PDP, we should think about caching. There are pros and cons against caching, for example AWS IAM never cache any decision which ensure that updates to policies are reflected immediately. With a cache we can reduce the number of calls to AVP and by that reduce cost. We can also either cache decisions in the client or in our backend systems. I will not go in to deep in this topic more than that in this solution we will use an external cache in our backend system. The cache will be placed in DynamoDB. The reason to cache in an external source, like DynamoDB, instead of in Lambda function memory, is for all invocations of the Lambda function to benefit from the cache. With a cache like this we would need a way to clear the cache if a permission set for a user is changed. I will not implement any cache invalidation mechanism in this post.

Understanding AVP and the Cedar Policy Language

Amazon Verified Permissions (AVP) uses Cedar, a purpose-built, policy-as-code language designed for fine-grained authorization. Cedar enables us to define and enforce access control policies that dictate who can perform what actions on which resources.

Cedar policies are declarative, meaning they explicitly state the permissions without requiring procedural logic. They are designed to be easy to read while supporting powerful conditions and attribute-based access control (ABAC).

Basic Cedar Policy Example

Let's start with a simple RBAC (Role-Based Access Control) policy that allows users in the "Admin" group to perform certain actions on all resources

permit (
    principal in UserGroup::"Admin",
    action in [
        Action::"create",
        Action::"read",
        Action::"update",
        Action::"delete"
    ],
    resource
);

This policy means that any principal (user) who belongs to the UserGroup Admin is permitted to perform the listed actions on all resources, since resource is not restricted.

A second example, where we lean towards a ABAC method would be.

permit (
    principal in UserGroup::"JimmysFriends",
    action == Action::"viewPhoto",
    resource
)
when {
    resource.tags.contains("Shared")
};

This means that users in JimmysFriends can view photos when the photo has the Shared tag.

In a multi-tenant SaaS system, we might want to restrict access to data belonging to a specific tenant. Cedar allows attribute-based conditions for this.

permit (
    principal,
    action == Action::"view",
    resource
)
when {
    principal.tenant == resource.tenant
};

This ensures that the user can only view resources that belong to the same tenant.

In the implementation for our AVP based PDP we will use RBAC and policy similar to the first example.

Using AVP for scalable and flexible access control

By integrating (AVP) as part of our central PDP, we have simplified our policy management and enhanced the scalability and flexibility of the authorization system. AVP’s support for dynamic policies, and fine-grained access control are powerful tools for any SaaS application.

Implementation

With the introduction to AVP let's convert our PDP to use AVP for Role Based Access (RBAC) instead of a DynamoDB permission mapping. We will just deploy a second PDP into our solution and the swap so our PEP use the new PDP based on AVP instead. This entire setup is based on the solution introduced in part one, PEP and PDP for Secure Authorization with Cognito,, and as prerequisite that solution should be deployed. You find the entire solution on Serverless Handbook PEP and PDP.

Architecture Overview

Just as a reminder, the entire code and all of the architecture can be found on Serverless Handbook PEP and PDP

Looking at the overview of the architecture and call flow we can see that there is not any major changes, but instead of fetching a permission mapping from DynamoDB we will call AVP and let the service use it's policy engine to allow/deny decision.

To better understand the flow during an API access.

Setup and deploy AVP based PDP

First we need to deploy all the resources needed by AVP, what we need to do is to create a Policy Store with our policy schema. The Policies for our three different Roles with a Cedar based policy to determine what the Role has permission to do, and we need to setup our Identity Source.

This template can be fairly long as the Cedar polices and schema tend to be rather huge. So parts of this template has been left out, therefor visit Serverless Handbook PEP and PDP for the full template.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: PDP Service
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
  UserManagementStackName:
    Type: String
    Description: The name of the stack that contains the user management part, e.g the Cognito UserPool
  AVPNameSpace:
    Type: String
    Description: The name space for Amazon Verified Permissions
    AllowedPattern: "[a-z]+"
  UserPoolId:
    Type: String
    Description: The ID of the Cognito User Pool

Resources:
  PolicyStore:
    Type: AWS::VerifiedPermissions::PolicyStore
    Properties:
      Description: !Sub Policy Store for ${ApplicationName}
      ValidationSettings:
        Mode: "OFF"
      Schema:
        CedarJson: !Sub |
          {
            "${AVPNameSpace}": {
                "entityTypes": {
                    "CognitoUser": {
                        "shape": {
                            "type": "Record",
                            "attributes": {}
                        },
                        "memberOfTypes": [
                            "CognitoUserGroup"
                        ]
                    },
                    "CognitoUserGroup": {
                        "shape": {
                            "attributes": {},
                            "type": "Record"
                        }
                    },
                    "Application": {
                        "shape": {
                            "attributes": {},
                            "type": "Record"
                        }
                    }
                },
                "actions": {
                    "get /rider": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    },
                    "get /riders": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    },
                    "get /trainer": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    },
                    "get /trainers": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    },
                    "get /unicorn": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    },
                    "get /unicorns": {
                        "appliesTo": {
                            "context": {
                                "type": "Record",
                                "attributes": {}
                            },
                            "principalTypes": [
                                "CognitoUser"
                            ],
                            "resourceTypes": [
                                "Application"
                            ]
                        }
                    }
                }
            }
          }

  CognitoIdentitySource:
    Type: AWS::VerifiedPermissions::IdentitySource
    Properties:
      Configuration: 
        CognitoUserPoolConfiguration: 
          ClientIds: 
            - Fn::ImportValue: !Sub ${UserManagementStackName}:app-audience
          GroupConfiguration: 
            GroupEntityType: !Sub ${AVPNameSpace}::CognitoUserGroup
          UserPoolArn: 
            Fn::ImportValue: !Sub ${UserManagementStackName}:user-pool-arn
      PolicyStoreId: !Ref PolicyStore
      PrincipalEntityType: !Sub ${AVPNameSpace}::CognitoUser

  AdminUserPolicy:
    Type: AWS::VerifiedPermissions::Policy
    Properties:
      Definition:
        Static:
          Description: Policy for Admin User group in Cognito
          Statement: !Sub |
            permit(
              principal in ${AVPNameSpace}::CognitoUserGroup::"${UserPoolId}|Admin", 
              action in [ ${AVPNameSpace}::Action::"get /rider", ${AVPNameSpace}::Action::"get /riders", ${AVPNameSpace}::Action::"get /trainer", ${AVPNameSpace}::Action::"get /trainers", ${AVPNameSpace}::Action::"get /unicorn", ${AVPNameSpace}::Action::"get /unicorns" ], 
              resource
            );
      PolicyStoreId: !Ref PolicyStore

  LambdaPDPFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/AuthZ
      Handler: authz.handler
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref AVPCacheTable
        - Version: "2012-10-17"
          Statement:
            Effect: Allow
            Action:
              - "verifiedpermissions:EvaluatePolicy"
              - "verifiedpermissions:GetPolicy"
              - "verifiedpermissions:IsAuthorizedWithToken"
            Resource: "*"
      Environment:
        Variables:
          JWKS_URL:
            Fn::ImportValue: !Sub ${UserManagementStackName}:jwks-url
          AUDIENCE:
            Fn::ImportValue: !Sub ${UserManagementStackName}:app-audience
          POLICY_STORE_ID:
            !Ref PolicyStore
          NAMESPACE:
            !Ref AVPNameSpace
          TOKEN_TYPE: "accessToken"

Authorization logic

The PDP Lambda will get the jwt-token, action, and resource from the PEP, our Lambda Authorizer in the API Gateway. The function will get the principal, the subject, from the JWT-Token and see if there is a cached decision. If there is not it will call AVP with all information to get an authorization decision.

import json
import os
import base64
import boto3
import time
from datetime import datetime, timedelta, timezone

policy_store_id = os.getenv("POLICY_STORE_ID")
namespace = os.getenv("NAMESPACE")
token_type = os.getenv("TOKEN_TYPE")
resource_type = f"{namespace}::Application"
resource_id = namespace
action_type = f"{namespace}::Action"
table_name = os.environ["PERMISSION_CACHE_TABLE"]

avp_client = boto3.client("verifiedpermissions")
dynamodb_client = boto3.client("dynamodb")


def decode_token(bearer_token):
    return json.loads(base64.b64decode(bearer_token.split(".")[1]).decode("utf-8"))


def generate_access(principal, effect, action, resource):
    auth_response = {
        "statusCode": 200 if effect == "Allow" else 403,
        "principalId": principal,
        "effect": effect,
        "action": action,
        "resource": resource,
    }
    return auth_response


def get_auth_cache(principal, action):
    try:
        response = dynamodb_client.get_item(
            TableName=table_name, Key={"PK": {"S": principal}, "SK": {"S": action}}
        )

        item = response.get("Item")
        if not item:
            return None

        ttl = int(item.get("TTL")["N"])
        if ttl and ttl < int(time.time() * 1000):  # TTL is in milliseconds
            return None

        return item.get("Effect")["S"]

    except Exception as e:
        print(f"Error getting auth cache: {e}")
        return None


def store_auth_cache(principal, action, auth_response):
    try:
        ttl = int((datetime.now(timezone.utc) + timedelta(hours=12)).timestamp() * 1000)

        effect = (
            "Allow" if auth_response.get("decision", "").upper() == "ALLOW" else "Deny"
        )

        dynamodb_client.put_item(
            TableName=table_name,
            Item={
                "PK": {"S": principal},
                "SK": {"S": action},
                "TTL": {"N": str(ttl)},
                "Effect": {"S": effect},
            },
        )

    except Exception as e:
        print(f"Error storing auth cache: {e}")


def validate_permission(event):
    jwt_token = event["jwt_token"]
    resource = event["resource"]
    action = event["action"]
    parsed_token = decode_token(jwt_token)

    try:
        action_id = f"{action.lower()} {resource.lower()}"
        user_principal = parsed_token["sub"]

        cached_auth = get_auth_cache(user_principal, action_id)

        if cached_auth is None:
            auth_response = avp_client.is_authorized_with_token(
                accessToken=jwt_token,
                policyStoreId=policy_store_id,
                action={"actionType": action_type, "actionId": action_id},
                resource={"entityType": resource_type, "entityId": resource_id},
            )

            store_auth_cache(user_principal, action_id, auth_response)

            response_body = generate_access(
                user_principal,
                "Allow" if auth_response["decision"].upper() == "ALLOW" else "Deny",
                action,
                resource,
            )

            return {
                "statusCode": 200,
                "body": json.dumps(response_body),
                "headers": {"Content-Type": "application/json"},
            }

        else:
            response_body = generate_access(
                user_principal, cached_auth, action, resource
            )

            return {
                "statusCode": 200,
                "body": json.dumps(response_body),
                "headers": {"Content-Type": "application/json"},
            }

    except Exception as e:
        print(f"Error validating permissions: {e}")

    response_body = generate_access(parsed_token["sub"], "Deny", action, resource)

    return {
        "statusCode": 200,
        "body": json.dumps(response_body),
        "headers": {"Content-Type": "application/json"},
    }


def handler(event, context):
    print(f"Event: {json.dumps(event)}")
    permissions = validate_permission(event)
    return permissions

Update PEP to use AVP Based PDP

To update our PEP to use the new AVP based PDP instead of the DynamoDB based, navigate to the API folder and modify the samconfig.yaml file, update the value of PDP to PDPStackName=pep-pdp-cognito-pdp-auth-service-avp then redeploy the API part of the solution. This will now swap the PDP that is used.

Testing

To test the setup of AVP we can navigate to the AVP part of the console.

Under Policy Stores you should see the policy store that was created for our PDP.

By clicking on the ID of the policy store and selecting Policies in the menu we see the list of the three created policies.

By selecting the Riders policy we can now inspect the create policy.

By selecting Schema in the menu we can inspect the created schema in a visual form.

Now, we can navigate to the Test Bench to test out our policies, fill in the information as shown in the image below. The group must be prefixed with the Cognito User Pool Id and follow pattern <COGNITO_USER_POOL_ID>|<GROUP_NAME>

If we select the action /get trainers and click Run Authorization request we should get a deny back, as the Trainer role don't have access to that Action

Swapping to the /get trainer action should instead give us an allow back.

Summary and conclusion

Implementing PEP and PDP in our authorization flow offers a highly scalable, flexible, and secure way to control access to resources. By leveraging AWS Lambda and API Gateway, we can build a serverless authorization system that separates authentication and authorization concerns, scales with demand, and simplifies policy management.

With the addition of Role-Based Access Control and Amazon Verified Permissions (AVP), combined with caching for enhanced performance, we can create an authorization solution that fits both current and future needs. Using AVP in our SaaS solutions give us a very powerful way to handle multi tenancy.

Happy coding, and stay secure!

Source Code

The entire setup, with detailed deployment instructions, and all the code can be found on Serverless Handbook PEP and PDP

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

PEP and PDP for Secure Authorization with Cognito

Jimmy Dahlqvist — Mon, 03 Feb 2025 11:33:15 +0000

In one of my previous posts Building a serverless connected BBQ as SaaS - Part 4 - AuthZ I touched on the topic around Authentication and Authorization with distributed PEPs (Policy Enforcement Points) and a centralized PDP (Policy Decision Point). In this post I will dig a bit deeper and expand on that setup. I'll explore how these concepts work in practice, the benefits they offer, and how we can leverage them in our serverless architecture using AWS Lambda, API Gateway, and Cognito User Pools.

Additionally, I’ll talk about Role-Based Access Control (RBAC) model, how to implement it using Cognito Groups and DynamoDB, and how caching can boost the performance of our authorization system.

The entire setup, with detailed deployment instructions, and all the code can be found on Serverless Handbook PEP and PDP

Let's start with a short recap.

Authentication (AuthN) vs. Authorization (AuthZ)

It’s crucial to distinguish between Authentication and Authorization, two terms that often get mixed up, and that I have had to explain on so many occasions, but serve very different purposes.

Authentication (AuthN)

Authentication is all about verifying identity. It answers the question, Who are you? When a user logs into our application, authentication ensures that they are who they claim to be. This could involve something as simple as a username and password or more complex multi-factor authentication (MFA).

Authorization (AuthZ)

Once a user’s identity is authenticated, authorization kicks in. This process answers the question, What can you do? Authorization determines what resources, data, and actions a user is permitted to access based on their roles and permissions.

What are PEP and PDP?

Before diving into how to implement them in AWS, it’s essential to understand the roles that PEP and PDP play in authorization.

PEP (Policy Enforcement Point)

In simple terms, the PEP is the gatekeeper. It is the points in our system where access control decisions are enforced. When a user attempts to access a protected resource, the PEPs is the component responsible for checking whether the request is allowed or denied based on the user’s permissions.

In our case, the Lambda Authorizer in API Gateway acts as the PEP. The Lambda Authorizer intercepts every incoming API request, validates the JWT token (typically from Cognito User Pool or any identity provider), and forwards the user’s information (claims) to the PDP for authorization evaluation.

The PEP ensures that the JWT is valid, checks its expiration, verifies its signature, and validates claims (like aud, iss, and sub). It then passes the claims to the PDP for a final decision on whether the user is authorized to access the requested resource.

PDP (Policy Decision Point)

The PDP is where the authorization logic resides. Once the PEP checks the JWT and ensures that the token is valid, the PDP determines whether the user is allowed to access the requested resource based on their roles, permissions, or policies.

The PDP is a separate, implementation, a separate micro service. In our case a Lambda function, that performs the actual authorization decision. It checks the user’s roles, which are stored in the groups claim in the JWT (from Cognito), and compares them against the permissions required to access a specific resource, stored in a data store. In our case we'll use DynamoDB.

The PDP validates if the user has the necessary permissions (like Admin, User, or Manager) to access the resource (e.g., GET /admin, POST /profile). The PDP can also incorporate additional business logic, such as checking time-based access or geo-fencing.

Benefits of using PEP and PDP in Authorization

Implementing distributed PEP and centralized PDP offers several benefits, especially as our applications scales.

Separation of Concerns By splitting the concerns of enforce (PEP) and decision (PDP), we gain cleaner, more maintainable code. The Lambda Authorizer (PEP) is focused purely on validation and enforcement. While the PDP is dedicated to policy evaluation.

Reduced Latency: By placing PEPs close to where decisions need to be enforced, we can reduce the latency, with an caching strategy this can be reduced even more.

Management: With a centralized PDP, all of our authorization logic is centralized in one location. This makes it easier to manage and update policies as our requirements evolve. Whether it’s modifying roles or adding new permission sets, having a central PDP reduces the overhead of updating policies in multiple places.

Consistency and Compliance: Every request is evaluated against the same set of policies, ensuring consistent decision-making across our system.

Scalability: Both the PEP and PDP components scale independently based on demand. If our system needs to handle a larger volume of requests, API Gateway and Lambda can scale automatically. Additionally, the PDP can be optimized for performance by implementing caching.

Flexibility: A PDP allows us to adapt the authorization model to our needs. If our requirements change (for example, moving to attribute-based access control (ABAC) or introducing a more granular permission system), we can easily modify the PDP to accommodate these changes without affecting other parts of the system.

Using PEP and PDP in AWS with Serverless Architecture

In AWS, the PEP and PDP integration fits perfectly with serverless components like Lambda and API Gateway.

PEP - API Gateway Lambda Authorizer

When a client sends a request to our API Gateway endpoint, the Lambda Authorizer (PEP) intercepts the request before it hits our backend service. Our implementation will perform several key steps.

JWT Validation: It decodes the JWT, validates the signature, and checks if the token has expired.

Forwarding Claims: After verifying the token, the Lambda Authorizer forwards the claims (such as sub, groups, and role) to the PDP for further authorization checks. In our solution we will actually forward the entire JWT token.

To reduce the number of calls to our PEP and also PDP we can utilize the authorization cache that exists in API Gateway.

PDP - Authorization logic Lambda function

The PDP is implemented as separate Lambda function, in our case, and will receive the entire JWT token, or claims, to perform the authorization logic, that will include several steps.

Check the user’s role (using the groups claim from Cognito).
Query a DynamoDB table that contains role-to-permission mappings (e.g., which roles have access to which API endpoints).
Evaluates whether the user’s role matches the required permissions for the requested, resource or API endpoint.

ID Token vs Access Token

As we implement the PEP and PDP workflow, it’s essential to understand the difference between ID Tokens and Access Tokens, as both are often used in authorization workflows.

ID Token

The ID Token is primarily used for authentication and contains information about who the user is. It contains claims about the identity of the authenticated user, such as name, email, and phone_number.

Access Token

The Access Token is used to grant the user access to protected resources, authorization. The Access Token contains information about the user’s permissions, such as what resources they are allowed to access and the scopes they have been granted, which define what the user can do (e.g., read:profile, write:profile). The access token do not include the aud claim.

Token customization in Cognito

With the Pre token generation Lambda trigger we could before only customize the ID Token, therefor it was often used for authorization as well. With the introduction of new V2 event in Cognito User Pools we can customize both the ID anf Access token.

Implementing PEP and PDP

With that introduction completed let's dig into implementing a PEP and PDP with RBAC. Our PEP will be the Lambda Authorizer in API Gateway and our PDP will be a separate Lambda function. The PDP will use using Cognito Groups and DynamoDB for the RBAC authorization logic.

Architecture Overview

Just as a reminder, the entire code and all of the architecture can be found on Serverless Handbook PEP and PDP

In this solution we will implement our PEP using Lambda Authorizer in API Gateway. The PDP in this case will also be implemented using a Lambda function. We will assign users a Role using Cognito Groups and we keep an Role - Permission mapping in DynamoDB.

To better understand the flow during an API access.

As seen we will not use an API Gateway for our PDP. Instead our PEP will invoke the PDP Lambda function. There are pros and cons with this approach of course.
On the pro side we have lower latency, a direct Lambda invocation is often faster than an API call. Lower cost as we don't have to pay for the API Gateway invocation. On the backside, we do create a more tight coupling and changing the PDP implementation might get harder. We would need to implement a separate cache in the PDP, using an API Gateway we could rely on the API Gateway cache.

However, the approach you choose need to be a case by case approach, there is not a golden rule exactly how to implement this.

Deploy authentication and Cognito

The first thing we will do is to deploy and setup Cognito and the resources needed for login. We will setup the Cognito User Pool, configure the managed login, and a simple website that will handle the callbacks from Cognito and display our JWT tokens. For simplicity it will just be a static html page from CloudFront and some Lambda@Edge functions. I will use the setup that I have described in this blog post, so for a deep dive I recommend that you read that.

So as a first step deploy the Lambda@Edge, CloudFront distribution, and SSL certificate from Serverless Handbook PEP and PDP

Next, let's deploy and setup Cognito. We will create the UserPool, a client, login style, etc.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Creates the User Pool and Client used for Authentication
Parameters:
  ApplicationName:
    Type: String
    Description: The application that owns this setup.
  DomainName:
    Type: String
    Description: The domain name to use for cloudfront
  HostedAuthDomainPrefix:
    Type: String
    Description: The domain prefix to use for the UserPool hosted UI <HostedAuthDomainPrefix>.auth.[region].amazoncognito.com

Resources:
  UserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UsernameConfiguration:
        CaseSensitive: false
      AutoVerifiedAttributes:
        - email
      UserPoolName: !Sub ${ApplicationName}-user-pool
      Schema:
        - Name: email
          AttributeDataType: String
          Mutable: false
          Required: true
        - Name: name
          AttributeDataType: String
          Mutable: true
          Required: true

  UserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      UserPoolId: !Ref UserPool
      GenerateSecret: True
      AllowedOAuthFlowsUserPoolClient: true
      CallbackURLs:
        - !Sub https://${DomainName}/signin
      AllowedOAuthFlows:
        - code
        - implicit
      AllowedOAuthScopes:
        - phone
        - email
        - openid
        - profile
      SupportedIdentityProviders:
        - COGNITO

  HostedUserPoolDomain:
    Type: AWS::Cognito::UserPoolDomain
    Properties:
      Domain: !Ref HostedAuthDomainPrefix
      ManagedLoginVersion: 2
      UserPoolId: !Ref UserPool

  ManagedLoginStyle:
    Type: AWS::Cognito::ManagedLoginBranding
    Properties:
      ClientId: !Ref UserPoolClient
      UserPoolId: !Ref UserPool
      UseCognitoProvidedValues: true

  UserPoolIdParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Name: !Sub /${ApplicationName}/userPoolId
      Type: String
      Value: !Ref UserPool
      Description: SSM Parameter for the User Pool Id
      Tags:
        ApplicationName: !Ref ApplicationName

  UserPoolHostedUiParameter:
    Type: AWS::SSM::Parameter
    Properties:
      Name: !Sub /${ApplicationName}/userPoolHostedUi
      Type: String
      Value: !Sub https://${HostedAuthDomainPrefix}.auth.${AWS::Region}.amazoncognito.com/login?client_id=${UserPoolClient}&response_type=code&scope=email+openid+phone+profile&redirect_uri=https://${DomainName}/signin
      Description: SSM Parameter for the User Pool Hosted UI
      Tags:
        ApplicationName: !Ref ApplicationName

Outputs:
  CognitoUserPoolJwksUri:
    Value: !Sub https://cognito-idp.${AWS::Region}.amazonaws.com/${UserPool}/.well-known/jwks.json
    Description: The UserPool jwks uri
    Export:
      Name: !Sub ${AWS::StackName}:jwks-url
  CognitoUserPoolID:
    Value: !Ref UserPool
    Description: The UserPool ID
  CognitoAppClientID:
    Value: !Ref UserPoolClient
    Description: The app client
    Export:
      Name: !Sub ${AWS::StackName}:app-audience
  CognitoUrl:
    Description: The url
    Value: !GetAtt UserPool.ProviderURL
  CognitoHostedUI:
    Value: !Sub https://${HostedAuthDomainPrefix}.auth.${AWS::Region}.amazoncognito.com/login?client_id=${UserPoolClient}&response_type=code&scope=email+openid+phone+profile&redirect_uri=https://${DomainName}/signin
    Description: The hosted UI URL

With this deployment done we can move over to the Console and create groups that users can be added to. I will create three groups, Admin, Developer, and Test. Click on Create Group and give it a name. The group name would represent the Role that the user will have and determine what permission he/she will get, more on that setup further down.

We can then create some users and assign them to one of the groups.

To test this setup we can navigate to the webpage deployed with the CloudFront distribution and inspect the JWT tokens, cookies.

If we copy the access token and decode that, I use jwt.io, we can see that my user has the claim cognito:groups that our PEP and PDP will use later for permissions.

Setup and deploy PDP

Next we can deploy our Authorization service, our PDP, responsible for making permission decisions.

Logic will be implemented in a Lambda function and to manage role-based permissions, we will create a DynamoDB tabe that stores permissions for each role. Each permission defines what resources the user can access this could be a specific API endpoint and HTTP method, but of course not limited to that. We'll model the table data

PK (Partition Key): The Role (e.g., Admin, User).
SK (Sort Key): The resource, for example endpoint and Method e.g. GET /unicorn.
Action: The action e.g. GET, PUT, WRITE, READ, LIST etc
Resource: The Resource, for example the endpoint /unicorn
Effect: The Effect, Allow or Deny
Description: A description of the permission.

PK	SK	Action	Resource	Effect	Description
Admin	GET /unicorn	GET	/unicorn	Allow	Admin can access all unicorns
Test	POST /unicorn	POST	/unicorn	Allow	Test can post on unicorns
Developer	DELETE /unicorn	DELETE	/unicorn	Deny	Manager cannot delete a unicorn

This allows us to efficiently look up permissions for each role using a simple DynamoDB query.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Connected BBQ Application Tenant Service
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
  UserManagementStackName:
    Type: String
    Description: The name of the stack that contains the user management part, e.g the Cognito UserPool

Globals:
  Function:
    Timeout: 30
    MemorySize: 2048
    Architectures:
      - arm64
    Runtime: python3.12

Resources:
  PermissionsTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName:
        Fn::Sub: ${ApplicationName}-pdp-role-permission-map
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
      - AttributeName: PK
        AttributeType: S
      - AttributeName: SK
        AttributeType: S
      KeySchema:
      - AttributeName: PK
        KeyType: HASH
      - AttributeName: SK
        KeyType: RANGE

  LambdaPDPFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/AuthZ
      Handler: authz.handler
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref PermissionsTable
      Environment:
        Variables:
          JWKS_URL:
            Fn::ImportValue: !Sub ${UserManagementStackName}:jwks-url
          AUDIENCE:
            Fn::ImportValue: !Sub ${UserManagementStackName}:app-audience
          PERMISSIONS_TABLE:
            !Ref PermissionsTable

Outputs:
  PDPLambdaArn:
    Value: !GetAtt LambdaPDPFunction.Arn
    Description: The ARN of the PDP Lambda Function
    Export:
      Name: !Sub ${AWS::StackName}:pdp-lambda-arn
  PDPLambdaName:
    Value: !Ref LambdaPDPFunction
    Description: The Name of the PDP Lambda Function
    Export:
      Name: !Sub ${AWS::StackName}:pdp-lambda-name

Role Authorization logic

The PDP Lambda will decode the JWT, retrieve the role from the cognito:groups claim, and query the DynamoDB table to check if the role has permission to access the requested resource.

import os
import json
import jwt
import boto3
from jwt import PyJWKClient
from botocore.exceptions import ClientError

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["PERMISSIONS_TABLE"])
JWKS_URL = os.environ["JWKS_URL"]
AUDIENCE = os.environ["AUDIENCE"]


def handler(event, context):
  data = event
    jwt_token = data["jwt_token"]
    resource = data["resource"]
    action = data["action"]

    return check_authorization(jwt_token, action, resource)


def check_authorization(jwt_token, action, resource):
    try:
        jwks_client = PyJWKClient(JWKS_URL)
        signing_key = jwks_client.get_signing_key_from_jwt(jwt_token)

        decoded_token = jwt.decode(
            jwt_token,
            signing_key.key,
            algorithms=["RS256"],
            audience=AUDIENCE,
        )

        role = (
            decoded_token["cognito:groups"][0]
            if "cognito:groups" in decoded_token
            else None
        )

        if not role:
            raise Exception("Unauthorized: Role not found in the token")

        if validate_permission(role, action, resource):
            response_body = generate_access(
                decoded_token["sub"], "Allow", action, resource
            )

            return {
                "statusCode": 200,
                "body": json.dumps(response_body),
                "headers": {"Content-Type": "application/json"},
            }

    except Exception as e:
        print(f"Authorization error: {str(e)}")

    response_body = generate_access(decoded_token["sub"], "Deny", action, resource)

    return {
        "statusCode": 403,
        "body": json.dumps(response_body),
        "headers": {"Content-Type": "application/json"},
    }


def validate_permission(role, action, resource):
    print(f"validate_permission Role: {role}, Action: {action}, Resource: {resource}")
    try:
        response = table.query(
            KeyConditionExpression="PK = :role AND SK = :endpoint",
            ExpressionAttributeValues={
                ":role": role,
                ":endpoint": f"{action} {resource}",
            },
        )
        if response["Items"] and response["Items"][0]["Effect"] == "Allow":
            return True
        else:
            return False
    except ClientError as e:
        print(f"Error querying DynamoDB: {e}")
        return False


def generate_access(principal, effect, action, resource):
    auth_response = {
        "principalId": principal,
        "effect": effect,
        "action": action,
        "resource": resource,
    }
    return auth_response

Deploy API and PEP

Now we can deploy our API and PEP, Lambda Authorizer.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create the API for self service certificate management
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
  UserManagementStackName:
    Type: String
    Description: The name of the stack that contains the user management part, e.g the Cognito UserPool
  PDPStackName:
    Type: String
    Description: The name of the stack that contains the PDP service

Globals:
  Function:
    Timeout: 30
    MemorySize: 2048
    Runtime: python3.12

Resources:
  LambdaGetUnicorn:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GetUnicorn
      Handler: handler.handler
      Events:
        GetUnicorns:
          Type: Api
          Properties:
            Path: /unicorn
            Method: get
            RestApiId: !Ref UnicornApi

  UnicornApi:
    Type: AWS::Serverless::Api
    Properties:
      Description: API for creating and managing Unicorns
      Name: !Sub ${ApplicationName}-api
      StageName: prod
      OpenApiVersion: '3.0.1'
      AlwaysDeploy: true
      EndpointConfiguration: REGIONAL
      Cors:
        AllowMethods: "'GET,PUT,POST,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"
      Auth:
        AddDefaultAuthorizerToCorsPreflight: false
        Authorizers:
          LambdaRequestAuthorizer:
            FunctionArn: !GetAtt LambdaApiAuthorizer.Arn
            FunctionPayloadType: REQUEST
            Identity: 
              Headers: 
                - Authorization
              ReauthorizeEvery: 600
        DefaultAuthorizer: LambdaRequestAuthorizer

  LambdaApiAuthorizer:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/Authorizer/
      Handler: auth.handler
      Policies:
        - LambdaInvokePolicy:
            FunctionName: 
              Fn::ImportValue: !Sub ${PDPStackName}:pdp-lambda-name
      Environment:
        Variables:
          JWKS_URL:
            Fn::ImportValue: !Sub ${UserManagementStackName}:jwks-url
          AUDIENCE:
            Fn::ImportValue: !Sub ${UserManagementStackName}:app-audience
          PDP_AUTHZ_ENDPOINT: 
            Fn::ImportValue: !Sub ${PDPStackName}:pdp-lambda-name

We set our PEP as the default authorizer that way it will be added to each resource and method. To reduce the number of calls to our PDP the Authorization cache in API gateway is used with a TTL of 600 seconds.

PEP Authorization logic

The PEP Lambda Authorizer will decode the JWT, check the validity, and then call the PDP for a final permission decision.

import os
import json
import jwt
import boto3
from jwt import PyJWKClient

lambda_client = boto3.client("lambda")


def handler(event, context):
    print(f"Event: {json.dumps(event)}")
    token = event["headers"].get("authorization", "")
    path = event["path"]
    method = event["httpMethod"]

    if not token:
        raise Exception("Unauthorized")

    token = token.replace("Bearer ", "")

    decoded_token = None
    try:
        jwks_url = os.environ["JWKS_URL"]

        jwks_client = PyJWKClient(jwks_url)
        signing_key = jwks_client.get_signing_key_from_jwt(token)

        decoded_token = jwt.decode(
            token,
            signing_key.key,
            algorithms=["RS256"],
            audience=os.environ["AUDIENCE"],
        )

        data = {
            "jwt_token": token,
            "resource": path,
            "action": method,
        }

        response = lambda_client.invoke(
            FunctionName=os.environ["PDP_AUTHZ_ENDPOINT"],
            InvocationType="RequestResponse",
            Payload=json.dumps(data),
        )

        response_payload = json.loads(response["Payload"].read())
        body = json.loads(response_payload["body"])
        effect = body["effect"]

        return generate_policy(
            decoded_token["sub"], effect, event["methodArn"], decoded_token
        )

    except Exception as e:
        print(f"Authorization error: {str(e)}")

    return generate_policy(
        decoded_token["sub"], "Deny", event["methodArn"], decoded_token
    )


def generate_policy(principal_id, effect, resource):
    auth_response = {
        "principalId": principal_id,
        "policyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {"Action": "execute-api:Invoke", "Effect": effect, "Resource": resource}
            ],
        },
    }
    return auth_response

Importance of caching

Caching is important in optimizing our authorization flow. By reducing calls to the PDP and speeding up decision-making, caching helps improve the overall performance, scalability, and cost-efficiency of our application.

Reduce Latency: By caching role and permission data, the PEP avoids repeated calls our PDP, leading to faster response times and lower latency for each request.
Decrease PDP Load: Caching minimizes the number of calls made our PDP, reducing the risk of hitting rate limits or throttling.
Improve Scalability: With fewer requests hitting our PDP, our architecture can scale more efficiently.
Lower Costs: Caching reduces the need for repeated PDP invocations, which directly lowers Lambda invocation costs.

Summary and conclusion

With the addition of Role-Based Access Control and DynamoDB for storing permissions, combined with in-memory caching for enhanced performance, we can create an authorization solution that fits both current and future needs.

Understanding the difference between ID Tokens and Access Tokens ensures that our system uses each appropriately, helping us build a more secure and efficient authorization system.

Happy coding, and stay secure!

Source Code

The entire setup, with detailed deployment instructions, and all the code can be found on Serverless Handbook PEP and PDP

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

Serverless self-service IoT certificate management - Part 2

Jimmy Dahlqvist — Mon, 23 Dec 2024 13:57:44 +0000

This is the second part in the series about a Certificate Self Service setup for IoT projects. In this part we'll extend the API that we started in the first part. We'll add the possibility to create multiple intermediate CAs, and thereby the possibility to create server certificates from one CA and client/device certificates from a different.

Using multiple intermediate CAs is a good practice from a security and management perspective. If your intermediate CA would become compromised you can revoke and rotate only the affected certificates, signed by the compromised CA. If the same intermediate CA is used to sign all device certificates, in case of a a breach, all device certificates would need to be rotated. We can split device certificates into "cells" and handle each cell independently.

As a reminder!!

WARNING

The solution I build in this series of posts is NOT suited for a production setup. This is purely meant for development environments and for learning purpose!

In a production environment we need:

Continuous monitoring and automated renewal of certificates.
Integration with hardware security modules (HSMs) for key storage.
Compliance with security standards.

For these needs, managed services like AWS Private CA, DigiCert IoT Trust Manager, and Let’s Encrypt are ideal.

Get the source code

As the source code for this project is fairly large not all code is available in this post.
To get the full source code and deploy it your self, visit Serverless-Handbook Self Service IoT Certificate management

Why Build a Self-Service API?

Once again I like to revisit why we like to create this self service system? Why not just use a private CA or IoT Core from AWS or a SaaS solution like DigiCert IoT Trust Manager.

For several of the SaaS solutions that exists you pay per certificate. In many of the teams I have been working with, related to projects in IoT, we have been issuing many certificates per day for testing. Several certs per device, per tenant, and so on. Automatic tests has generated certs over and over again. We have discovered that using some form of self signed certificates, with a self service API, made us more cost efficient. It also enabled us to test different scenarios with several intermediate CAs.

From a learning perspective, new engineers that didn't have that much experience with IoT and certificates could test and learn in a safe and good way, without breaking the bank.

So for my teams a self-service API for certificate management allowed:

Automation: Devices and servers can request and renew certificates programmatically.
Scalability: As our IoT environment grows, the API can handle the increasing demand for certificates.
Learning and Testing: Before adopting a managed service, building your own certificate system helps you understand how PKI works.

Architecture overview

Let's start by going back to the architecture and look at the overview for this setup. There are a some new parts introduced this time.

First there is a certificate inventory introduced. Information about certificates are stored in this DynamoDB table, allowing for querying for certificates, based on the signing parent. To populate the inventory the Lambda functions responsible for creating certificates will post an event onto an Amazon EventBridge event-bus, that will invoke a StepFunction that populate the inventory. This StepFunction will use the newly released JSONata support.

Looking the creation flow it would look like this, depending on if it's a CA or leaf (server / client) certificate being created, everything below the dotted line is run asynchronously.

Last a new Lambda function, responsible for listing and fetching certificates is created.

Update REST API

Now, let's look at the updated API, we'll add three new endpoints, one for creating new device certificates and two for query and fetching.

Endpoint	Method	Description
`/certificates/root`	`POST`	Create a new Root CA.
`/certificates/intermediate`	`POST`	Create a new Intermediate CA.
`/certificates/server`	`POST`	Create a new server certificate.
`/certificates/device`	`POST`	Create a new device / client certificate.
`/certificates`	`GET`	List and search certificates
`/certificates/{certificate}`	`GET`	Get a single certificate

I decided to use separate paths (e.g., /certificates/root, /certificates/intermediate, /certificates/server, /certificates/device) rather than a single endpoint with a type parameter (e.g., /certificates with type as input) as I feel this aligns better with REST principles and improves the API’s readability and usability.

The /certificates/device is very similar to the other endpoints that create certificates, this will create a random uuid that will be the device ID.

For querying for certificates the /certificates endpoint accept two query parameters, parent which is the base64 encoded parent FQDN, e.g bbq.example.com and limit will restrict how many certs that are returned, a further extension would be to also include lastevaluatedkey parameter, allowing for a good pagination.

Add certificate inventory

To add the certificate inventory we start by extending the service with a DynamoDB table and Index that can be used to query for certificates and EventBridge and StepFuntions for updating the inventory.

DynamoDB table

We'll use the FQDN as partition key and the ParentFQDN as the sort key. In the index we'll use the ParentFQDN as the partition key. This gives us the possibility to query for a single certificate and all certificates signed by a specific CA.

  InventoryTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub ${ApplicationName}-certificate-inventory
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: FQDN
          AttributeType: S
        - AttributeName: ParentFQDN
          AttributeType: S
      KeySchema:
        - AttributeName: FQDN
          KeyType: HASH
        - AttributeName: ParentFQDN
          KeyType: RANGE
      GlobalSecondaryIndexes: 
        - IndexName: parent-index
          KeySchema:
            - AttributeName: ParentFQDN
              KeyType: HASH
            - AttributeName: FQDN
              KeyType: RANGE
          Projection:
            ProjectionType: ALL

EventBridge + StepFunctions

We prepared for this already in the first part. The event-bus is created by the template with common infrastructure.

To update the inventory we'll use an event driven approach where the Lambda functions will post an event as soon as the certificate is created. This will however create a eventually consistent solution, where a read after write might not get the latest result. As long as we are aware of this, it should not cause us any problems, and the benefits of using an event-driven approach outweighs that. By using event-driven architecture we can extend and decouple logic in the future.

The functions will post an event with this structure:

{
    "Source": "certificates",
    "DetailType": "created",
    "Detail": 
    {
      "FQDN": "domain name",
      "Type": "Root/Intermediate/Server/Client",
      "ParentFQDN": "Parent",
      "ValidUntil": "Valid to date",
    },
}

To create the StepFunction, we append it to CloudFormation template. We'll add an event matching our structure, so it will be invoked every time an certificate is created.

  CertificateCreatedExpress:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: certificate-created-statemachine/statemachine.asl.yaml
      Tracing:
        Enabled: true
      Logging:
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt CertificateCreatedStateMachineLogGroup.Arn
        IncludeExecutionData: true
        Level: ALL
      DefinitionSubstitutions:
        EventBridgeBusName:
          Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
        InventoryTable: !Ref InventoryTable
        ApplicationName: !Ref ApplicationName
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - DynamoDBCrudPolicy:
            TableName: !Ref InventoryTable
      Events:
        CertificateCreatedEvent:
          Type: EventBridgeRule
          Properties:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
            Pattern:
              source:
                - certificates
              detail-type:
                - created
      Type: EXPRESS

As of now our StateMachine definition is not that big, but it leaves room for us to extend on it later.

Comment: Certificate service - Store Certificate Info
QueryLanguage: JSONata
StartAt: Debug
States:
  Debug:
    Type: Pass
    Next: Store Certificate Info
    Assign:
      FQDN: "{% $states.input.detail.FQDN %}"
      ParentFQDN: "{% $states.input.detail.ParentFQDN %}"
      Type: "{% $states.input.detail.Type %}"
      ValidUntil: "{% $states.input.detail.ValidUntil %}"
  Store Certificate Info:
    Type: Task
    Resource: arn:aws:states:::dynamodb:putItem
    Arguments:
      TableName: ${InventoryTable}
      Item:
        FQDN:
          S: "{% $FQDN %}"
        ParentFQDN:
          S: "{% $ParentFQDN %}"
        Type:
          S: "{% $Type %}"
        ValidUntil:
          S: "{% $ValidUntil %}"
    End: true

As you might see we use two of the new features recently released for StepFunctions. That is variables and JSONata.

Variables

With variables we can use Assign to create variables that are available in all states in the StepFunctions. So now we can create and assign data in an early state and use it through out. No need to recreate the information in every state. This is a very welcome addition. To demonstrate this I create variables in the very first state that I then use. When we use JSONata variables are access {% $variable-name %}, with JSONPath $variable-name.

JSONata

JSONata is also a new addition and you can set the QueryLanguage to either JSONPath (default) or JSONata to set the query language. You have to select either one of them, there is no possibility to mix and match. In JSONata we use {%%} instead of the traditional $.

Add client cert creation

Creating a device (client) certificate is almost the same as creating a server certificate. With the difference that I don't want to specify the full domain, instead I only specify the FQDN for the signing intermediate certificate and the logic generates a new UUID that is used. E.g resulting in uuid.clients.bbq.example.com.

We add a new Lambda function and add it to our API.

  LambdaGenerateDeviceCertificate:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GenerateDeviceCert
      Handler: handler.handler
      Layers:
        - !Ref UtilsLayer
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"
        - EventBridgePutEventsPolicy:
            EventBusName:
              Fn::ImportValue: !Sub "${CommonInfraStackName}:event-bus-name"    
      Events:
        CreateDeviceCertApi:
          Type: Api
          Properties:
            Path: /certificates/device
            Method: post
            RestApiId: !Ref GenerateCertificatesApi

Introduce Lambda Layer

As we now introduce some common utility functions between five different Lambda Functions, I decided to put these in an Lambda Layer. I normally don't use Layers but this time I felt it would be a good approach and it made the code structure a bit easier. To create and use the layer we need to create the Layer version, and set the Lambda functions to use it.

We add the creation of the Layer to the template and update our Lambda functions.

  UtilsLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      LayerName: UtilsLayer
      ContentUri: Lambda/Layer
      CompatibleRuntimes:
        - python3.12
    Metadata:
      BuildMethod: python3.12
      Description: "Utils code for Lambda functions"

  LambdaGenerateRootCA:
    Type: AWS::Serverless::Function
    Properties:
      ....
      Layers:
        - !Ref UtilsLayer
      .....

List / Get certificates

Finally we add the possibility to list and get certificates.

  LambdaListGetCertificates:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/ListGetCertificates
      Layers:
        - !Ref UtilsLayer
      Handler: handler.handler
      Environment:
        Variables:
          DYNAMODB_TABLE: !Ref InventoryTable
          DYNAMODB_INDEX: parent-index
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"
        - EventBridgePutEventsPolicy:
            EventBusName:
              Fn::ImportValue: !Sub "${CommonInfraStackName}:event-bus-name"
        - DynamoDBCrudPolicy:
            TableName: !Ref InventoryTable
      Events:
        ListCertificatesApi:
          Type: Api
          Properties:
            Path: /certificates
            Method: get
            RestApiId: !Ref GenerateCertificatesApi
        GetCertificatesApi:
          Type: Api
          Properties:
            Path: /certificates/{certificate}
            Method: get
            RestApiId: !Ref GenerateCertificatesApi

One major difference between this function and the other four is that this will not be a function with single responsibility. This function will be responsible for both listing certificates and fetching a single certificates. It will handle all the GET methods. This is one of many design approaches you can use when building an API with Lambda functions, single purpose, Lambdalith, read/write separation. I decided to use single purpose functions for several of the write functions, even if there are similarities between them, I felt they was different enough to be single purpose. For the read functionality I decided to put the logic in one function, as the functionality is very similar, getting one certificate or several is just the matter of a list.

This approach has also created a nice Command Query Responsibility Segregation (CQRS).

Conclusion

In this second part we extended our self service API with functionality to create device certificates, we introduced an inventory and the possibility to list and get certificates. Stay tuned for the next part where we will increase the security of the API and the certificate storage, we will also extend the functionality for listing and fetching certificates.

As a reminder!!

This is build for Learning. Use Managed Services for production

While this API is a great learning tool, services like AWS Private CA or Let’s Encrypt are better suited for production.

To get the full source code and deploy it your self, visit Serverless-Handbook Self Service IoT Certificate management

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

Serverless self-service IoT certificate management - Part 1

Jimmy Dahlqvist — Fri, 29 Nov 2024 06:19:29 +0000

I have been working with several different IoT solutions over the years. One thing they all have in common is the need for trusted certificates that can be used both to establish connection but also device identities. Devices and servers need to trust each other to establish secure communication, ensure data integrity, and prevent malicious attacks. A important part of this trust is the Public Key Infrastructure (PKI), where certificates and Certificate Authorities (CAs) play vital roles, there is also a need to manage a large amount of certificates in an easy way.

In some project we have been rolling our own internal PKI solution, this does come with complexity and security requirements. In other we have been using SaaS solutions, like DigiCert IoT Trust Manager, AWS IoT Core and Amazon Private CA.

However, when it came to development, and sometime even test environments, it was not uncommon that we used some sort of self signed certificates, with an easy self service portal to create different certificates.

In this post, which is the first part of two, we will introduce some basics around certificates and PKI. We will also start to create the foundations for Serverless API that can be used to create a self-service portal for generation of certificates.

WARNING

The solution I build in this series of posts is NOT suited for a production setup. This is purely meant for development environments and for learning purpose!

In a production environment we need:

Continuous monitoring and automated renewal of certificates.
Integration with hardware security modules (HSMs) for key storage.
Compliance with security standards.

For these needs, managed services like AWS Private CA, DigiCert IoT Trust Manager, and Let’s Encrypt are ideal.

In this first version of the API we do not using any form of Authorization! This will be added in the next part!

Why Build a Self-Service API?

As said, in an IoT system, thousands of devices need to interact with servers, IoT brokers, and each other. Each of these interactions must be secure, meaning:

Servers must authenticate devices to ensure they’re legitimate.
Devices must authenticate servers to ensure they’re connecting to a trusted source.

Certificates are therefor issued to servers and devices, creating a chain of trust anchored at a Root CA.

A self-service API for certificate management allows:

Automation: Devices and servers can request and renew certificates programmatically.
Scalability: As our IoT environment grows, the API can handle the increasing demand for certificates.
Learning and Testing: Before adopting a managed service, building your own certificate system helps you understand how PKI works.

Overview of Certificates, CAs, and Trust

What Is a Certificate Authority (CA)?

A CA is a trusted entity responsible for issuing digital certificates. These certificates bind a public key to an identity (e.g., a server, device, or user), enabling trust between entities.

CAs are structured in a hierarchy:

Root CA

The ultimate trust anchor.
Self-signed and highly secure.
Should never be used to issue end-entity certificates directly.

Intermediate CA

Issued and signed by the Root CA.
Delegates the responsibility of issuing certificates to end entities (e.g., devices and servers).
Limits the exposure of the Root CA.

End-Entity, leaf, Certificates

Certificates for servers, IoT devices, or users.
Issued by an Intermediate CA and used in client-server communication or mutual authentication.

Certificate Chains and Trust

A certificate chain is a sequence of certificates, where each certificate in the chain is signed by the subsequent certificate. It represents the hierarchical relationship between a certificate and its issuer.

A chain can consist of one or several intermediate certificates. In the image above the chain is illustrated with two intermediate CAs.

During a TLS handshake

Grossly simplified the handshake would be

The server presents its certificate to the client.
The client validates the server certificate by tracing the chain back to a trusted Root CA in its certificate store. When using a self-signed Root CA you need to ensure the bundle is present in the trust store or included in the connection attempt.

In a scenario when mutual authentication is required, the server performs the same process for the client certificate.

Trust across several Intermediate CAs

In an IoT setup, it’s common to have one Intermediate CA issuing server certificates (e.g., for IoT brokers), and another Intermediate CA issuing client certificates (e.g., for devices).
For a client certificate (signed by one Intermediate CA) to trust a server certificate (signed by a different Intermediate CA), both must:

Be part of the same trust hierarchy.

Both Intermediate CAs must be signed by the same Root CA.
The Root CA is the common trust anchor.

Be validated against the chain.

The client verifies the server’s certificate chain, tracing it back to the Root CA.
The server verifies the client’s certificate chain similarly.

This setup ensures scalability and separation of responsibilities. The Server Intermediate CA focuses on servers and brokers. The Device Intermediate CA focuses on IoT devices.

Implementation

Now let's start to implement this self service API. We will build this completely serverless using Amazon API Gateway and Lambda functions, with storage of the certificates in S3 and Certificate Manager. This first part in the blog series will only create the first very basic part of the API, which we will extend in the second part. You can find all of the source code on Serverless-Handbook Self Service IoT Certificate management

We will setup an API Gateway with three endpoints for creating our Root CA, Intermediate CA, and Server cert. Everything will be stored in an S3 bucket, and the server certificate will also be imported to Certificate Manager.

REST API

First look at the REST API and the structure.

Endpoint	Method	Description
`/certificates/root`	`POST`	Create a new Root CA.
`/certificates/intermediate`	`POST`	Create a new Intermediate CA.
`/certificates/server`	`POST`	Create a new server certificate.

I decided to use separate paths (e.g., /certificates/root, /certificates/intermediate, /certificates/server) rather than a single endpoint with a type parameter (e.g., /certificates with type as input) as I feel this aligns better with REST principles and improves the API’s readability and usability.

It's easier to extend the API, with a new certificate type (e.g., device), as we can create a new path like /certificates/device without impacting existing paths. Separate paths naturally segment resources, reducing the need for filtering on the client side. I feel that retrieving all Root CAs is simpler with GET /certificates/root than GET /certificates?type=root.

Common infrastructure

First of all, let's deploy the common infrastructure, in this case it's just the S3 bucket, we will add more things in this template later.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Certificate Self Service Common Infrastructure
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
    Default: image-moderation

Resources:
  StorageBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain
    UpdateReplacePolicy: Retain
    Properties:
      BucketName: !Sub ${ApplicationName}-storage-bucket

Outputs:
  StorageBucketName:
    Description: The name of the certificate bucket
    Value: !Ref StorageBucket
    Export:
      Name: !Sub ${AWS::StackName}:certificate-bucket-name

Next we can setup the endpoints using SAM and AWS::Serverless::Api and the three Lambda function backing the API.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Create the API for self service certificate management
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
  CommonInfraStackName:
    Type: String
    Description: The name of the common stack that contains the EventBridge Bus and more

Globals:
  Function:
    Timeout: 30
    MemorySize: 2048
    Runtime: python3.12
    Environment:
        Variables:
          CERTIFICATE_BUCKET_NAME: 
            Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"

Resources:
  LambdaGenerateRootCA:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GenerateRootCA
      Handler: handler.handler
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"
      Events:
        CreateRootCAApi:
          Type: Api
          Properties:
            Path: /certificates/root
            Method: post
            RestApiId: !Ref GenerateCertificatesApi

  LambdaGenerateIntermediateCA:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GenerateIntermediateCA
      Handler: handler.handler
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"
      Events:
        CreateIntermediateCAApi:
          Type: Api
          Properties:
            Path: /certificates/intermediate
            Method: post
            RestApiId: !Ref GenerateCertificatesApi   

  LambdaGenerateDeviceCertificate:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GenerateDeviceCert
      Handler: handler.handler
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"      

  LambdaGenerateServerCertificate:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/API/GenerateServerCert
      Handler: handler.handler
      Policies:
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub "${CommonInfraStackName}:certificate-bucket-name"
        - Version: "2012-10-17"
          Statement:
            Action:
              - acm:*
            Effect: Allow
            Resource: "*"
      Events:
        CreateServerCAApi:
          Type: Api
          Properties:
            Path: /certificates/server
            Method: post
            RestApiId: !Ref GenerateCertificatesApi

  GenerateCertificatesApi:
    Type: AWS::Serverless::Api
    Properties:
      Description: API for creating and managing certificates
      Name: !Sub ${ApplicationName}-api
      StageName: prod
      OpenApiVersion: '3.0.1'
      AlwaysDeploy: true
      EndpointConfiguration: REGIONAL

Our Lambda functions is in Python and we use the cryptography library to create the certificates, below is the implementation for creating a server certificate. For a full implementation visit Serverless-Handbook Self Service IoT Certificate management.

import boto3
from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization, hashes
import datetime
import os
import json

s3_client = boto3.client("s3")
acm_client = boto3.client("acm")


def create_server_certificate(
    intermediate_private_key_pem, 
    intermediate_cert_pem, 
    fqdn,
    country,
    state,
    organization,
    validity_days,
):
    # Load Intermediate CA private key and certificate
    intermediate_private_key = serialization.load_pem_private_key(
        intermediate_private_key_pem, password=None
    )
    intermediate_cert = x509.load_pem_x509_certificate(intermediate_cert_pem)

    # Generate a private key for the server certificate
    server_private_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
    server_private_key_pem = server_private_key.private_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PrivateFormat.PKCS8,
        encryption_algorithm=serialization.NoEncryption(),
    )

    subject = x509.Name(
        [
            x509.NameAttribute(NameOID.COUNTRY_NAME, country),
            x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, state),
            x509.NameAttribute(NameOID.ORGANIZATION_NAME, organization),
            x509.NameAttribute(NameOID.COMMON_NAME, fqdn),
        ]
    )
    server_certificate = (
        x509.CertificateBuilder()
        .subject_name(subject)
        .issuer_name(intermediate_cert.subject)  # Signed by Intermediate CA
        .public_key(server_private_key.public_key())
        .serial_number(x509.random_serial_number())
        .not_valid_before(datetime.datetime.now(datetime.timezone.utc))
        .not_valid_after(
            datetime.datetime.now(datetime.timezone.utc) + datetime.timedelta(days=validity_days)
        )
        .add_extension(
            x509.SubjectAlternativeName([x509.DNSName(fqdn)]),
            critical=False,
        )
        .add_extension(
            x509.BasicConstraints(ca=False, path_length=None),
            critical=True,
        )
        .add_extension(
            x509.KeyUsage(
                digital_signature=True,
                key_encipherment=True,
                key_cert_sign=False,
                crl_sign=False,
                content_commitment=False,
                data_encipherment=False,
                encipher_only=False,
                decipher_only=False,
                key_agreement=False,
            ),
            critical=True,
        )
        .sign(intermediate_private_key, hashes.SHA256())
    )
    server_cert_pem = server_certificate.public_bytes(serialization.Encoding.PEM)

    return server_private_key_pem, server_cert_pem


def handler(event, context):

    body = json.loads(event["body"])

    bucket_name = os.environ.get("CERTIFICATE_BUCKET_NAME")

    # Fetch Intermediate CA private key and certificate from S3
    intermediate_private_key = s3_client.get_object(
        Bucket=bucket_name, Key="intermediate_ca/private_key.pem"
    )["Body"].read()

    intermediate_cert = s3_client.get_object(
        Bucket=bucket_name, Key="intermediate_ca/certificate.pem"
    )["Body"].read()

    # Fetch the certificate chain from S3
    cert_chain_pem = s3_client.get_object(
        Bucket=bucket_name, Key="intermediate_ca/certificate_chain.pem"
    )["Body"].read()

    # Create Server Certificate
    server_private_key_pem, server_cert_pem = create_server_certificate(
        intermediate_private_key,
        intermediate_cert,
        body["fqdn"],
        body["country"],
        body["state"],
        body["organization"],
        body["validity_days"],
    )

    s3_folder = f"server_certificates/{body["fqdn"]}/"

    # Upload Server Certificate, Private Key to S3
    s3_client.put_object(
        Bucket=bucket_name,
        Key=f"{s3_folder}private_key.pem",
        Body=server_private_key_pem,
    )
    s3_client.put_object(
        Bucket=bucket_name, Key=f"{s3_folder}certificate.pem", Body=server_cert_pem
    )

    # Import the certificate to ACM
    response = acm_client.import_certificate(
        Certificate=server_cert_pem,
        PrivateKey=server_private_key_pem,
        CertificateChain=cert_chain_pem,
    )

    return {
        "statusCode": 200,
        "body": json.dumps(
            {
                "message": "Server certificate created, uploaded to S3, and imported to ACM.",
                "fqdn": body["fqdn"],
                "s3_folder": s3_folder,
                "acm_certificate_arn": response["CertificateArn"],
            }
        ),
    }

Conclusion

This blog covered everything from certificate chains to validating trust in IoT systems. Building your own self-service API is a good way learn about certificates and PKI. In production, however, always opt for managed solutions that offer automation, compliance, and scalability.

Certificates Are the Foundation of IoT Security

Certificates and CAs ensure secure, authenticated communication in IoT ecosystems.

The Role of Intermediate CAs

Delegating certificate issuance to Intermediate CAs improves scalability and limits exposure.

Trust Is Built on Hierarchies

Trust between devices and servers relies on shared Root CAs and validated certificate chains.

Build for Learning, Use Managed Services for Production

While this API is a great learning tool, services like AWS Private CA or Let’s Encrypt are better suited for production.

To get the full source code and deploy it your self, visit Serverless-Handbook Self Service IoT Certificate management

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

Serverless AI powered content moderation service

Jimmy Dahlqvist — Thu, 31 Oct 2024 17:44:29 +0000

Around one year ago I created a blog post about the creation of a File Manager service. in this post we will use this service as our base and extend it with content moderation. We'll use GuardDuty and Rekognition to assist in this task. As usual everything will be serverless and event-driven.

Recap

To refresh everyone's memory let's start with a short recap.

The service will store files in S3 and keep a record of all the files in a DynamoDB table. The system overview looks like this, with and API exposing functionality to the user and then carry out the work in a serverless and event-driven way.

The upload flow is initiated by a client calling the API where a Lambda function will creat pre-signed S3 url that the client can use to upload the file. We don't upload file directly over the API, since Amazon API gateway has a max payload size of 10mb, and to support all kinds of files this will become a limitation.

When the client then uploads the file to S3 this will generate an event, the part below the dashed line in the image, this will invoke a StepFunction that will update the file inventory.

Extended architecture

In the extended architecture we add functionality to use GuardDuty S3 malware scanning and Rekognition for image moderation. GuardDuty will scan new files that arrive in the S3 bucket, that I call staging, a tag will will be added to the object and the scan result posted to the default event-bus. The scan result, if OK, will invoke a StepFunction next that utilize Rekognition for image moderation. I have implemented the same logic in this StepFunction and add a tag on the object and post an event onto a event-bus. Finally files are moved to either a quarantine ocr storage bucket.

Every part of the solution is decoupled and can run independently and a saga pattern is applied to move the logic to the next phase.

Now let's dig a bit deeper into each of the parts of this solution.

Malware scanning

The GuardDuty Malware scanning doesn't require much setup. This is a fully managed feature in GuardDuty and the only thing that is required is that a configuration of it. GuardDuty will then pick up new object automatically.

To achieve this flow the only thing we need to do is to create a S3MalwareProtectionPlan and assign it appropriate permissions. One important thing to remember, if you encrypt your objects in S3 with a Customer Managed Key, don't forget to give GuardDuty permissions to decrypt using this key.

  S3MalwareProtectionPlan:
    Type: AWS::GuardDuty::MalwareProtectionPlan
    Properties:
      Actions:
        Tagging:
          Status: ENABLED
      ProtectedResource:
        S3Bucket:
          BucketName:
            Fn::ImportValue: !Sub ${CommonInfraStackName}:staging-bucket-name
      Role: !GetAtt S3MalwareProtectionPlanRole.Arn

Image moderation

The moderation part with Rekognition involves a couple of more steps. A StepFunction is invoked by the result from the Malware scan, and call Rekognition to moderate the image. This StepFunction will then tag the object and post the scan result onto EventBridge custom service bus. One important thing to remember, if you encrypt your objects in S3 with a Customer Managed Key, don't forget to give permissions to decrypt using this key. Rekognition will give you a strange error Unsupported and not a clear error to why it failed in this case.

To achieve this flow the only thing we need to do is to create the StateMachine and setup the events it should be invoked on.

  ModerateImageStateMachineStandard:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: StateMachine/moderation.asl.yaml
      DefinitionSubstitutions:
        EventBusName: 
          Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
      Tracing:
        Enabled: true
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - S3FullAccessPolicy:
            BucketName: 
              Fn::ImportValue: !Sub ${CommonInfraStackName}:staging-bucket-name
        - EventBridgePutEventsPolicy:
            EventBusName: 
              Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
        - RekognitionDetectOnlyPolicy: {}
      Events:
        GuardDutyMalwareScanResult:
          Type: EventBridgeRule
          Properties:
            InputPath: $.detail
            Pattern:
              source:
                - aws.guardduty
              detail-type:
                - GuardDuty Malware Protection Object Scan Result
              detail:
                scanResultDetails:
                  scanResultStatus:
                    - NO_THREATS_FOUND

The StateMachine definition is rather large, and there is need for some magic. Since you can't append tags to an S3 object, we first need to fetch all existing tags, append our new tag and put the entire array of tags on the object. This would probably be easier to do in a Lambda function, but where is the fun in that. Intrinsic functions for the win....

Comment: Moderate images using Rekognition
StartAt: Debug
States:
  Debug:
    Type: Pass
    Next: Get Object Metadata
  Get Object Metadata:
    Type: Task
    Parameters:
      Bucket.$: $.s3ObjectDetails.bucketName
      Key.$: $.s3ObjectDetails.objectKey
    Resource: arn:aws:states:::aws-sdk:s3:headObject
    Next: Get Object Tags
    ResultPath: $.S3MetaData
  Get Object Tags:
    Type: Task
    Parameters:
      Bucket.$: $.s3ObjectDetails.bucketName
      Key.$: $.s3ObjectDetails.objectKey
    Resource: arn:aws:states:::aws-sdk:s3:getObjectTagging
    Next: Is File Supported?
    ResultPath: $.s3Tags
  Is File Supported?:
    Type: Choice
    Choices:
      - Or:
          - Variable: $.S3MetaData.ContentType
            StringMatches: image/png
          - Variable: $.S3MetaData.ContentType
            StringMatches: image/jpeg
        Next: Moderate Image
    Default: File Not Supported
  File Not Supported:
    Type: Pass
    Next: Add FILE_NOT_SUPPORTED to Object Tags Array
    Parameters:
      ContentTypes: []
      ModerationLabels: []
      ModerationModelVersion: "7.0"
      ThreatsFound: "-1"
    ResultPath: $.RekognitionModeration
  Add FILE_NOT_SUPPORTED to Object Tags Array:
    Type: Pass
    ResultPath: $.scanResult
    Parameters:
      status: FILE_NOT_SUPPORTED
      newTagSet.$: >-
        States.StringToJson(States.Format('[{},{}]',
        States.ArrayGetItem(States.StringSplit(States.JsonToString($.s3Tags.TagSet),
        '[]'),0), '{"Key":"ImageModerationStatus","Value":"FILE_NOT_SUPPORTED"}'))
    Next: Tag S3 Object
  Moderate Image:
    Type: Task
    Parameters:
      Image:
        S3Object:
          Bucket.$: $.s3ObjectDetails.bucketName
          Name.$: $.s3ObjectDetails.objectKey
    Resource: arn:aws:states:::aws-sdk:rekognition:detectModerationLabels
    Next: File Supported
    ResultPath: $.RekognitionModeration
  File Supported:
    Type: Pass
    Parameters:
      ThreatsFound.$: States.ArrayLength($.RekognitionModeration.ModerationLabels)
      ContentTypes.$: $.RekognitionModeration.ContentTypes
      ModerationLabels.$: $.RekognitionModeration.ModerationLabels
      ModerationModelVersion.$: $.RekognitionModeration.ModerationModelVersion
    ResultPath: $.RekognitionModeration
    Next: Was Threats Found?
  Was Threats Found?:
    Type: Choice
    Choices:
      - Variable: $.RekognitionModeration.ThreatsFound
        NumericGreaterThan: 0
        Next: Add THREATS_DETECTED to Object Tags Array
    Default: Add NO_THREATS to Object Tags Array
  Add THREATS_DETECTED to Object Tags Array:
    Type: Pass
    Parameters:
      status: THREATS_FOUND
      newTagSet.$: >-
        States.StringToJson(States.Format('[{},{}]',
        States.ArrayGetItem(States.StringSplit(States.JsonToString($.s3Tags.TagSet),
        '[]'),0), '{"Key":"ImageModerationStatus","Value":"THREATS_FOUND"}'))
    ResultPath: $.scanResult
    Next: Tag S3 Object
  Add NO_THREATS to Object Tags Array:
    Type: Pass
    Next: Tag S3 Object
    Parameters:
      status: NO_THREATS_FOUND
      newTagSet.$: >-
        States.StringToJson(States.Format('[{},{}]',
        States.ArrayGetItem(States.StringSplit(States.JsonToString($.s3Tags.TagSet),
        '[]'),0), '{"Key":"ImageModerationStatus","Value":"NO_THREATS_FOUND"}'))
    ResultPath: $.scanResult
  Tag S3 Object:
    Type: Task
    Parameters:
      Bucket.$: $.s3ObjectDetails.bucketName
      Key.$: $.s3ObjectDetails.objectKey
      Tagging:
        TagSet.$: $.scanResult.newTagSet
    Resource: arn:aws:states:::aws-sdk:s3:putObjectTagging
    ResultPath: null
    Next: Post Scan Result Event
  Post Scan Result Event:
    Type: Task
    Resource: arn:aws:states:::events:putEvents
    Parameters:
      Entries:
        - Detail:
            metadata: {}
            data:
              id.$: $.s3ObjectDetails.objectKey
              status.$: $.scanResult.status
              scanData.$: $.RekognitionModeration
          DetailType: Moderation Scan Completed
          EventBusName: ${EventBusName}
          Source: ImageModeration
    End: true
    ResultPath: null

Finalize the upload

The last part of this solution is to react to the moderation and place the content either in the quarantine bucket or the long term bucket. For this I use two different StepFunction with some difference in event that invokes them.

To achieve this flow a new StateMachine is created.

  MoveFilesToPermanentStorageStateMachineStandard:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: StateMachine/move-to-permanent-storage.asl.yaml
      DefinitionSubstitutions:
        EventBusName:
          Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
        StorageBucketName:
          Fn::ImportValue: !Sub ${CommonInfraStackName}:storage-bucket-name
        StagingBucketName:
          Fn::ImportValue: !Sub ${CommonInfraStackName}:staging-bucket-name
      Tracing:
        Enabled: true
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - S3FullAccessPolicy:
            BucketName:
              Fn::ImportValue: !Sub ${CommonInfraStackName}:staging-bucket-name
        - S3FullAccessPolicy:
            BucketName:
              Fn::ImportValue: !Sub ${CommonInfraStackName}:storage-bucket-name
        - EventBridgePutEventsPolicy:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
      Events:
        NoModerationThreatsFoundEvent:
          Type: EventBridgeRule
          Properties:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonInfraStackName}:event-bus-name
            InputPath: $.detail
            Pattern:
              source:
                - ImageModeration
              detail-type:
                - Moderation Scan Completed
              detail:
                data:
                  status:
                    - NO_THREATS_FOUND

With a StateMachine definition that is a bit easier to follow then the image moderation part.

Comment: Handle result and copy files to storage bucket
StartAt: Debug
States:
  Debug:
    Type: Pass
    Next: Get Object Metadata
  Get Object Metadata:
    Type: Task
    Parameters:
      Bucket: ${StagingBucketName}
      Key.$: $.data.id
    Resource: arn:aws:states:::aws-sdk:s3:headObject
    Next: CopyObject
    ResultPath: $.S3MetaData
  CopyObject:
    Type: Task
    Parameters:
      Bucket: ${StorageBucketName}
      CopySource.$: >-
        States.Format('${StagingBucketName}/{}',$.data.id)
      Key.$: $.data.id
    Resource: arn:aws:states:::aws-sdk:s3:copyObject
    ResultPath: null
    Next: DeleteObject
  DeleteObject:
    Type: Task
    Parameters:
      Bucket: ${StagingBucketName}
      Key.$: $.data.id
    Resource: arn:aws:states:::aws-sdk:s3:deleteObject
    ResultPath: null
    Next: Post Event File Moved
  Post Event File Moved:
    Type: Task
    Resource: arn:aws:states:::events:putEvents
    Parameters:
      Entries:
        - Detail:
            metadata: {}
            data:
              id.$: $.data.id
              status: STORED
              contentType.$: $.S3MetaData.ContentType
              fileSize.$: $.S3MetaData.ContentLength
          DetailType: Completed
          EventBusName: ${EventBusName}
          Source: ImageModeration
    End: true
    ResultPath: null

Conclusion

This was a short post on how I extended my previous built file manager with malware scanning and image moderation. Using only managed services made this a fairly easy task.

To get the full source code and deploy it your self, visit Serverless-Handbook Image Moderation

Final Words

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

The art of surviving re:Invent: Tricks from a eight year veteran

Jimmy Dahlqvist — Wed, 25 Sep 2024 12:34:56 +0000

It's that time of the year again, when AWS re:Invent come sneaking up on you. I have attended re:Invent every year since 2016 and in 2022 I was also a speaker. This will be my eight year in-person and one virtual in 2020.

Over the years I have collected a fair amount of tips and tricks how to survive not only re:Invent but also Vegas as a city. In this post I will try and share what I think you need to know before going to re:Invent as an (first or any timer) attendee. It will include what you need to understand about re:Invent, sessions, keynotes, Vegas, and how not to get scammed.

Prepare to walk a lot, get very little sleep, a ton of learning, and new friends!!

Buckle up and let's start out journey to the party capital of the world!!

Viva Las Vegas

Let's start with some Las Vegas tips and tricks that will make you re:Invent so much better!

Travel

So you have decided to visit re:Invent, hopefully by the time I publish this post you have already booked your travel, but still there are some good things to know.

First of all I don't recommend that you get to Vegas later then the Saturday before re:Invent, November 30th this year. Specially if you travel from other parts of the world. You will need Sunday to relax and fight the jet lag. But even if you are based in the US it's still good to have Sunday to settle in and prepare. Badge pickup also opens on Sunday, see the 'Intro to re:Invent' section.

I normally travel back home again on the Friday of the re:Invent week, there are session planned until lunch so getting a flight in the afternoon is good option. This year I will stay till Saturday.

Getting from / to the airport

Basically you two ways to get from the airport to your hotel, Taxi or Rideshare (Uber/Lyft). Don't assume that Uber/Lyft always will be the cheapest, they often are, but during busy hours the price go up, and the wait can easy be an hour or even more. There is a fixed rate between the airport and the hotels on The Strip (Las Vegas Blvd). Sometimes the drivers insist you use the meter, it will always be more expensive the fixed price, so don't fall for that trick. This is the first scam to lookout for.

For more information about fixed rates visit Nevada Taxicab Authority

Where to stay?

AWS offers deals on several hotels in Vegas. However don't assume that the prices via AWS is always the best, you need to do some research, and most years I manage to book at a cheaper price outside of the re:Invent portal. Here is a tip for you if you lan to come back again next year. All hotels in Vegas offers free cancellations, all money back. So by booking already in February I have managed to book good rooms as low as $45 / night (plus tax and fees).

There are many good hotels in Vegas, and where you stay depends on your budget. To be honest you will not spend much time in your room so for med a comfortable bed and a clean room is all I ever ask for. Over the years I have been staying more closer and closer to the Venetian, see the sections below. However, there are some benefits staying in the south end of The Strip, I feel that it's a more quite area. I have stayed in both MGM Grand and New York New York, yes this is far from The Venetian but I actually enjoy the morning walk with a cup of coffee in my hand.

Some of my favorite hotels include Paris and Horseshoe, I feel that Horseshoe is a underestimated hotel. It's very close to Caesars forum and you can walk down Linq Ln which is a great shortcut.

Vegas Weather

The weather in the winter in Vegas is very unpredictable. I have seen everything from snow to very warm weather. In the morning it can go as low as 5-10C (40-50F), and in the afternoon in the sun easily be 25C (75F). Layers, light jacket, hoodies are your friend. It rarely rain but when it does it RAINS, if you have room packing an umbrella or a rain poncho can be useful.

Things to do in and around Vegas

Vegas is the party capital of the world and there are plenty of entertainment and things to do. During re:Invent week you will be fully busy with re:Invent so don't try and add a lot of other things onto you schedule. If you arrive a few days before and want to change things up a bit this is what I would recommend.

Hoover dam and Grand Canyon

Rent a car and drive to Hoover dam and Grand Canyon. I have on several occasions done exactly this. Eagle point in Grand Canyon west is a good location. This is around 2,5h hours drive.

The Sphere

The Sphere opened up last year (2023) and there are some good show, I would recommend Postcard from earth

Las Vegas sign

The Las Vegas sign is a bit overrated but as a first timer it for sure should be on your list.

Walk The Strip

Just go for a walk down The Strip, take photos of all of the hotels. Walk into M&M Store, Hershey's, CocaCola, and several others.

Freemont street experience
Freemont street is the older part of Vegas and this is where you find classic casinos like the Golden Nugget. Freemont is well worth a visit, go in the evening, buy a beer, listen to some music, hang around and eat some cheap food. The prices on Freemont are significantly lower than on The Strip.

Don't get scammed

When visiting Las Vegas and re:Invent for the first time, or any time for that, you don't want to go home and feel that you got scammed. In this section I'll try and list some of the most common scams in Vegas, avoid them!

Pedestrian overpass

This is not scam per say, but as of very recent it's no longer allowed to stop on the pedestrian overpasses. This can lead to a hefty fine, if you stop to take some photos. I don't know it Police will enforce it, but hey, just avoid stopping.

Hotel convenience stores

NEVER ever buy things from the hotel convenience stores, this is seriously overpriced, you can have to pay upwards $10 for a bag of candy or snacks. Instead leave the hotel and go to Target or Walgreens.

Show girls and characters

All over The Strip you will find all kinds of characters, from Disney to Transformers, and show girls with big feather attire. All of them want you to take a picture with them, afterwards they will require a hefty tip, $50 easy. Now it is a tip and they can't legally charge you but at the same time you probably don't wanna make a big scene. IF you like to take a picture with them as a memory, make sure to agree on the "price" before snapping the image!

Monk scam

This is not unique to Vegas. All over you will run into these fake monks that try and hand you a bracelet, just don't take it. If you do they will start require that you make a donation, if you then try and hand the bracelet back they will argue that since you touched it, it has lost its "magic". Just avoid them, and put your hands up if they try and put the bracelet on you.

Fake CD scam

Also not unique to Vegas, is these people that try and hand you a "free CD". They claim to be struggling artists and even sign the CD for you. Of course they also want a donation for the CD, if you would take it and hand them some cash, the CD is 100% blank.

Freemont street stands

The street stands on Freemont street is crazy overpriced. If you want to buy from them, make sure to haggle a lot!

Five Vegas rookie mistakes to avoid

When in Vegas as a rookie there are things you should know, that maybe not class a scam, rather mistakes to avoid.

Tips tips tips everywhere

Vegas runs on tips, many of the hospitality workers rely on tip. I personally don't like the tipping culture, but when in Vegas make sure you tip the cocktail waitress, housekeeping, taxi drivers, and other hospitality staff. If you go to a re:Invent party with free drinks, tip the waitress. First of all you will get great service but second as this is a free event they don't make tips, so be kind and leave a tip.

Not using in room safe
One of the biggest mistakes would be to not secure your room door or use the in room safe. You don't want anyone to get into your room, but if they do you don't want them to easy steal your laptop. Use the safe people!

Hotel minibar

This almost classify as a scam! Be aware of the minibar in the room. The drinks and snacks are crazy expensive and many of the minibars has sensors that detect if you remove an item and automatically charge you! There normally are som vague signs about it, just don't touch the items in the minibar, and absolutely don't remove something to keep the drink you bought at Target cold.

ATM fees

Make sure you bring cash! The fees charged by the ATM machines in the hotels can be upwards $10 per withdraw!

Pot builder slot machine

If you plan to play some slot machines, understand that the "Pot builder" machine, this is machines that show some form of graphics like gold in a pot, fireworks about to go off, pigs that get bigger and bigger. If the pot fill up or the firework go off you win the bonus. This graphic is just a graphic and does not indicate that you are "close to winning".

Packing tips

There are so many re:Invent guides that give great advice regards packing. So I will not focus too much on it. The four things that you MUST bring is:

Shoes
Great walking shoes, trust me, you will be walking. On average I walk between 25-30K steps per day. Also, bring more than one pair.

Chapstick and moisturizer
Vegas is in the desert, trust me! Your lips and skin will thank you!

Earplugs
Hotels and Casinos are loud. RePlay is loud. Your ears will thank you!

Travel adapter and power bank
If you live outside of the US, don't forget your travel adapter! For all attendees a power bank is good to have, to keep your phone charged all day.

More Vegas tips

For even more Vegas tips and trick I would recommend to check out Jacobs life in Vegas and Travel Ruby both on YouTube.

re:Invent

That was a lot about Vegas and as a city and things to remember when visiting. Now let's focus on why we are here, re:Invent! This is such a huge event with 60-70K attendees every year, multiple days of learning, and late night partying. There are tricks to surviving this. A week of vacation when coming home for sure, just joking, but often it feels like I need it.

I will go over some major topics from meals, to sessions, reserved seating, parties, and much much more!

re:Invent Campus

The re:Invent campus stretch from Mandalay Bay in the south and Wynn/Encore in the north. This is a huge distance, we talk about 75minutes walking or 15minutes by car. It is across the entire Strip.

The campus consists of two main categories of hotel, venue and sleeping hotel. A venue is where you will find sessions and content, most venues also serve as sleeping room hotel. Then the sleeping room hotel is just for accommodations, there are no session or content. For all Venues and sleeping hotels checkout re:Invent Campus

Getting around

To get an good overview of the Campus area, and the different venues, check out the maps from conferenceparties.com.

First of all, don't underestimate the distance between venues. They might look close but the time it take to get from one venue to another can be very long. Also just walking from the entrance of the hotel to your session can be an easy 15minutes, these hotels are huge.

The transportation options you have are:

Walking

I like walking! I tend to walk everywhere. Often this can be the quickest route, depending on traffic. BUT!! Distance, don't underestimate it. Walking from Venetian to Mandalay Bay is easy one hour walk. So before opting for this, make sure to check Google / Apple maps to see how long time it will take.

Shuttle

AWS provide a free shuttle service that run between the different venues. This is a good option if you don't feel like walking. The shuttle time will be clearly displayed in the re:Invent app and at each venue. Make sure to check before you board, sometimes walking is faster.

Monorail

There is a monorail that run between MGM Grand and Sahara, AWS does offer, at the time of writing, free rides between MGM Grand and The Linq station.

For any changes in the services that AWS provide during re:Invent make sure to check out re:Invent Transportation

Session Levels

All sessions come with a set level and there are four different levels, 100, 200, 300, and 400. Going to a session on the wrong level, to easy or to hard, can ruin the experience. There are a few tricks I use when selecting the correct level, only targeting a certain level is a mistake in my opinion. Unfortunately level mismatch happens, where a session is advertised as a 300 level but the presentation is still on a 200 level.

During re:Invent I mainly focus on level 300 and 400, but mix it up with a couple of 200 level session on topics that I'm not familiar with or customer presentations.

100 Beginner
This is entry level sessions, I normally never attend 100 level session, that however doesn't mean they are not right for you. There are not that many 100 level sessions.

These sessions are a good fit if you are:

New to AWS and new to the topic.

200 Intermediate
Intermediate level sessions, I find that 200 level sessions can be interesting if it's for a topic that I don't know much about. Many customer presentations, where a customer tells about what they build and how, are often 200 level. I have been in very many super interesting customer presentations where they talk about their solution.

These sessions are a good fit if you are:

New to AWS and the topic
New to AWS but familiar with the topic
Familiar to AWS but new to the topic.

300 Advanced
Advanced level sessions, these sessions are on a deeper level and assume that you have previous experience with AWS and the topic. If you are not familiar with the topic and AWS you will probably not be able to keep up with the sessions.

These sessions are a good fit if you are:

Familiar to AWS and the topic

400 Expert
This is the true deep dive sessions. In these sessions the presenter goes deep into the topic and AWS. These sessions assume that you not only have previous experience with AWS and the topic, but also that you understand the details of services, features, topic.

These sessions are a good fit if you are:

Experienced in AWS and the topic

Session Types

There are several different session types. Knowing the difference between them is key when planning you schedule.

Breakout sessions
This is the most common type, 1 hour lecture style session. Most breakout sessions are recorded and published on YouTube. You might hear from other people that don't focus on breakout sessions, as they are recorded and you can watch them later. Even if you can watch a session recording later there is still something special being in the room, nuances are lost in the recording. I can enjoy a 200 level breakout session on a topic that I don't know, or a 400 level with deep dives.

Chalk talks
A smaller highly interactive session. Chalk talks normally start with a short lecture but then opens up for Q&A, discussions and whiteboarding. Chalk talks are great and I enjoy them, the room is often smaller and seats fill up fast. However, the quality on these session very much depends on the audience, as most of the session is interactive. Unlike a breakout session, if the audience don't open up for discussion or ask questions on the wrong level, a chalk talk can quickly be ruined.

Code talks
A smaller highly interactive session just as Chalk talks, but with more focus on live coding and code samples. Just as with Chalk talks the audience is encouraged to ask questions and follow along.

Lightning talks
Short 20 min talks that take place in the Expo Hall, this can be talks from the DEV track, delivered by the community, or partner demos and presentations. The DEV talks delivered by the community are great, partner talks and demos I don't get much value from.

Workshops
Interactive session where you work in small groups or alone to build a solution. Workshops can be a great way to test a new service or feature on a topic that you don't know. However, workshops quickly become a "copy & paste exercise" even on a 400 level. I have done my fair amount of workshops and still have not found one that I truly enjoyed. NOTE! That you MUST bring your own laptop

Builders’ sessions
Just like workshop but in a shorter and smaller setup. Builders’ sessions focus more on experimenting than workshops, a good way to mix up your day. NOTE! That you MUST bring your own laptop

Peertalks

Peertalks was introduced a couple of years back. This is a interesting concept where you can book 1:1 meetings with AWS staff, presenters, and other attendees.

Community activities

There are many community activities where you can network with your peers. There are something for everyone, from 5K run, Bingo night, Giant slide, and much more.

Keynotes

There are five keynotes happening during the week, I would say that you should attend at least one keynote in-person. You can also live stream some of the keynotes or watch them on YouTube later. I personally enjoy being in the keynote hall getting the news first hand. If you are a builder ans only want to attend one keynote, I would say that you should pick Dr. Werner Vogels Keynote.

Monday Night Live with Peter DeSantis
The week kick off with a keynote on Monday night with Peter DeSantis, Senior Vice President of AWS Utility Computing. He continues the Monday Night Live tradition of diving deep into the engineering that powers AWS services.

CEO Keynote with Matt Garman
Matt Garman, new CEO of AWS, takes the stage, normally Tuesday morning, and talk about new releases, customer cases, new experiences and much much more. This will be Matt's first keynote as CEO and I have a feeling it can be a interesting one.

Dr. Werner Vogels Keynote
Dr. Werner Vogels, CTO Amazon.com, takes the stage, normally Thursday morning, and talk about technology trends. Last year he introduced the Frugal Architect and the year before it was all about the asynchronous world. Werner often also reveals the DJ on the rePlay main stage. This is my favorite keynote and I would never miss it. 2023 I also had a front row seat.

Dr. Swami Sivasubramanian Keynote
Dr. Swami Sivasubramanian, VP of AI and Data at AWS, talks about AI, Data, and ML and how to create innovative and differentiated solutions for your customers. He invite customers on stage to talk about with real-world examples of how they’ve used data and generative AI to support a variety of use cases.

AWS Partner Keynote with Dr. Ruba Borno
Dr. Ruba Borno, Vice President, Global Specialists and Partners. She dive deep into the strategic partnerships to show the positive impact AWS and its partners are making possible for customers. She invites partners on stage that talk about their journey and how they are helping their customers innovate. This keynote is probably not for you unless you work for an AWS partner, as it focus 100% on this.

Build your schedule (plan your week)

NOW!! This is the most important part about re:Invent and it can make or break your week. Building your schedule and planing your week is crucial. Trust me, you don't wing re:Invent. Your plan need to be more than sessions, you need to include breaks, networking, evening activities, The Expo, The Hallway track and much much more. I will try and give you some tips and tell you how I normally plan out my week. There is one crucial thing you need to know about sessions so let's start with that.

Reserved seating
Each session has a fixed number of reserved seats, around 75% or so. The possibility to reserve a seat in a session normally opens in beginning / mid October. Therefor it's crucial that you have your schedule ready by then. You will reserve your seat in the session catalog and you must be signed in to your re:Invent account. It normally open up in the morning Seattle time. The most popular sessions become full in minutes, no joke, you need to be fast. What I normally do is have a list of session IDs and ordered by the importance, then I copy & paste and filter the catalog. Since sessions fill up fast, I normally recommend that you have one or two backup sessions. Even if you don't get a reserved seat, you can always line up as around 25% is set aside for walk up. Also is people don't show up the room will be filled with people from the walkup line.

Hallway track
You will probably hear about the hallway track, and I will mention it as well. This is those spontaneous meetings and people you run into in the hallway.

Session planning
To plan your sessions you can use the official session catalog or you can use one of the unofficial planners that exists. I would recommend https://reinvent-planner.cloud/ by AWS community builder Raphael Manke.

Planning tips & tricks
So now, let's go into some tips & tricks when it come to planning you week, and how I normally do it. My first advice is not to pack your days to much, leave room for breaks and food. If you plan to switch venues make sure to leave room for transportation, and don't book back-2-back sessions, you will not make it. You need at least 30min between sessions within the same venue, and at least 1 hour if you switch venue. There are exception to this rule.... And that is if you move between Venetian and Caesars Forum or Venetian and Wynn/Encore as there are connected walkways that are quick to use. I try and not do more than four sessions per day, it might not sound that much, but you need to stay alert to pick up the information, so include breaks and don't skip food!!

A second tip is to pick up your badge on Sunday so you are ready when it all starts Monday morning!

Expo and swag

The Expo is nice, it can be very loud so remember those ear plugs. Walk around the different vendors, get some swag from them, meet AWS employees in the AWS village, visit the Hero and Community Lounge. Plan to spend a few hours in the Expo for sure.

Swag is nice! BUT! Don't grab swag that you don't plan to use, be environmentally friendly and only grab what you need / use. Bringing swag home just to throw it away is not the eco-smart. If you plan to grab swag make sure to leave room in your suitcase.

Eat and relax

Don't skip food, don't skip breaks, don't pack your days! You need the energy to handle the days. Drink a lot of water, Vegas is still in the dessert, that chapstick.

Breakfast
Is served in several venues Monday to Friday, this is buffet style breakfast with different cold and hot options, it's decent but not anything fancy.

Lunch
Is served in several venues Monday to Thursday, just as with breakfast it's a buffet style serving. The lunch is also great but not fancy, they serve different things every day like pasta or tacos.

Plan for breakfast and lunch! You will need the energy!!

Breaks
Plan for breaks, go outside and get ome fresh air. If you are certified visit the Certification Lounge. If you are a Community Builder, User Group Leader, or Hero visit the Community Lounge.
Breaks are important and often you meed the most fun people in these areas!

Parties, networking, and evening activities

Vegas - The party capital of the world!

There are always something going on. AWS arrange several evening activities like the Welcome Reception on Monday and Networking Reception on Tuesday and Wednesday. Sponsors and vendors also arrange a ton on parties that you can attend to get free food and drinks. Don't party to hard though, there is always tomorrow. You can checkout Conference Parties for a list of parties, there will be more and more added very week up till re:Invent.

One big advice is to socialize, say hi to people, join strangers at lunch, join one of the community events. You never know who you meet, and I have had so many great conversations over lunch!

rePlay

On Thursday night re:Invent go out with a BANG with rePlay. On the main stage you will have some of the best DJs in the world, if you like dancing this is the place to be. There are free food and drinks during the party, the lines can sometimes be very long.

The party will be outside at the Las Vegas Festival Grounds, it can be very chilly so dress accordingly, remember those layers again!

Shuttle busses go from the venues to the party or you can take the monorail, that is what I normally do, and AWS often arrange free monorail rides.

Final Words

This was a long post where I try to make your first re:Invent a great success. If I leave you with one thing, these are my top three advice for you:

Don't pack your schedule, leave room for meetings and breaks!
Talk to people, join strangers at lunch, you never know whi you meet!
But most important! Relax and have fun!!!

See you all in Vegas! Send me a message on LinkedIn if you like to meet up!

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

Building a serverless connected BBQ as SaaS - Part 4 - AuthZ

Jimmy Dahlqvist — Tue, 24 Sep 2024 05:47:44 +0000

We’ve already touched on data isolation and row-level security in a multi-tenant SaaS environment. Now, it’s time to look at how to handle authorization, the process of controlling access to data and functionality based on who the user is and what they’re allowed to do.

In this post, we’ll explore the architecture of a centralized Policy Decision Point (PDP) with distributed Policy Enforcement Points (PEPs) to ensure consistent and secure authorization across your SaaS platform. We’ll also break down the key differences between Authentication (AuthN) and Authorization (AuthZ).

If you have not already checked it out, here is part one, part two, and part three

Authentication (AuthN) vs. Authorization (AuthZ)

Before we get into building the architecture we need to establish a common understanding. It’s crucial to distinguish between Authentication and Authorization, two terms that often get mixed up, and that I have had to explain on so many occasions, but serve very different purposes.

Authentication (AuthN)

Authentication is all about verifying identity. It answers the question, Who are you? When a user logs into our SaaS application, authentication ensures that they are who they claim to be. This could involve something as simple as a username and password or more complex multi-factor authentication (MFA).

We are using Amazon Cognito User Pools for our authentication in this series.

Authorization (AuthZ)

In a multi-tenant environment, getting authorization right is important. We need to ensure that users can only access the data and functionality relevant to their tenant and role. This is where a robust, scalable authorization system comes into play.

Centralized PDP and Distributed PEPs: The Authorization Backbone

Now that we’ve clarified the difference between AuthN and AuthZ, let’s focus on building a authorization system using a centralized PDP and distributed PEPs.

Centralized PDP

A Policy Decision Point (PDP) is where all your authorization decisions are made. It’s the brain of the authorization system, evaluating each request against predefined policies or roles and determining whether to allow or deny the request.
There are some clear benefits with a centralized PDP:

Consistency: Every request is evaluated against the same set of policies, ensuring consistent decision-making across our systems.

Management and Compliance: With all policies managed in one place, updating and audits are made easy. Logging decisions for audit purpose is important for some regulatory compliance.

However, some drawback would be increase latency. Routing every authorization request through a single PDP can slow things down, especially in a distributed system.

Using Amazon Verified Permissions (AVP) a centralized PDP is required.

Distributed PEPs

Policy Enforcement Points (PEPs) are where the authorization decisions made by the PDP are enforced. These points are distributed across our system. Our Lambda based Authorizer is an example of a PEP. Front door of our microservices can then also act as PEP and call our PDP.

We have some benefits running our PEPs in a distributed way.

Reduced Latency: By placing PEPs close to where decisions need to be enforced, we can reduce the latency, with an caching strategy this can be reduced even more.

Scalability: As your system grows, distributed PEPs ensure that no single point becomes a bottleneck, specially with caching enabled.

Architecture Overview

In our new authorization architecture we will introduce a new authorization service, that will be our central PDP. The logic for authorization will be moved from our previous Lambda Authorizer to the PDP, instead our Lambda Authorizer will now call the PDP that will return access decision.

Create Authorization service (PDP)

The first thing we need to do is to create our Authorization Service (PDP), as shown this will be an API Gateway and a Lambda function. One interesting approach that we could use is to remove the API Gateway and just use Lambda Function URLs. However, for some easier extensibility later I decided to use an API Gateway.


AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Connected BBQ Application Tenant Service
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
    Default: bbq-iot
  UserPoolStackName:
    Type: String
    Description: The name of the Stack with the Cognito User Pool

Globals:
  Function:
    Timeout: 30
    MemorySize: 2048
    Runtime: python3.12

Resources:
  LambdaPDPFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/AuthZ
      Handler: authz.handler
      Environment:
        Variables:
          JWKS_URL:
            Fn::ImportValue: !Sub ${UserPoolStackName}:jwks-url
          AUDIENCE:
            Fn::ImportValue: !Sub ${UserPoolStackName}:app-audience
      Events:
        AuthZ:
          Type: Api
          Properties:
            Path: /authz
            Method: post
            RestApiId: !Ref PDPApi
            Auth:
              AuthorizationType: AWS_IAM

  PDPApi:
    Type: AWS::Serverless::Api
    Properties:
      Name: !Sub ${ApplicationName}-pdp-api
      StageName: prod
      EndpointConfiguration: REGIONAL
      Cors:
        AllowMethods: "'GET,PUT,POST,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"
      Auth:
        DefaultAuthorizer: AWS_IAM

Outputs:
  PDPApiID:
    Value: !Ref PDPApi
    Description: The ID of the PDP API
    Export:
      Name: !Sub ${AWS::StackName}:pdp-api-id

Our API will use AWS IAM for machine-2-machine authorization, just as our admin API that we created in our previous part.

Next we move our AuthZ logic from our Lambda Authorizer to our new PDP implementation. We will handle both authorization for user and tenant access and introduce two new resources tenant and tenantUser.


import os
import json
import jwt
from jwt import PyJWKClient


def handler(event, context):
    data = json.loads(event["body"])
    jwt_token = data["jwt_token"]

    try:
        jwks_url = os.environ["JWKS_URL"]

        jwks_client = PyJWKClient(jwks_url)
        signing_key = jwks_client.get_signing_key_from_jwt(jwt_token)

        decoded_token = jwt.decode(
            jwt_token,
            signing_key.key,
            algorithms=["RS256"],
            audience=os.environ["AUDIENCE"],
        )

        # Check resource and make an AuthZ decision
        if data["resource"] == "tenant":
            tenant_id = data["tenant_id"]
            token_tenant_id = decoded_token.get("tenant")

            if token_tenant_id == tenant_id:
                response_body = generate_access(
                    "Allow", data["action"], data["resource"]
                )
                return {
                    "statusCode": 200,
                    "body": json.dumps(response_body),
                    "headers": {"Content-Type": "application/json"},
                }
        elif data["resource"] == "tenantUser":
            user_id = data["user_id"]
            token_user_id = decoded_token.get("cognito:username")

            if token_user_id == user_id:
                response_body = generate_access(
                    "Allow", data["action"], data["resource"]
                )
                return {
                    "statusCode": 200,
                    "body": json.dumps(response_body),
                    "headers": {"Content-Type": "application/json"},
                }

    except Exception as e:
        print(f"Authorization error: {str(e)}")

    # Generate a default response that deny access
    response_body = generate_access("Deny", data["action"], data["resource"])

    return {
        "statusCode": 403,
        "body": json.dumps(response_body),
        "headers": {"Content-Type": "application/json"},
    }


def generate_access(effect, action, resource):
    auth_response = {
        "effect": effect,
        "action": action,
        "resource": resource,
    }
    return auth_response

Next we of course need to update the Lambda Authorizer to call our PDP for authorization.


api_endpoint = os.environ.get("PDP_AUTHZ_API_ENDPOINT")

def handler(event, context):
    token = event["headers"].get("authorization", "")

    if not token:
        raise Exception("Unauthorized")

    token = token.replace("Bearer ", "")

    try:

        ......

        path_tenant_id = event["pathParameters"]["tenantId"]

        session = boto3.Session()
        credentials = session.get_credentials().get_frozen_credentials()

        region = os.environ["AWS_REGION"]
        auth = AWS4Auth(
            credentials.access_key,
            credentials.secret_key,
            region,
            "execute-api",
            session_token=credentials.token,
        )

        data = {
            "jwt_token": token,
            "resource": "tenant",
            "action": "read",
            "resource_path": event["path"],
            "tenant_id": event["pathParameters"]["tenantId"],
        }
        headers = {"Content-type": "application/json"}
        response = requests.post(
            api_endpoint + "/authz", data=json.dumps(data), headers=headers, auth=auth
        )

        ......

    except Exception as e:
        print(f"Authorization error: {str(e)}")

Summary

With those changes we have moved our authorization to a centralized PDP with a distributed PEP setup. This create a good foundation for us to keep evolving our AuthZ. In later posts we will introduce a RBAC (Role Based Access Control) and ABAC (Attribute Based Access Control), we will also move to use Amazon Verified permissions.

Get the code

The complete setup with all the code is available on Serverless Handbook

Final Words

In this post, we looked at the architecture of a centralized authorization system using a PDP and distributed PEPs. We’ve also highlighted the differences between Authentication (AuthN) and Authorization (AuthZ).

In the coming posts we will start looking at device onboarding and data, stay tuned!

Check out My serverless Handbook for the code to the solution built in this series of posts.

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!

Building a serverless connected BBQ as SaaS - Part 3 - Tenants

Jimmy Dahlqvist — Fri, 23 Aug 2024 13:09:13 +0000

The time has come for Part 3 in the series of creating a Serverless Connected BBQ as SaaS. In this third post we'll look into tenant creation, authentication, and authorization. We will create a new tenant service and tie it together with the user service from part two.

If you have not already checked it out, here is part one and part two

Tenants in a SaaS

In a Software as a Service (SaaS) solution, a tenant is the organization or individual that subscribe to and use the service.

One of the most crucial parts of an multi-tenant SaaS solutions, is tenant data isolation, ensuring that each tenant’s data remains separate and secure from others within the same environment. This is especially important in maintaining data privacy, complying with regulations, and protecting against data breaches.

Approaches for Tenant Data Isolation in AWS

To isolate tenant data there are several different approaches we can use.

Dedicated database / data store per tenant

Each tenant has a separate database instance or data store, like S3. This provides the strongest isolation since each tenant’s data is entirely segregated at the database / data store level. In this approach we can also use separate KMS keys for each tenant, making sure that data is secured even on an encryption level.

Some of the challenges with this approach would be higher cost due to multiple database instances. Scaling can become complex as the number of tenants grows.

Table per tenant

Each tenant has a separate table in the database. For a data store like S3 each tenant can have a unique prefix that data is stored under. To use separate KMS keys data need to be encrypted on the client level and not on the storage level, which can add even more isolation.

This approach provides a good compromise between cost and isolation. and easier to enforce data isolation at the table level.

Some of the challenges with this approach are that it still requires careful management of table names and schema evolution. Performance could be impacted.

Row-Level Security (RLS)

Data for all tenants is stored in a single table, but with row-level access controls to ensure that each tenant can only access their own data. This access control can be on the data storage level with built in RLS or on a client level with authorization checks with services like Amazon Verified Permissions. For a data store like S3 each tenant still can have a unique prefix that data is stored under.

Some of the challenges with this approach are that it's more complex to enforce correctly; any bugs in logic could expose data. Performance can be impacted by complex query filtering.

What approach should I use?

The approach you use depends on your use case, level of compliance you need, and customer requirements.

In this series of BBQ data I will take the approach and use row-level security.

Architecture Overview

In this architecture we continue to build on the event-driven setup introduced in part 2. When a user signs up, is created, an event will be sent from the user service onto an EventBridge event-bus, this will now invoke the tenant service. The tenant service has two primary DynamoDB tables, one that store the tenant information, ID, name, etc and one that map users access to a tenant. The service will start by creating an tenant ID store it and then map the user to this new tenant.

Our tenant service can be called via two separate APIs. One for our web application that will fetch and update the tenant information, and one admin API that will not only be used by SaaS admins but also for inter service communication. We will introduce the first round of API authorization in this post, IAM for machine-2-machine and Oauth and JWT token validation for the web application API. We will dig deeper into both the APIs and authorization further down in this post.

Tenant creation

First let's create the tenant creation resources needed. Here we create two new DynamoDB tables. In one table we store the tenants and information for it like tenant name etc. In the second table we will store a mapping between tenants and users for access. We also need to query which tenants a user has access to, for this we also setup a index for the table.

AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Connected BBQ Application Tenant Service
Parameters:
  ApplicationName:
    Type: String
    Description: Name of owning application
    Default: bbq-iot
  CommonStackName:
    Type: String
    Description: The name of the common stack that contains the EventBridge Bus and more

  TenantTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub ${ApplicationName}-tenants
      AttributeDefinitions:
        - AttributeName: tenantid
          AttributeType: S
      KeySchema:
        - AttributeName: tenantid
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST

  TenantUserTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub ${ApplicationName}-tenant-users
      AttributeDefinitions:
        - AttributeName: tenantid
          AttributeType: S
        - AttributeName: userid
          AttributeType: S
      KeySchema:
        - AttributeName: tenantid
          KeyType: HASH
        - AttributeName: userid
          KeyType: RANGE
      GlobalSecondaryIndexes: 
        - IndexName: user-index
          KeySchema:
            - AttributeName: userid
              KeyType: HASH
            - AttributeName: tenantid
              KeyType: RANGE
          Projection:
            ProjectionType: ALL
      BillingMode: PAY_PER_REQUEST

When a user is created we need to create the tenant that this user owns. User service will send an event on the application event-bus when a user is created, so here we keep building on a event-driven saga pattern and create the tenant.

Using StepFunctions makes it easy to write to DynamoDB and post an event for the next part to take over.

After the tenant has been created and stored we need to map the first admin user to it, basically the user who owns the tenant. This StepFunction is invoked by the event that the tenant was created.

Final step in this StepFunction also publish to EventBridge for future use.

Let's add these to StepFunctions to our template.


  TenantCreateExpress:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: create-tenant-statemachine/statemachine.asl.yaml
      Tracing:
        Enabled: true
      Logging:
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt TenantCreateStateMachineLogGroup.Arn
        IncludeExecutionData: true
        Level: ALL
      DefinitionSubstitutions:
        EventBridgeBusName:
          Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
        TenantTable: !Ref TenantTable
        ApplicationName: !Ref ApplicationName
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - EventBridgePutEventsPolicy:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
        - DynamoDBCrudPolicy:
            TableName: !Ref TenantTable
      Events:
        CreateTenantEvent:
          Type: EventBridgeRule
          Properties:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
            Pattern:
              source:
                - !Sub ${ApplicationName}.user
              detail-type:
                - created
      Type: EXPRESS


  TenantAddFirstAdminExpress:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: add-tenant-first-admin-statemachine/statemachine.asl.yaml
      Tracing:
        Enabled: true
      Logging:
        Destinations:
          - CloudWatchLogsLogGroup:
              LogGroupArn: !GetAtt TenantAddFirstAdminStateMachineLogGroup.Arn
        IncludeExecutionData: true
        Level: ALL
      DefinitionSubstitutions:
        EventBridgeBusName:
          Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
        TenantUserTable: !Ref TenantUserTable
        ApplicationName: !Ref ApplicationName
      Policies:
        - Statement:
            - Effect: Allow
              Action:
                - logs:*
              Resource: "*"
        - EventBridgePutEventsPolicy:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
        - DynamoDBCrudPolicy:
            TableName: !Ref TenantUserTable
      Events:
        CreateTenantEvent:
          Type: EventBridgeRule
          Properties:
            EventBusName:
              Fn::ImportValue: !Sub ${CommonStackName}:eventbridge-bus-name
            Pattern:
              source:
                - !Sub ${ApplicationName}.tenant
              detail-type:
                - created
      Type: EXPRESS

Comment: Tenant service - Create Tenant On User Created
StartAt: Debug
States:
  Debug:
    Type: Pass
    Next: Genetate Tenant ID
  Genetate Tenant ID:
    Type: Pass
    Parameters:
      tenantid.$: States.UUID()
    ResultPath: $.TenantID
    Next: Create Tenant
  Create Tenant:
    Type: Task
    Resource: arn:aws:states:::dynamodb:putItem
    Parameters:
      TableName: ${TenantTable}
      Item:
        tenantid:
          S.$: $.TenantID.tenantid
    ResultPath: null
    Next: Prepare Event
  Prepare Event:
    Type: Pass
    Parameters:
      tenantId.$: $.TenantID.tenantid
      email.$: $.detail.email
      userName.$: $.detail.userName
      name.$: $.detail.name
    ResultPath: $.TenantData
    Next: Post Event
  Post Event:
    Type: Task
    Resource: arn:aws:states:::events:putEvents
    Parameters:
      Entries:
        - Source: ${ApplicationName}.tenant
          DetailType: created
          Detail.$: $.TenantData
          EventBusName: ${EventBridgeBusName}
    End: true

Comment: Tenant Service - Create Tenant Admin User Mapping
StartAt: Debug
States:
  Debug:
    Type: Pass
    Next: Create Tenant User Mapping
  Create Tenant User Mapping:
    Type: Task
    Resource: arn:aws:states:::dynamodb:putItem
    Parameters:
      TableName: ${TenantUserTable}
      Item:
        tenantid:
          S.$: $.detail.tenantId
        userid:
          S.$: $.detail.userName
        name:
          S.$: $.detail.name
        email:
          S.$: $.detail.email
        role: admin
    ResultPath: null
    Next: Prepare Event
  Prepare Event:
    Type: Pass
    Parameters:
      tenantId.$: $.detail.tenantId
      email.$: $.detail.email
      userName.$: $.detail.userName
      name.$: $.detail.name
      role: admin
    ResultPath: $.TenantUser
    Next: Post Event
  Post Event:
    Type: Task
    Resource: arn:aws:states:::events:putEvents
    Parameters:
      Entries:
        - Source: ${ApplicationName}.tenant
          DetailType: adminUserAdded
          Detail.$: $.TenantUser
          EventBusName: ${EventBridgeBusName}
    End: true

APIs

The tenant service have two separate APIs. First of all an API that is used by the web application to load, display, and update information about the tenant. This API will be central in coming parts as we extend how users get access to tenants. The second API is a management admin API that can be used for special service and system integration.

For the application API we will use the tokens issued by Cognito to authorize the users, and for the management API we will use machine-2-machine authorization with AWS Iam, at least for no.

Both of the APIs will be backed by Lambda functions for implementation of the business logic.

To start let us create the application API. Users will call the API with the token they got from Cognito. API Gateway have three Lambda integrations that will fetch the correct data, and we have a Lambda Authorizer to validate the token.

Let's start by adding the API definition to the template, we will use AWS::Serverless::Api resources and these MUST be defined in the same template as the Lambda functions for us to easy create the integration.


  TenantAPi:
    Type: AWS::Serverless::Api
    Properties:
      Name: !Sub ${ApplicationName}-tenant-api
      StageName: prod
      EndpointConfiguration: REGIONAL
      Cors:
        AllowMethods: "'GET,PUT,POST,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"
      Auth:
        AddDefaultAuthorizerToCorsPreflight: false
        Authorizers:
          LambdaRequestAuthorizer:
            FunctionArn: !GetAtt LambdaApiAuthorizer.Arn
            FunctionPayloadType: REQUEST
            Identity: 
              Headers:
                - Authorization
          LambdaUserRequestAuthorizer:
            FunctionArn: !GetAtt LambdaApiUserAuthorizer.Arn
            FunctionPayloadType: REQUEST
            Identity: 
              Headers:
                - Authorization
        DefaultAuthorizer: LambdaRequestAuthorizer

Next we create the Lambda functions that will back out application API.

  LambdaTenantInfoGet:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/Api/TenantInfoGet
      Handler: get.handler
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref TenantTable
      Environment:
        Variables:
          DYNAMODB_TABLE_NAME: !Ref TenantTable
      Events:
        GetTenantInfoApi:
          Type: Api
          Properties:
            Path: /tenant/{tenantId}
            Method: get
            RestApiId: !Ref TenantAPi

  LambdaTenantInfoPut:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/Api/TenantInfoPut
      Handler: put.handler
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref TenantTable
      Environment:
        Variables:
          DYNAMODB_TABLE_NAME: !Ref TenantTable
      Events:
        PutTenantInfoApi:
          Type: Api
          Properties:
            Path: /tenant/{tenantId}
            Method: put
            RestApiId: !Ref TenantAPi

  LambdaGetTenants:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/Api/TenantsList
      Handler: get.handler
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref TenantUserTable
      Environment:
        Variables:
          DYNAMODB_TABLE_NAME: !Ref TenantUserTable
          DYNAMODB_INDEX_NAME: user-index
      Events:
        GetTenantsForUserApi:
          Type: Api
          Properties:
            Path: /tenants/{userId}
            Method: get
            RestApiId: !Ref TenantAPi
            Auth:
              Authorizer: LambdaUserRequestAuthorizer

Next we can move over to the management API, which will AWS Iam to authorize the calls and have one Lambda based integration to fetch all tenants for a user.

  AdminTenantAPi:
    Type: AWS::Serverless::Api
    Properties:
      Name: !Sub ${ApplicationName}-tenant-admin-api
      StageName: prod
      EndpointConfiguration: REGIONAL
      Cors:
        AllowMethods: "'GET,PUT,POST,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"
      Auth:
        DefaultAuthorizer: AWS_IAM

  LambdaGetTenantsForUser:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: Lambda/Internal/GetTenantForUser
      Handler: handler.handler
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref TenantUserTable
      Environment:
        Variables:
          DYNAMODB_TABLE_NAME: !Ref TenantUserTable
          DYNAMODB_INDEX_NAME: user-index
      Events:
        GetTenantsForUserApi:
          Type: Api
          Properties:
            Path: /tenants/{userId}
            Method: get
            RestApiId: !Ref AdminTenantAPi
            Auth:
              AuthorizationType: AWS_IAM

Cors cors cors everywhere

Let me start by quoting the one and only Eric Johnson. If CORS had a face I would punch it in its nose that basically sums up my feeling around CORS.

First of all we need to ser CORS on our API, by specifying AllowMethods, AllowHeaders, AllowOrigin on our API resource, this will add OPTIONS allowing preflight fetching from the browser.

TenantAPi:
    Type: AWS::Serverless::Api
    Properties:
    ......
      Cors:
        AllowMethods: "'GET,PUT,POST,DELETE,OPTIONS'"
        AllowHeaders: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"

If you set a authorizer on the API resource and you don't explicit set AddDefaultAuthorizerToCorsPreflight: false authorization will be added to OPTIONS which in most cases will make the preflight fail generating a cors error in the browser.

But, since we use a proxy Lambda integration setting cors on the API is not enough. The Lambda function is responsible for setting and returning the cors headers, so we need to add them to the return in our functions as well.

return {
    "statusCode": 200,
    "body": json.dumps(response["Items"]),
    "headers": {
        "Access-Control-Allow-Origin": "*",  # Allow all origins
        "Access-Control-Allow-Headers": "Content-Type,Authorization",
        "Access-Control-Allow-Methods": "GET,POST,OPTIONS,PUT,DELETE",  # Allowed methods
    },
}

User authorization

In this part we slowly start to introduce APIs and for that we of course need some form of user authorization. We will use Lambda Authorizer on the public web application API. In the first version and fairly primitive functionality we will rely on the User Pool enriching the JWT token with a tenant ID.

Cognito Pre Token Generation

To add custom claims and enrich the JWT token we hook into the User Pool authentication flow and use the Pre Token Generation hook. Right now I'm using the version 1 of this hook which will add the claims to the ID Token. It was not that long ago that the possibility to customize the access token was added. In later parts we will expand on this and move over to version 2 of the event which will customize both the ID and access token.

During the process we will call the tenant API and fetch the tenant ID for the user and add this to the token.

Now, when using the version 1 event it's still only possible to add custom string values, it's not possible to add arrays, which I feel is a bit of a drawback.

For machine-2-machine authorization we will rely on AWS IAM, which means we need to sign our requests using sigv4.

import boto3
import os
import json
import requests
from requests_aws4auth import AWS4Auth

api_endpoint = os.environ.get("TENANT_API_ENDPOINT")

def handler(event, context):
    user_id = event["request"]["userAttributes"]["sub"]
    try:
        # Get AWS credentials
        session = boto3.Session()
        credentials = session.get_credentials().get_frozen_credentials()

        # Set up AWS4Auth using the credentials
        region = os.environ["AWS_REGION"]
        auth = AWS4Auth(
            credentials.access_key,
            credentials.secret_key,
            region,
            "execute-api",
            session_token=credentials.token,
        )

        response = requests.get(api_endpoint + "/tenants/" + user_id, auth=auth)

        if response.status_code == 200:
            tenants = response.json()["tenants"]

            event["response"]["claimsOverrideDetails"] = {
                "claimsToAddOrOverride": {"tenant": tenants[0]}
            }
            return event
        else:
            print(f"Error fetching tenant: {response.status_code}")

    except requests.RequestException as e:
        # Handle any exceptions that occur during the API call
        print(f"Error making API request: {str(e)}")

    return event

Custom Lambda Authorizers

To authorize the calls and ensure that the user making the call to fetch tenant data, actually has access to that tenant, we use a custom Lambda Authorizer on our API. We will rely on the User Pool adding the tenant ID to the JWT token during the authentication process. We will therefor decode the JWT token, and validate the signature of the token, and read out the tenant id from the token. The tenant ID in the token will then be checked against the tenant the user try to access. It's a bit primitive but it works for our current use-case.

In later parts of this series we will create a central authorization service that will validate and authorize calls in all parts of the system.

import os
import json
import jwt
from jwt import PyJWKClient


def handler(event, context):
  token = event["headers"].get("authorization", "")

    if not token:
        raise Exception("Unauthorized")

    token = token.replace("Bearer ", "")
    try:
        path_tenant_id = event["pathParameters"]["tenantId"]

        jwks_url = os.environ["JWKS_URL"]
        jwks_client = PyJWKClient(jwks_url)
        signing_key = jwks_client.get_signing_key_from_jwt(token)

        # Decode the JWT and validate the signature
        decoded_token = jwt.decode(
            token,
            signing_key.key,
            algorithms=["RS256"],
            audience=os.environ["AUDIENCE"],
        )

        token_tenant_id = decoded_token.get("tenant")

        if token_tenant_id == path_tenant_id:
            return generate_policy(
                decoded_token["sub"], "Allow", event["methodArn"], decoded_token
            )

    except Exception as e:
        print(f"Authorization error: {str(e)}")
        raise Exception("Unauthorized")

    # Generate a default policy that deny access
    return generate_policy(
        decoded_token["sub"], "Deny", event["methodArn"], decoded_token
    )

def generate_policy(principal_id, effect, resource, context):
    auth_response = {
        "principalId": principal_id,
        "policyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {"Action": "execute-api:Invoke", "Effect": effect, "Resource": resource}
            ],
        },
        "context": context,
    }
    return auth_response

Dashboard

We extend the dashboard with one more page that show the tenant information, and that have the possibility to set a new name on the tenant.

Get the code

The complete setup with all the code is available on Serverless Handbook

Final Words

This was a post where I go through part three in the series about connected BBQ as Saas and discuss tenants and tenant creation.

Check out My serverless Handbook for some of the concepts mentioned in this post.

Don't forget to follow me on LinkedIn and X for more content, and read rest of my Blogs

As Werner says! Now Go Build!