DEV Community: Jakob Ondrey

Why Am I Getting Access Denied in One Region S3 When I Have Permissions? (the answer is interesting)

Jakob Ondrey — Sat, 17 Aug 2024 17:50:31 +0000

Sometimes I do contract work helping companies with their transitions to AWS. It is great experience and a wonderful way to level up in services that you may not typically encounter in your "real" job.

This story of errors recently occurred on one such contract. The client is in a highly regulated industry and required me to use an Amazon WorkSpaces virtual desktop that the company had provisioned for me.

I installed the AWS CLI on my workstation, authenticated and ran the first command I always run on a new install and CLI login: aws s3 ls which is short and usually shows that I am successfully logged in (and is easier to type than aws sts get-caller-identity).

 $ aws s3 ls
Access Denied
 $

I begrudgingly ran aws sts get-caller-identity which showed me that I was, in fact, authenticated and assuming the appropriate role.. So I checked the AWS Web Console where I could see the buckets. Strange..

I then added the region flag to the S3 command:

 $ aws s3 ls --region us-east-1
Access Denied
 $ aws s3 ls --region us-east-2
bucket1 (in us-east-1)
bucket2 (in us-east-1)
bucket3 (in us-east-1)
etc...
 $

Odd. What is going on? I looked in IAM at the role and confirmed it had the s3:ListAllMyBuckets permission and there was no SCP preventing the action in us-east-1. But wait!

There was an inline policy intended to only allow requests from known company IPs.

{
    "Sid": "RestrictIPs",
    "Effect": "Deny",
    "Action": "*",
    "Resource": "*",
    "Condition": {
        "NotIpAddress": {
            "aws:SourceIp": [
                "workspaces_public_range",
                "other_corp_ranges"
            ]
        }
    }
}

I checked my IP from the workspace and it was within the allowed range. I checked my IP from the CLI and it was the same... What on Earth was going on here.

I reached out to the client's full time Infrastructure people but they don't have the issue. Even when they assumed my role they were unable to reproduce. We conclude that there appears to be nothing that should block my request.

Finally I go to the CloudWatch logs and looked at some of my denied requests.

The requests to --region us-east-2 came from the expected IP address and succeed. But requests to --region us-east-1 are not coming from the IP that is expected and also not one that is in the allowed IP range! Well, that's why they were failing, but why were those coming from a different IP?

Oh! That unfamiliar IP looks like a private IP address! Why was it using a private IP address for S3 calls to us-east-1? I checked the WorkSpaces VPC in the WorkSpaces account and THERE IT WAS! An S3 Gateway VPC Endpoint in us-east-1 that keeps S3 traffic internal in that region to save on data egress costs!

I recreated these conditions in my own account and verified that the blocking was the combination of the SourceIP rule and the S3 Gateway. Removing either of these resulted in a successful request. Per the documentation:

You can't use the aws:SourceIp condition in an identity policy or a bucket policy for requests to Amazon S3 that traverse a VPC endpoint. Instead, use the aws:VpcSourceIp condition.

Note: You can also use the aws:SourceVpc condition.

Modification of the policy to allow requests from the VPC looks like this:

{
    "Sid": "RestrictIPs",
    "Effect": "Deny",
    "Action": "*",
    "Resource": "*",
    "Condition": {
        "NotIpAddress": {
            "aws:SourceIp": [
                "workspaces_public_range",
                "other_corp_ranges"
            ]
        },
        "StringNotEqualsIfExists": {
            "aws:SourceVpc": "vpc-id"
        }
    }
}

And that addition resolved the issue.

I shared my findings and solutions with the infrastructure team (and shared the documentation outlining the need to allow the VPC) and in short order I was back in business!

TLDR: If you have roles that Deny based on SourceIP conditions, be aware that using AWS compute to make their requests, (in my case an Amazon WorkSpace connecting to S3 in one region through a S3 Gateway VPC Endpoint) may not use their public IP and be blocked.

AWS SSO Aliases you need to have

Jakob Ondrey — Tue, 21 Nov 2023 22:34:17 +0000

Hey kid.. yeah you.. Are you tired of adding --profile <something> to every CLI command? Stop doing that. You can use these easy aliases to set a "context" for your CLI to use until you clear it or change it really easily.

Make sure you have your ~/.aws/config file configured properly for AWS SSO . Then go to your shell's "rc" file (eg. .bashrc or .zshrc) and add the following:

Now you can quickly hop back and forth between different accounts and not type out extra stuff.

Is it not working? Run that awsclear alias you just made and get rid of that dumb .aws/credentials file. It's almost 2024 and you don't need long lived keys anymore! (at least comment it all out while you test)

Note: Make sure you know what account you are in at all times by getting into the habit running the alias for that account when you aren't sure.

Playing with PartyRock, AWS's GenAI Playground

Jakob Ondrey — Thu, 16 Nov 2023 17:26:49 +0000

AWS just came out with a new Generative AI playground which makes it a breeze to experiment with its Bedrock service!

Why is it great that a GenAI playground exists? glad you asked! let me tell you a little story...

A couple of months ago I built a serverless assistant backed by chatGPT. The project itself took a couple days in the process I had to sign up for an OpenAI account so that I could get API access, give them my credit card so that they could charge me for tokens, and figure out how to use their Python library.

That was all to just send my initial system message to ChatGPT so that I could test "programming" ChatGPT to "only answer in rhyming couplets no matter what the user says." It worked and was really fun, but that is a pretty high bar for creating a simple chat bot.

Well, with AWS PartyRock I did basically the same thing in 60 seconds. All you need is a free Builder account (no credit card needed) and you can use the PartyRock playground to test out some of the different models Amazon Bedrock has on offer. You can even publish and share the things that you make with others! No infrastructure to manage and plenty of credits to get you going.

Building A PartyRock App

When I first got access to the tool I was super excited to start playing, but my 1st grader had just gotten back from school and needed some math homework help. I am pretty decent at math (and supremely confident in my ability to do 1st grade math) but the way math is taught today is a little different from the way it was taught to me. It feels like a different language and sometimes that can make me feel stupid.

So I told the initial prompt that I wanted to have "new math" explained to me in "old math" terms so that I could help and communicate with my kid about their homework.

That was it. PartyRock took that prompt and built a simple interface that allowed me to ask questions about number lines and ten frames and put things into perspective for "old math" me.

Play with it if you want here, but better yet, make your own at PartyRock.aws. If you don't have a Builder account you can just sign up with SSO from an account you probably already have and then you just.. play. No friction. Do it. Now!

(and yes, I built another little PartyRock app to make the banner image based on the post title that you can play with here because why not make the machines work for you?!)

Building a Serverless ChatGPT Powered Resume Assistant - Deploying a Front End

Jakob Ondrey — Tue, 20 Jun 2023 13:32:37 +0000

If you have been following along, we now have our back end deployed to AWS and can interact with it through the CLI on our computer. But that really isn't something you can show friends or share as a portfolio project. Now we will build out a (very) simple front end that we can deploy to interact with our resume assistant Lambda function. Lets get to work.

A simple web page

Lets get started in the root of CDK project by making a static_site directory that will hold our website files. For now we can just put a basic index.html file in there.

mkdir static_site
touch static_site/index.html

<!-- static_site/index.html -->

<!DOCTYPE html>
<html>
  <head>
    <title>It's alive!</title>
  </head>
  <body>
    <h1>Hello from S3</h1>
  </body>
</html>

S3 Bucket for Front-End

Now that we have a simple index for our site, we will need an S3 bucket to store it in, and a way to deploy the code from our repo to that bucket (much in the same way we needed a Lambda and a way to package and deploy it.)

Bucket

The bucket is simple enough. We will ensure that the objects are encrypted and non-public, and set an index. We will also set the removal_policy and auto_delete_objects to the values that make cleanup easier. Note that we are also setting a value for website_index_document.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_s3 as s3
from aws_cdk import RemovalPolicy

from constructs import Construct

class ResumeAssistantStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        site_bucket = s3.Bucket(self, "SiteBucket",
            versioned=True,
            encryption=s3.BucketEncryption.S3_MANAGED,
            enforce_ssl=True,
            block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
            removal_policy=RemovalPolicy.DESTROY,
            auto_delete_objects=True,
            public_read_access=False,
            website_index_document="index.html",
        )

Note: all future additions to the CDK code (minus module imports) will be done inside that __init__ function in the ResumeAssistantStack class at the same indentation level as site_bucket above. So that means if you copy and paste code blocks you will have to do some indentation yourself.

Bucket Deployment

This is how we automate the movement of code from out static_site directory to out bucket.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_s3_deployment as s3deploy

s3deploy.BucketDeployment(self, "DeployWebsite",
    sources=[s3deploy.Source.asset("./static_site")],  
    destination_bucket=site_bucket,
)

OAI and CloudFront Distribution

It's not enough to have just the bucket and the deployment. We need that sweet s at the end of http and to make an OAI that can actually access the objects in our bucket (we did block access to it after all). We will also set a ttl of 0 seconds so that CloudFront doesn't cache while we are actively developing (we can change that later if we release this to the world).

Also we might as well have an output so that we can see the CloudFront URL of our page.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_cloudfront as cloudfront
from aws_cdk import Duration
from aws_cdk import CfnOutput


oai = cloudfront.OriginAccessIdentity(self, "OAI",
    comment="OAI for Resume Assistant"
)

distribution = cloudfront.CloudFrontWebDistribution(self, "Distribution",
    origin_configs=[
        cloudfront.SourceConfiguration(
            s3_origin_source=cloudfront.S3OriginConfig(
                s3_bucket_source=site_bucket,
                origin_access_identity=oai,
            ),
            behaviors=[
                cloudfront.Behavior(
                    is_default_behavior=True,
                    default_ttl=Duration.seconds(0),
                    min_ttl=Duration.seconds(0),
                    max_ttl=Duration.seconds(0),
                )
            ]
        )
    ]
)

CfnOutput(self, "WebsiteURL", value=distribution.distribution_domain_name)

Note: CloudFront distributions take a while to provision and propagate. Don't be surprised if you are waiting for 10+ minutes or so for the CDK to finish deploying. You can get a sneak preview of what the CloudFront URL will be by going to CloudFront in the console.

Woohoo!!

Route53 and ACM Certificate (Very Optional)

Right now you have a website at the URL <bunchacrap>.cloudfront.net and if you ask me, that is good enough for a little proof of concept site. But if you want to have it accessible from a domain name you might own it's easy enough to add some CDK to make the site accessible from subdomain.yourdomain.com

I find it very convenient to have a wildcard certificate outside of IAC for use in projects like this. A wildcard certificate is one that is valid for *.yourdomain.com and can be used for any number of subdomains. it makes projects like these easier to share with people and add to a portfolio. I think it should be outside of your IAC because you don't want to mistakenly tear down a project where the certificate is managed and break all of your other projects that depend on it!

We can set that up in AWS Certificate Manager by requesting a certificate for *.yourdomain.com and then adding the CNAME name and CNAME value as a CNAME record wherever that domain is managed. Once you have that certificate validated you can export its ARN and the domain and subdomain you want to use. The dexample below would be if you wanted the address to be resume.yourdomain.com

export CERTIFICATE_ARN='arn:aws:acm:us-east-1:123456789100:certificate/some-uuid'
export DOMAIN_NAME='yourdomain.com'
export SUB_DOMAIN='resume'

You can then use it in your CDK like you have been using the OpenAI API key. We will import the certificate object and then use it in our CloudFront distribution

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_certificatemanager as acm
from aws_cdk import aws_cloudfront as cloudfront
from aws_cdk import Duration
import os

cert = acm.Certificate.from_certificate_arn(self, "cert", os.environ.get("CERTIFICATE_ARN", "empty"))

domain = os.environ.get("DOMAIN_NAME", "empty")
sub_domain = os.environ.get("SUB_DOMAIN", "empty")

distribution = cloudfront.CloudFrontWebDistribution(self, "Distribution",
    origin_configs=[
        cloudfront.SourceConfiguration(
            s3_origin_source=cloudfront.S3OriginConfig(
                s3_bucket_source=site_bucket,
                origin_access_identity=oai,
            ),
            behaviors=[
                cloudfront.Behavior(
                    is_default_behavior=True,
                    default_ttl=Duration.seconds(0),
                    min_ttl=Duration.seconds(0),
                    max_ttl=Duration.seconds(0),
                )
            ]
        )
    ],
    viewer_certificate=cloudfront.ViewerCertificate.from_acm_certificate(
        certificate=cert,
        aliases=[f"{sub_domain}.{domain}"],
        security_policy=cloudfront.SecurityPolicyProtocol.TLS_V1_2_2021,
        ssl_method=cloudfront.SSLMethod.SNI,
    )
)

Finally you need to add an A record to your hosted zone to send traffic that arrives for resume.yourdomain.com to the CloudFront URL. You can do that manually if you are using a DNS provider that is not Route53 (or can also do it manually if your provider is Route53 I suppose..) but lets manage that with the CDK if we can.

You will need to get and import one more environmental variable and that is the ID of the yourdomain.com hosted zone.

Export that:

export HOSTED_ZONE_ID='NUM83R5AND13TT3R5'

Then we can add that A record.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_route53 as route53,
from aws_cdk import aws_route53_targets as route53_targets,
import os

domain = os.environ.get("DOMAIN_NAME", "empty")
sub_domain = os.environ.get("SUB_DOMAIN", "empty")

dns_zone = route53.HostedZone.from_hosted_zone_attributes(self, "HostedZone",
    hosted_zone_id=os.environ.get("HOSTED_ZONE_ID", "empty"),
    zone_name=domain,
)

route53.ARecord(self, "SiteAliasRecord",
    zone=dns_zone,
    record_name=sub_domain,
    target=route53.RecordTarget.from_alias(
        route53_targets.CloudFrontTarget(distribution)
    ),
)

There! Now you have a friendly URL for your site. But it isn't much of a site yet is it? Lets start building that out.

Building a Front End

Note: If You are a front end guru and love to write React or some other high-speed front end framework that can be served statically you can just stop paying attention and do that (remember to modify te S3 deployment to point to the appropriate directory).

I, however, am terrible at that stuff. So for those of you remaining, we are about to write some janky HTML/JavaScript/CSS that was the product of a conversation I had with ChatGPT. Its not very good, but gets the job done. You have been warned!

Lets Write Some HTML!

So the first thing we are going to need are two text boxes to allow users to add their resume and job descriptions to and a button to submit that data to our Lambda for our initial request. We will also need a place for the assistant's response as well as a place to input text for our future replies.

Lets start in the <body> of our index.html by making a div where our elements will live.

<!-- static_site/index.html -->
<html>
  <body>
    <div>
      <textarea id="resume-input" placeholder="Add resume here"></textarea>
      <br>
      <textarea id="jd-input" placeholder="Add job description here"></textarea>
      <br>
      <button>Submit Documents</button>
      <br>
      <div id="api-response"></div>
      <br>
      <textarea id="additional-input" placeholder="Response"></textarea>
      <br>
      <button>Reply</button>
    </div>
  </body>
</html>

A nice base but our buttons don't do anything yet. Lets fix that by adding some onclick properties and loading the javascript file we are about to make.

<!-- static_site/index.html -->
<html>
    <body>
        <div>
            <textarea id="resume-input" placeholder="Add resume here"></textarea>
      <br>
      <textarea id="jd-input" placeholder="Add job description here"></textarea>
      <br>
      <button onclick="handleSubmit()">Submit Documents</button>
      <br>
      <div id="api-response"></div>
      <br>
      <textarea id="additional-input" placeholder="Response"></textarea>
      <br>
      <button onclick="handleReply()">Reply</button>
    </div>
    <script src="script.js"></script>
  </body>
</html>

Some initial JavaScript

Lets add that script.js file to our static_site directory and review what our Lambda Function is expecting.

Initial Submission:

A resume at resume:
A job description at jd:
An empty uuid at uuid:
An empty reply at user_response:

Followup Submissions

An empty resume at resume:
An empty job description at jd:
A uuid at uuid:
A reply at user_response:

This is why we will be creating both a handleSubmit and handleReply function so that we can form the appropriate request to our Lambda. Lets work on the initial submission first.

We first set the values of chatUuid and apiURL and then create the function that the initial request will be sent into. There we will load the values in the text boxes into variables and then create our initial payload for the first request to our Lambda.

Because the call to the Lambda will be the same we will create a second function to pass that payload to that actually calls the API and posts the response to the api-response div.

// static_site/script.js

let chatUuid // this is initialized but left empty
let apiURL = 'The lambda url from your cdk stack'


function handleSubmit() {
    var resumeInput  = document.getElementById('resume-input').value;
    var jdInput = document.getElementById('jd-input').value;

    var initalPayload = {
        'resume': resumeInput,
        'jd': jdInput,
        'uuid': '',
        'user_response': ''
    };
    callAPI(initalPayload);
};

function callAPI(requestPayload) {
    fetch(apiURL, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
        },
        body: JSON.stringify(requestPayload)
    })
    .then(function(response) {
        return response.json();
    })
    .then(function(data) {
        chatUuid = data.uuid;
        document.getElementById('api-response').textContent = data.message;
    })
    .catch(function(error) {
        loadingElement.style.display = 'none';
        console.log('Error:', error);
    });
};

Remember that you will have to refresh or even open in a private window if browser caching is happening. It can also be helpful to add a console.log("fixed misspelled url") sort of message to make sure you are getting the version you think you are. Go ahead and test by just adding some sparse information and see what happens.

Obviously we have some work to do with our styling. But lets get the reply part working first.

We will add a the handleReply function and format the request in the way that the Lambda is expecting.

// static_site/script.js

let chatUuid // this is initialized but left empty
let apiURL = 'The lambda url from your cdk stack'


function handleReply() {
    var additionalInputValue = document.getElementById('additional-input').value;

    var replyPayload = {
        'resume': '',
        'jd': '',
        'uuid': chatUuid,
        'user_response': additionalInputValue
    };
    callAPI(replyPayload);
};

function callAPI(requestPayload) {
    document.getElementById('api-response').textContent = '';
    document.getElementById('additional-input').value = '';

    fetch(apiURL, {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*'
        },
        body: JSON.stringify(requestPayload)
    })
    .then(function(response) {
        return response.json();
    })
    .then(function(data) {
        chatUuid = data.uuid;
        document.getElementById('api-response').textContent = data.message;
    })
    .catch(function(error) {
        loadingElement.style.display = 'none';
        console.log('Error:', error);
    });
};

Note that in addition to the handleReply() function that we added, we also added a couple lines to callAPI() to clear out the assistant's last response as well as the reply we just submitted. That should clean things up a bit.

We could also do a bunch of things like remove the initial input items upon our fist submission so that the user doesn't accidentally restart the conversation. We could also add some fancy loading spinners so that the used doesn't think that their request isn't coming back. I will leave that for you to improve and implement on your own.

But we should make it a little bit prettier... First lets slightly modify index.html to import the stylesheet and add some class names to the two divs.

<!-- static_site/index.html -->

<!DOCTYPE html>
<html>
  <head>
    <title>It's alive!</title>
    <link rel="stylesheet" type="text/css" href="style.css">
  </head>
  <body>
    <div class="container">
      <textarea id="resume-input" placeholder="Add resume here"></textarea>
      <br>
      <textarea id="jd-input" placeholder="Add job description here"></textarea>
      <br>
      <button onclick="handleSubmit()">Submit Documents</button>
      <br>
      <div id="api-response" class="api-response"></div>
      <br>
      <textarea id="additional-input" placeholder="Response"></textarea>
      <br>
      <button onclick="handleReply()">Reply</button>
    </div>
    <script src="script.js"></script>
  </body>
</html>

Lets get working on that style.css. We will make the main area of the page 75% of the window width and set all the boxes to be that width. We will also make it so the buttons are a little nicer.

/* static_site.style.css */

.container {
    max-width: 75%;
    margin: 0 auto;
    padding: 20px;
}

textarea {
    width: 100%;
    height: 100px;
    resize: both;
    margin-bottom: 10px;
    padding: 5px;
}

button {
    padding: 10px 20px;
}

.api-response {
    margin-top: 20px;
    border: 1px solid #ccc;
    padding: 10px;
    white-space: pre-wrap;
}

Easy (But not really. Like I said, ChatGPT helped bootstrap most of this front end and I have just edited it down into a minimal and straightforward package for your consumption).

Now that you have that deployed yourself give it a nice try. Break it and see what needs to be modified in your Lambda code. Here are some ways you could improve it:

What happens when the OpenAI API is down? What about when you hit your billing limit? (you better have set one!) Add some sensible exceptions.
What happens if the user doesn't enter any information? Add some validations.
Can you make the site more user friendly? Add a loading or "thinking" animation. Collapse the initial user inputs down once they er submitted.
We are saving all sorts of conversations that is probably in violation of GDPR and other laws. Set up lifecycle rules on out bucket to delete conversations as soon as they are over or allow the user to purge their conversation.
Do something else and let me know what you did! I would love to hear about it!

Building a Serverless ChatGPT Powered Resume Assistant - Deploying to AWS

Jakob Ondrey — Tue, 20 Jun 2023 13:32:23 +0000

Previously we made a neat little command line tool that we could use to help us build a better resume. Now we will be deploying that application to the cloud!

I will be using the AWS CDK because I love it and it makes things like this relatively easy. You could certainly adapt these instructions to Terraform if you wanted to. First lets get a CDK project going and then we can talk about what we will need to provision for this part of the project.

Start a new CDK project

Create a new project directory and cdk init the project.

mkdir resume-assistant
cd resume-assistant
cdk init app --language python

Set up a virtual environment and install packages.

python3 -m venv .venv
source .venv/bin/activate   
pip install -r requirements.txt

Now that we have our repo set up, here are the things we are going to be putting in it.

A directory with our Lambda code (we will be modifying it a bit)
CDK code to create an S3 bucket to store our conversation "state"
CDK code to create a Lambda Function to run our code
CDK code to create a function URL so that we can call the function

Lets get started!

A new place for your python application to live

Create a new folder at the root of your CDK repo called chat_lambda and make a copy of your app.py and requirements.txt files from the chat_app directory from our previous session there.

We are essentially forking that code and will be modifying each of them in different ways to get to where we want to go. This chat_lambda directory is where the CDK is going to be told the code for a lambda function lives. Assuming you have docker running (make sure you have docker running) the CDK will create a container based on the runtime you specify, install the dependencies in requirements.txt, add your code, and then zip the whole thing up and deploy it as a Lambda function. Its pretty cool. Let's start Lambda-fying our code.

First, for some recent reason we need to decrease

Tradition dictates that we enter into the function at a lambda_handler method that takes some parameters (specifically an event parameter). Lets refactor with the lambda_handler and expect that our request that we are going to be sending is going to be in the body of the event.

# chat_lambda/app.py

import openai
import os
import json
import logging

openai.api_key = os.environ.get('OPENAI_API_KEY')

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def lambda_handler(event, context):
    logger.info(event)

    payload = json.loads(event["body"])
    resume = payload["resume"]
    jd = payload["jd"]

    messages = [
        {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferrable and soft skills."},
    ]
    messages.append({"role": "user", "content": f"Candidate information: {resume}"})
    messages.append({"role": "user", "content": f"Description of the job they want: {jd}"})

    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
    )

    assistant_response = completion.choices[0].message.content
    logger.info(assistant_response)

    return {
        "statusCode": 200,
        "headers": {
            "Access-Control-Allow-Headers": "*"
        },
        "body": json.dumps({
            "message": assistant_response
            })
    }

So you will note that we also added some logging. Because it is at the INFO level we will not be able to see it in stdout if we were running this locally, but it WILL log in Lambda and be available in CloudWatch to help us troubleshoot if we did something wrong. And we usually do!

You will also note that the function will be looking for the resume and job description as the values of resume and jd in the body of the event. When we call this function we will need to make sure we supply them.

Finally, we are returning the assistant's response in a way that will not really allow us to continue this conversation. That will come later.

S3 Bucket for state

The bucket is simple enough. We will ensure that the objects are encrypted and non-public, and set an index. Because we are just playing around, we will also set the removal_policy and auto_delete_objects to the values that make cleanup easier. We will make this before the function itself so that we can pass in its identity to the function as an environmental variable.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_s3 as s3
from aws_cdk import RemovalPolicy

class ResumeAssistantStack(Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        chat_log_bucket = s3.Bucket(self, "ChatLogBucket",
            versioned=True,
            encryption=s3.BucketEncryption.S3_MANAGED,
            enforce_ssl=True,
            block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
            removal_policy=RemovalPolicy.DESTROY,
            auto_delete_objects=True,
            public_read_access=False,
        )

Note: all future additions to the CDK code (minus module imports) will be done inside that __init__ function in the ResumeAssistantStack class at the same indentation level as chat_log_bucket above. So that means if you copy and paste code blocks you will have to do some indentation yourself.

Lambda Function

The star of this application! We will need to refactor a number of things in the body of the code of the function ( the one that we "forked" before), but first, lets create the function infrastructure in the CDK. To get the CDK to actually build and package your lambda, you need to use an alpha module.

# requirements.txt

aws-cdk.aws-lambda-python-alpha

Install it with

pip install -r requirements.txt

Now you will be able to use both modules in your CDK code.
Lets also add permission for your lambda to read and write to that last bucket.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_lambda_python_alpha as p_lambda
from aws_cdk import aws_lambda as _lambda
from aws_cdk import Duration
import os

chat_lambda = p_lambda.PythonFunction(self, "ChatFunction",
    runtime=_lambda.Runtime.PYTHON_3_10,
    timeout=Duration.seconds(60),
    memory_size=256,
    entry="chat_lambda",
    index="app.py",
    handler="lambda_handler",
    environment={
        "CHAT_LOG_BUCKET": chat_log_bucket.bucket_name,
        "OPENAI_API_KEY": os.environ.get("OPENAI_API_KEY", "empty")
    },
)
chat_log_bucket.grant_read_write(chat_lambda)

There are a number of things to note in the above code. First, the lambda doesn't have much memory provisioned for it. That is probably okay. It really isn't going to be doing much computing, just sending requests to OpenAI and then passing on the response. Second, we are setting OPENAI_API_KEY from our local environmental variables because I'm going to be building and deploying this locally and therefore it will have access to my environmental variables. If you were going to build and deploy this from a pipeline or something, you would have to make sure that your pipeline was also loading that value from a pipeline secret.

This is your second reminder to PLEASE remember to set really low usage limits for yourself in the OpenAI billing tab. You likely won't ever spend more than a dollar or so, but if you make this public, then you don't want a billing surprise.

Function URL

Because the time it is going to take for the OpenAI API to respond to us in a lot of cases (eg. when we tell it to use all of the context we have given it to parse a fully formatted resume) we can't use API Gateway which times out after 29 seconds. We are going to instead use Lambda Function URLs. We will leave CORS open for now, and be sure to make an output of that function URL.

# resume_assistant/resume_assistant_stack.py

from aws_cdk import aws_lambda as _lambda
from aws_cdk import CfnOutput

function_url = chat_lambda.add_function_url(
    auth_type=_lambda.FunctionUrlAuthType.NONE,
    cors=_lambda.FunctionUrlCorsOptions(
        allowed_origins=["*"],
        allowed_headers=["*"],
    )
)

CfnOutput(self, "ChatFunctionURL", value=function_url.url)

Now, your S3 bucket, function and function URL should be deployed to AWS and you should have the address of your function URL.

Lets Start Testing

We are now going to test our function and function URL. Lets go back to the python project that we had from before we copied it into the CDK project and modify it so that it calls our Lambda URL instead of the OpenAI API.

We are going to start simple and set those resume and jd values as the files we have, send them to the Lambda via the Lambda URL and see if we get a coherent response.

# chat_app/app.py

import requests

# Add your function URL from your CDK output here
function_url = "https://th1si5n0tmyactua1lam8daur1.lambda-url.us-east-1.on.aws/"

with open("resume.txt") as f:
    resume = f.readlines()

with open("jd.txt") as f:
    jd = f.readlines()

response = requests.get(function_url, json={ # this will be added to the event body
    "resume": resume,
    "jd": jd
})

print(response.json()["message"])

And my response:

⚡  python app.py
Can you please provide me with more details on your responsibilities as a team member at Old Navy? Also, what were some of your duties as a crew member at McDonald's? This would help me understand your previous roles and identify the transferable skills you have that align with the requirements mentioned in the job description.

Nice! So we are now sending that initial request to our Lambda, it is adding our system message and sending that initial completion request to OpenAI, parsing the response and sending it back to us!

But we need to continue a conversation

Yes, we will need to refactor a little bit in order to hold a conversation. We will need a way of storing that conversation (the S3 bucket) and a way to identify which conversation there is the one we are having now.

Here is the plan for that. We are going to refactor our Lambda to look for some things in addition to the jd and the resume. We are going to add to the body of our request some additional information: a uuid and a user_response.

The uuid key will hold an empty string "" as its value when we initiate the conversation and this will tell the Lambda that it is a new conversation and to generate a new uuid to track the conversation. This uuid will then be returned to the user in the response so that the user can pass it back to the Lambda with its next request allowing it to continue the conversation.

This will also allow us to store (and retrieve) the state of the conversation by putting (and getting) an object named <uuid>.txt in the S3 bucket we made.

The user_response key will initially also hold an empty string as its value at first (because the first message will just contain the resume and jd), but will hold the replies of the user as the conversation continues.

Lets first modify the requirements of the Lambda code so that it can use AWS's boto3 python library.

# chat_lambda/requirements.txt

boto3>=1.26

Now we can use boto to work with the S3 bucket in our Lambda.

# chat_lambda/app.py

import openai
import os
import json
import logging
import boto3
import uuid

openai.api_key = os.environ.get("OPENAI_API_KEY")
CHAT_LOG_BUCKET = os.environ.get("CHAT_LOG_BUCKET")
s3 = boto3.client('s3',
    region_name=os.environ.get('AWS_REGION', "us-east-1"))


logger = logging.getLogger()
logger.setLevel(logging.INFO)

def load_file_from_s3(bucket, key):
    response = s3.get_object(
        Bucket=bucket,
        Key=key,
    )
    return response['Body'].read().decode('utf-8')

def save_file_to_s3(bucket, key, data):
    response = s3.put_object(
        Bucket=bucket, 
        Key=key, 
        Body=data,
    )
    return response

def lambda_handler(event, context):
    logger.info(event)

    payload = json.loads(event["body"])
    chat_uuid = payload["uuid"]
    resume = payload["resume"]
    jd = payload["jd"]
    user_response = payload["user_response"]

    if chat_uuid == "": # this is a new chat
        chat_uuid = str(uuid.uuid4())
        logger.info(f"Creating new uuid: {chat_uuid}")
        messages = [
            {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferrable and soft skills."},
        ]
        messages.append({"role": "user", "content": f"Candidate information: {resume}"})
        messages.append({"role": "user", "content": f"Description of the job they want: {jd}"})
        logger.info(save_file_to_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt", json.dumps(messages)))

    else: # this is an existing chat
        logger.info(f"Loading uuid: {chat_uuid}")
        messages = json.loads(load_file_from_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt"))
        messages.append({"role": "user", "content": user_response})


    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages,
    )

    assistant_response = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response})
    logger.info(save_file_to_s3(CHAT_LOG_BUCKET, f"chat_history/{chat_uuid}.txt", json.dumps(messages)))

    return {
        "statusCode": 200,
        "headers": {
            "Access-Control-Allow-Headers": "*"
        },
        "body": json.dumps({
            "message": assistant_response,
            "uuid": chat_uuid,
            })
    }

You should primarily note the addition of the save and load to S3 methods and how they make use of uuid value and how the function flows depending on if it is a new conversation without a uuid or a conversation follow-up that arrives with a pre-existing uuid.

Now lets modify our python application that calls the Lambda to pass in the required information and loop over the request to our Lambda until we exit out. Note I purge the resume and jd values after the initial submission. We don't need to be flinging around all that extra data.

# chat_app/app.py

# Add your function URL from your CDK output here
function_url = "https://th1si5n0tmyactua1lam8daur1.lambda-url.us-east-1.on.aws/"

with open("resume.txt") as f:
    resume = f.readlines()
with open("jd.txt") as f:
    jd = f.readlines()
user_response = ""
chat_uuid = ""
while user_response not in ["quit", "Quit", "stop", "Stop", "exit", "Exit"]:
    response = requests.get(function_url, json={
        "resume": resume,
        "jd": jd,
        "uuid": chat_uuid,
        "user_response": user_response
    })
    print(response.json()["message"])
    chat_uuid = response.json()["uuid"]
    user_response = input("\nResponse:\n")
    resume = ""
    jd = ""

When we run this we will see that we can continue our conversation. Users are not exposed to any of the interesting stuff that you may have going on in the Lambda like your system message. Yay!

So now that we have the back-end of this application working in the cloud, our final post will be setting up some sort public static site that we can let users use instead of forcing them to use the command line! Woop woop!

Building a Serverless ChatGPT Powered Resume Assistant - The Python Function

Jakob Ondrey — Tue, 20 Jun 2023 13:32:08 +0000

While the end goal of this project is to deploy it onto AWS and make it available to the world, we are first going to be making a python function that can be run locally. We can modify it later to run in AWS Lambda. So, even though this code will eventually need to be placed into the directory of a CDK project, we are just going to skip that part and refactor later when we need it. We want to get some of that glorious feedback of some working code as fast as possible!

Setting up the files and virtual environment

So lets make a directory called chat_app that we will first use to run our chat bot locally, and later to communicate with our deployed AWS Lambda function. Once we make the directory we can add the files we need to a small python project:

mkdir chat_app
cd chat_app
touch app.py requirements.txt

Now add the OpenAI library to your requirements.txt. I will be pinning the latest version available as I write this, but you can check what the latest version is when you are reading this.

# chat_app/requirements.txt

openai==0.27.8

Please note that I will try to be clear about which file I am working in by adding a comment in all of my code blocks telling you where I am. I will also do my best to, when I am using an external library or module, show the import of that at the top of the file. You can skip over that part in your code if the import has already happened from a previous step.

Now that we have our first external dependency declared in our requirements.txt lets create a virtual environment and install that dependency in the virtual environment. If you plan on committing any of this to a git repo, be sure to add .venv/ to your .gitignore.

python3 -m venv .venv
source .venv/bin/activate   
pip install -r requirements.txt

Now lets get into our python file. We are going to import our modules, set the OpenAPI key that we have gotten from our OpenAI account. (You did do that right? If you haven't please go and do that!)

Add your key an a local environmental variable like this but with your own key:

export OPENAI_API_KEY="sk-Th1si5n0tmyactua1k3y"

Lets test your key with just a single simple request and see what happens:

# chat_app/app.py

import openai
import os

openai.api_key = os.environ.get('OPENAI_API_KEY')

messages = [
    {"role": "system", "content": "You are a Cat named Mr. Boots"},
    {"role": "user", "content": "What is your name?"},
]

completion = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=messages)

print(completion.choices[0].message.content)

So we are creating a conversation of messages primed by that system message and sending a question to the user about what the assistant's name is.

The test response can be accessed at .choices[0].message.content and so we print out that response.

This is what I got as a response when I ran it:

 ⚡  python app.py                                                             
I am a language model AI and my name is OpenAI. But for the purpose of this conversation, you can refer to me as Mr. Boots, the cat. Meow!

You will probably get something slightly different. Language models are like that. But guess what! We have made chat completion work! Now lets turn it into an infinite loop (with some escapes) so that we can have an actual conversation with Mr. Boots.

We will take some input from the user and, assuming it is not one of our escape commands, enter our loop. We will add that input to the list of messages and request a completion. Then we will print and append that response to the list of messages, and await another user response. The user input will determine if the loop is entered again, or if the loop is exited.

# chat_app/app.py

import openai
import os

openai.api_key = os.environ.get('OPENAI_API_KEY')

messages = [
    {"role": "system", "content": "You are a Cat named Mr. Boots"},
]

user_input = input()

while user_input not in ["exit", "Exit", "quit", "Quit", "stop", "Stop"]:
    messages.append({"role": "user", "content": user_input })
    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages)

    assistant_response = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response })
    print(f"Mr. Boots: {assistant_response}")
    user_input = input()

print("Mr. Boots: GoodBye")

This is what happened when I ran it:

 ⚡  python app.py
Hello My name is Mr. Dog. What is your name?
Mr. Boots: My name is Mr. Boots. Nice to meet you, Mr. Dog.
Do you like laying in the sun?
Mr. Boots: Yes, I love to bask in the warmth of the sun. It makes me feel cozy and relaxed. How about you, Mr. Dog? Do you enjoy sunbathing?
exit
Mr. Boots: GoodBye

Our first conversation!

Goodbye Mr. Boots! We have a resume to work on!

So lets think about what sort of things our resume-assistant is going to have to do.

Get prepped with a system message
Get a resume or description of a job seeker (candidate)
Get a job description of some sort
Converse with the candidate to get more information about their experiences
Prepare a better resume

Once we have it in the cloud we will need to add some other things, but we will deal with that in the next post.

# chat_app/app.py

import openai
import os

openai.api_key = os.environ.get('OPENAI_API_KEY')

messages = [
    {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferable and soft skills."},
]
print("Please provide your resume or tell me about yourself")
user_input = input()
messages.append({"role": "user", "content": f"Candidate information: {user_input}"})

print("What job do you want?")
user_input = input()
user_input = f"Description of the job they want: {user_input}"
while user_input not in ["exit", "Exit", "quit", "Quit", "stop", "Stop"]:
    messages.append({"role": "user", "content": user_input })
    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages)

    assistant_response = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response })
    print(f"Mr. Boots: {assistant_response}")
    user_input = input()

print("Mr. Boots: GoodBye")

A simple start:

 ⚡  python app.py              
Please provide your resume or tell me about yourself
My name is Jimmy Jimbert and I worked at McDonalds as a team member from Jan 10 2022 to now. I did all of the jobs there. 
What job do you want?
I would like to be a bank teller. 
Mr. Boots: Great, Jimmy! It sounds like you have experience working in a customer service-oriented role, which is helpful when working as a bank teller. Can you please provide more information about your responsibilities at McDonald's, especially those that could be transferable to a bank teller role, such as handling financial transactions, upholding high levels of accuracy and attention to detail, and working as a team with colleagues and customers? Additionally, do you have any experience handling cash efficiently and accurately? Finally, do you have any additional education or certification related to banking or finance that may be relevant for a bank teller role?
exit
Mr. Boots: GoodBye

First of all, this seems like a great start to me. The AI assistant has picked up on some activities like cash handling and accuracy that would be relevant to both positions. That is the core of what we want it to do.

There are also some changes we need to make, such as we kept the formatting of the assistant responses as coming from "Mr. Boots" hahaha.

Second, it is clear that we will need a better way to provide the assistant with our resume. Pasting a multi-line text block is more work than its worth so it would probably be best to just have it load a text file called resume.txt. Same thing for the job description at jd.txt. That work will also not be wasted when we move on to further steps.

Lets make those files to test with. I am gonna keep trying to get Jimmy a job at the bank but if you are working on this yourself you might as well just use your information and a job you are looking for!

# chat_app/resume.txt

Jimmy Jimbert

Experience:

Old Navy 3/15/2022 - present
Team member

McDonalds 5/20/2020 - 3/1/2022
Crew member

This is from a bank teller job I found on Linkedin.

# chat_app/jd.txt

Bank Teller


PARKWAY BANK

Location: 4106 N Milwaukee Ave.
Chicago, IL. 60641

Hours: Tuesday through Friday 9:00 a.m. – 6:00 p.m.
Saturday 7:45 a.m. - 1:00 p.m.
Hours may change based on the needs of the bank, one day off during the week.

Parkway Bank is looking for a Full-Time Teller at our Six-Corners Branch in Chicago, IL. We are a locally owned and operated financial institution committed to offering the best in customer service, financial products, and services. We maintain a strong presence in our communities through donations, volunteer work, and education. Come be a part of our growing team!

Job Duties:
Responsible for receiving deposits for credit to customer’s deposit and loan accounts (personal and commercial); verifying cash withdrawals and signature endorsements within established limits and securing required approval as necessary; scanning daily work; balancing daily transactions. In addition, tellers are responsible for ensuring quality customer service; maintaining acceptable teller difference records; and referring sales opportunities to the appropriate banking staff. Tellers are required to meet and/or exceed appropriate referral goals and perform other job functions as assigned by supervisor/manager.

Requirements:


    High school diploma, GED or currently pursuing.
    Three months cash handling
    Effective customer service and sales skills
    Good math and clerical skills
    Ability to effectively communicate (oral and written) with fellow employees and customers.
    Must be accurate and detail oriented.
    Ability to work varied hours/days as needed.
    Proficient using a PC, keyboard, and computer mouse.
    Basic knowledge of Outlook, Microsoft Word and Excel preferred.
    Ability to walk, sit, stoop, and stand for long periods of time.
    Ability to lift, move, push, or pull up to 25 pounds.


Parkway Bank is an Equal Opportunity Employer
Minority/Female/Disability/Veteran

Now lets modify our python code to import those files, get rid of Mr. Boots (you were a good cat), and add some spacing to the output with some newlines (\n) to make things more readable.

# chat_app/app.py

import openai
import os

openai.api_key = os.environ.get('OPENAI_API_KEY')

messages = [
    {"role": "system", "content": "You are a resume review assistant. You will be provided information about a job candidate. that information might be in the form of a formatted resume, or just a sentence about themselves. You also may also receive a description of a job or position sought by the candidate. Your task as the assistant is to prompt the candidate for additional information they could add to their resume to make it better generally, and more well suited for the job they are seeking specifically. Don't shy away from asking and promoting transferable and soft skills."},
]

with open("resume.txt") as f:
    resume = f.readlines()
messages.append({"role": "user", "content": f"Candidate information: {''.join(resume)}"})

with open("jd.txt") as f:
    jd = f.readlines()

user_input = f"Description of the job they want: {''.join(jd)}"
while user_input not in ["exit", "Exit", "quit", "Quit", "stop", "Stop"]:
    messages.append({"role": "user", "content": user_input })

    completion = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=messages)

    assistant_response = completion.choices[0].message.content
    messages.append({"role": "assistant", "content": assistant_response })
    print(f"\nAssistant:\n{assistant_response}\n")

    user_input = input("User:\n")

print("Assistant: GoodBye")

And when we take it for a spin:

⚡  python app.py

Assistant:
Thank you for sharing your experience, Jimmy. Based on the requirements listed for the Bank Teller position at Parkway Bank, here are some potential areas you could add to your resume to make it more relevant to the job:

- Highlight your experience with cash handling, as it is an important requirement for the job.
- Include any instances where you excelled in customer service, as providing quality customer service is one of the main responsibilities of a bank teller.
- Be sure to mention your proficiency with basic computer programs such as Microsoft Word and Excel, as well as your ability to learn new systems quickly.
- Detail how you have been able to work varied hours or extended shifts in the past, as this is a requirement for the position.
- Are there any sales or referral goals you have met in previous jobs? If so, including that information on your resume would be helpful since tellers are expected to refer sales opportunities to the appropriate staff at Parkway Bank.

Additionally, Parkway Bank seeks individuals who are detail oriented and possess good math and clerical skills. Can you think of any instances where you have been especially detail-oriented or when you have had to use your math skills on the job? These could be good examples to include on your resume.

User:
Im really great and really qualified in all of those things.  add them to my resume and show me what that resume would look like. 

Assistant:
Certainly, Jimmy! Here's a revised version of your resume that incorporates your qualifications, experience, and the requirements from the job description:

**Jimmy Jimbert**

**Experience**

Old Navy | Team Member | 3/15/2022 - Present

- Assisted customers with product inquiries, fitting room requests, and checkout procedures.
- Maintained store appearance by restocking merchandise and organizing displays.

McDonald's | Crew Member | 5/20/2020 - 3/1/2022

- Prepared food and drinks, operated cash register, maintained cleanliness of the restaurant.
- Provided exceptional customer service and resolved customer concerns.

**Education**

High School Diploma | XYZ High School | Graduated June 2020

**Skills**

- Cash handling: Proficient in accurately counting, verifying, and processing cash transactions.
- Customer service: Able to provide friendly, efficient service and resolve customer concerns.
- Sales and referral: Consistently met or exceeded sales goals and referred opportunities to appropriate staff.
- Math and clerical skills: Possess strong skills in performing basic math calculations and maintaining accurate records.
- Computer proficiency: Proficient in using Microsoft Word, Excel, and Outlook.
- Detail-oriented: Meticulous when handling customer transactions and maintaining records.
- Flexible: Able to work varied hours and days as needed.

I hope this helps! Let me know if there's anything else I can do for you.

User:
exit
Assistant: GoodBye

Well that was pretty neat. It was able to fill in Jimmy's experience with those typical of a person working at the perspective jobs and touched on all the requirements in ways that, I would assume, most people could claim without feeling like they were lying.

Next up we will deploy this application to the cloud!

Building a Serverless ChatGPT Powered Resume Assistant - Foundation

Jakob Ondrey — Tue, 20 Jun 2023 13:31:52 +0000

Project Origin

A few weekends ago I was itching to build something with the OpenAI API. As a career changer into the cloud space and someone who works in the early career talent space, I am acutely aware how important and difficult it is for candidates to write targeted resumes for positions they are applying to. Therefore, I decided to make a resume assistant powered by ChatGPT.

There has been a lot of ink that has been spilled about Large Language Models (LLMs) in general and ChatGPT specifically, and I'm not really trying to add to that torrent. One thing that I would like to address is the criticism that LLMs just compose confident and compelling bullshit. Not a great trait for writing an academic paper, but are there applications where that could be an advantage?

A resume, if you think about it, could also be considered a work of confident compelling bullshit. No, I am not saying that lying is acceptable on a resume, but often we find ourselves stretching our experience a little bit while we write our resumes so that we can conform just a little bit better to the requirements of a given job description. Speaking of which..

Job descriptions are also a similar sort of aspirational bullshit. Requirements are based on an old job that doesn't really match the work anymore, or just a description of the experience the last person had (who left to earn 20% more than they will offer you). Thus, together resumes, job descriptions, and LLMs form what I call "The Triad of Truthiness"

Having decided that LMMs are, therefore, the perfect resume assistant, we will now build a serverless application that, given a "resume" and a job description, can tailor the resume for the job through a virtual resume review, ie. a conversation with you.

The Plan

Having written a large portion of this as a single post, and it being entirely too long, I have decided to split it up into 4 posts.

An introduction and some background (this post)
Building a Python function that we can run and test locally from the command line
Using the AWS CDK (Python) to deploy that function to the cloud (but still interacting with it via CLI)
Deploying a front-end so that we can allow others to use the service

This should keep things a little more bite sized and allow anyone who wants to also build it (or put their own spin on it) to generally focus in one file or area rather than jump around like a maniac.

How a Conversation with ChatGPT works

Lets talk a little about the OpenAI library and how ChatGPT conversations work.

First of all, there are three participants in every conversation, assistant - the chatbot, user - the person chatting with the chatbot, and system - a hidden but important participant that sets the tone of the conversation.

Second, that system message is your "secret sauce." The thing that separates your bot from just chatting with ChatGPT. With the system message, you can set the rules and constraints that will apply for the duration of the conversation. It is where we can instruct the chatbot that its a helpful resume review assistant and to "leverage transferable skills if they lack direct experience". It might be something you want to keep secret otherwise someone else could copy your million dollar idea!

Third, the OpenAI API doesn't "remember" anything. Every time you pass it a request, you need to provide ALL the context necessary to generate a relevant completion. That means that the first request might look like this:

[
  {'role':'system','content':'You are a resume bot named Resume Bot 3000'},
  {'role':'user','content':'My name is Jimmy and I want a job as a Bank Teller!'}
]

And your second request might look like this (we will talk about parsing the response in a minute):

[
  {'role':'system','content':'You are a resume bot named Resume Bot 3000'},
  {'role':'user','content':'My name is Jimmy and I want a job as a Bank Teller!'},
  {'role':'assistant','content':'Hi Jimmy, A bank teller is a job that typically requires money handling experience. Have you ever had a job or other experience where you had to handle cash or facilitate transactions?'},
  {'role':'user','content':'Well, maybe not a job but I volunteered at school basketball games selling tickets for admission and also working at the concession stand. Does that count?'}
]

For ChatGPT to understand what is going on, it basically needs the whole conversation every time. Everything is new to it. This means that the longer the conversation goes on the more context is built, and the more expensive your requests become.

gpt-3.5-turbo has a price of $0.0015/1k tokens (about 750 words according to the openAI pricing page) for context you send to it and $0.003/1k tokens for responses. That can be pretty cheap but can also easily get out of hand if you have a lot of people having long conversations. Be sure to set limits on your account in the billing section.

With all of that out of the way now we can move on to building something! Join me in post 2!

Lessons in AWS Python CDK: 3 - Sometimes You Just Need to Learn to Read Typescript

Jakob Ondrey — Sat, 25 Mar 2023 13:53:04 +0000

TLDR
The best way to get AWS CDK Python documentation is to learn to read and translate the Typescript documentation. Really.

Read and navigate through the Typescript docs so that you know what relevant arguments are and what type their values need to be, and then rely on your IDE's autocompletion to get the Python syntax right.

If you use an IDE without autocompletion, click over to the Python version of the docs to get the correct snake casing of the arguments.

You are pretty smart. You will catch on really fast.

About Me
My name is Jakob and I am a DevOps Engineer. I used to be a lot of other things as well (Dish Washer, Retail Employee, Camp Counselor, Army Medic, Infectious Disease Researcher), but now I am a DevOps Engineer. I received no formal CS education but I'm not self taught, because I had thousands of instructors who taught me through their tutorials and blog posts. The culture of information sharing within the software engineering community is vital to everyone, especially those like me who didn't have other options. So, as I learn new things I will be documenting them through the eyes of someone learning for the first time, because those are the people most in need of a guide. Happy Learning! And don't be a stranger.

Note: I am NOT going to be sourcing and fact checking everything here. This is not an O'Reilly book. The language and descriptions are intended to allow beginners to understand. If you want to be pedantic about details or critique my framing of what something is or how it works feel free to write your own post about it. These posts are intended to be enough to get you started so that you can begin breaking things in new ways on your own!

The Problem

Python documentation for the AWS CDK stinks. It's not good at all.

Here is an example of the Python documentation for an EKS Custer. It tells you all of the arguments, and whether they are optional, that you can use to customize your EKS Cluster. What more could you want?

Well what if you want to specify an alb_controller for your cluster? Union[AlbControllerOptions, Dict[str, Any], None] of course.. Wait what? tell me more! No! Python documentation doesn't supply links to anything!

Now let's compare that to the Typescript documentation.

It contains all the relevant information but includes links to all the subclasses that you may need to reference. ALB Controller options? They look like this!

And what of ALB Versions? Whats available? Just click on that link!

They Typescript documentation is just so much more rich.

So here is my advice. Just use the Typescript documentation as your main source. Will it take a little bit of time to get the hang of reading Typescript documentation then writing your CDK code in Python? yeah, but I know you are up to the challenge. A fully featured IDE will help with auto-completion and worst case if a particular class is giving you trouble you can follow the link to the Python docs, resolve your issue, and go back to the Typescript docs.

It is a bummer that the Python documentation for the AWS CDK lacks the interactivity of the Typescript documentation, but by learning to read in Typescript you are not only making your CDK building in Python easier for yourself, but also learning some Typescript along the way!

Deploying a Scratch App to AWS

Jakob Ondrey — Fri, 08 Jul 2022 21:37:34 +0000

Art: Calvin Ondrey

TLDR: Why write in Scratch? Why not? One of the most magical things about computers is seeing something you created available on the web to anyone who cares to use it. This post outlines a simple project I completed with my 10-year-old son to meet a real-world need.

Background:

I have a 10-year-old son who has expressed aspirations to become a software developer. Like most 10-year-olds (and many far older), he is pretty sure that his first crack at the art should be a AAA video game (or at the very least a super successful indie game). Perhaps I am being a bad father by breaking his spirit, but I have been trying to share the glorious feeling of creating software to solve simple, but real, problems.

One game that my kids play with the other kids in the neighborhood is their IRL version of Among Us. Think Wink Murder with an Among Us skin. It’s fun for them and is some great time spent off of screens. But one problem is that the kids need to be assigned the roles of Imposter or Crewmate each round.

Typically that job falls to an adult who would rather not be interrupted every 5 minutes for a new game to discreetly assign roles at random. But isn’t this something that software could do better? And could it be done in the programming language that my 10-year-old is most familiar with?

So we set out to see if we could write an “Imposter Picker” app in Scratch.

Requirements:

Accept input for number of players
Can randomly assign an imposter
Can discreetly inform players of their assignment
Can be deployed to the web

Lets Go!!

1. Accept input for number of players.

We don't want to exclude anyone, so we are going to need to allow users to enter the number of players so that everyone can eventually get an assignment.

This is pretty easy in Scratch, with an "ask __ and wait" block. We will ask "How Many Players?" at the start of the application.

2. Can randomly assign an imposter.

We need to assign exactly one Imposter to a player. Thankfully Scratch has has a "pick random" operator. We can use that operator to set a variable to a number between 1 and the number of players like so.

3. Can discreetly inform players of their assignment.

This is where it gets a little tricky. Scratch has the ability to create loops and we can even use the number of players to dictate the number of loops.

BUT there doesn't seem to be a concept of indexes that are tracked with each loop, so we will have to create a variable to keep track of what number of loop we are on. The variable "Loop" will be synonymous with the number of the player that is getting their assignment.

Okay, now we need to assign the "Imposter" role to the person who gets their assignment at the position that has been selected by the "pick random" operator and assigned to the Number variable.

Easy!

So this will work to assign the roles, but doesn't really take care of the "discreetly" requirement. For that we are going to add a "wait until" step to allow the next person to come to the screen and get their assignment when they are ready.

Here we have added another variable named Wait. It is initially set to 1 meaning it should wait. The sprite tells the user that it is waiting for them to touch it so that it can give the assignment and then waits until the value of Wait = 0. When the sprite is touched, it sets the Wait value to 0 allowing the loop to continue and announce a role.

3.5. Some additional functionality

Finally we added some functionality to allow the picker to loop forever as long as the number of players stay the same. It will still randomly pick an imposter each time, assign the role, and let them know everyone has gotten their roles before it loops pack to assigning the first player for the next game.

Additionally, it adds a second sprite that can be clicked to escape the infinite loop for the purposes of changing the number of players. Nice!

4. Can be deployed to the web.

While you can publish Scratch applications at the scratch website, I wanted my son to be able to remember and share the link. That isn't exactly easy when the link is scratch.mit.edu/projects/<9 digit number>/, so I looked for ways to convert it to html so that I could put it in the S3 bucket that hosts my personal website.

Thankfully first year CS student and sheep enthusiast SheepTester created a tool that packages scratch projects into a single HTML page (which includes the entire scratch engine). It is pretty awesome and can be found here.

After turning my exported project into an HTML file, it was as simple as naming the file and dumping it into an S3 bucket. Find out if you are the imposter with the Imposter Picker.

Conclusion:

Making simple projects is fun. Making projects available to the world is more fun. And realizing that programming languages are just tools, and the tool doesn't matter as long as you can make useful things is the most fun of all.

Automate a Papertrail with AWS Lambda

Jakob Ondrey — Tue, 05 Apr 2022 14:00:28 +0000

Photo by Nana Smirnova on Unsplash

TLDR
The aws_lambda_python_alpha module (or probably any aws_lambda_<language>_alpha module) is really great for writing, building, packaging, and deploying serverless functions with the AWS CDK.

Import Both:

import aws_cdk.aws_lambda as lambda_
import aws_cdk.aws_lambda_python_alpha as lambda_python

Create Lambda Infrastructure with alpha module
Build (w/ docker running) and Deploy:

cdk synth --no-staging
cdk deploy

The CDK will take care of the rest.

The Problem

A couple of months ago I caused quite a stir by pointing out that my local school district (the 3rd largest in the United States) was underrepresenting COVID-19 cases on their School Dashboards in the middle of the Omicron surge.

To summarize: They changed which cases were being displayed without changing the legends of the graphs and charts displaying the data. This resulted in thousands of cases missing that, based on the legends, should have been there. It was very bad work on their part.

One tool that helped me tell the story and counter the school district's excuses were archived snapshots of the district's web pages that had been captured by the Internet Archive's Wayback Machine.

The Solution

If you haven't heard of the Wayback Machine, go check it out. It is essentially a time capsule of web pages and it allowed me to prove that on (at least) certain dates that the district graphs were not showing what they said they were. It was very useful for me.

So what does this have to do with anything? Well, in the spirit of accountability, and wanting to play with the AWS-CDK I built a stack that you can deploy to automate the archival of web pages to the Internet Archive. Using a serverless Lambda function, it's so cost effective it is basically free (you would need to make over a million archive requests per day, every day to go over the AWS Free Tier.

You can find the repo here if you want to take a look or clone it for your own use and I will also outline the juicy bits below, but briefly:

Using Python and the AWS-CDK, I have created a CloudFormation stack consisting primarily of a Lambda function that uses a wayback machine python library to request the archival of a list of urls, and an EventBridge event that triggers the Lambda. The CDK also makes all the accessory roles and permission sets that are required for this all to function.

The Function

The function code (below) sits in a directory with a requirements.txt file. The CDK (with the help of docker) will create a container, install the requirements.txt file, and deploy that artifact for you to AWS. It's quite neat.

 1    from waybackpy import WaybackMachineSaveAPI
 2    import signal
 3    import logging
 4    
 5    logger = logging.getLogger()
 6    logger.setLevel(logging.INFO)
 7    
 8    user_agent = "Mozilla/5.0 (Windows NT 5.1; rv:40.0) Gecko/20100101 Firefox/40.0"
 9    url_list = [
10         "https://api.cps.edu/health/help",
11         ]
12    
13    def handler(sigum, frame):
14         raise Exception("Request Sent")
15    
16    def lambda_handler(event, context):
17         signal.signal(signal.SIGALRM, handler)
18         for url in url_list:
19              signal.alarm(5) # resets alarm
20              try:
21                   save_api = WaybackMachineSaveAPI(url, user_agent)
22                   save_api.save()
23              except Exception:
24                   logger.info("Request Sent: {}".format(url))
25                   continue

At the top import and logging are set as well as the user_agent for the Wayback API. Then there is a list for all the URLs you want to save.

Skipping the handler function, the lambda will enter into the lambda_handler function. the first thing it will do on line 17 is use the signal library to listen for a signal alarm and call the handler function if that alarm reaches 0. This is what will allow us to just throw requests at the wayback machine without having to wait the >30 seconds for a response that the archival is done.

Then we will iterate over url_list resetting a signal alarm to 5 seconds each time and then requesting the url be saved. Meanwhile the signal alarm is counting down and when it reaches 0 the requested exception is raised, the request is logged, and we move on to the next iteration of resetting the alarm and requesting the url be saved.

I think it a nice way to self rate limit, while not having to wait for a response. I suppose you could lower the timer down to 1 second if you really wanted to.

The Stack

The CDK stack is maintained in /wayback/wayback_stack.py.
Note that both aws_lambda and aws_lambda_python_alpha modules are imported and that aws_lambda_python_alpha is imported on its own. That is important.

The stack contains two things:

wayback_function

The function is created with the aws_lambda_python_alpha module. It is used instead of aws_lambda because it will automatically package and even allow local testing should we be interested in that.

16        wayback_function = lambda_python.PythonFunction(
17            self,
18            "wayback",
19            function_name="WaybackArchiver",
20            runtime=lambda_.Runtime.PYTHON_3_9,
21            entry="./wayback_app",
22            index="app.py",
23            handler="lambda_handler",
24            memory_size=128,
25            timeout=Duration.seconds(300),
26            )

Most of the parameters make sense when viewed in context, but I will explicitly point out that entry= on line 21 is the DIRECTORY that will be packaged up by the CDK. If there is a requirements.txt file in this directory all the libraries and modules indicated here will be installed.

index= and handler= on lines 22-23 together tell the CDK which file and function in the entry directory should be the starting point at execution.

Event Rule

This is not named because it isn't referenced anywhere, but this section creates the EventBridge rule that triggers the Lambda. There are only two important things here, schedule and targets.

28        event.Rule(self, "WaybackRule",
29            rule_name="WaybackRule",
30            schedule=event.Schedule.cron(
31                minute="0", 
32                hour="9",
33                ),
34            targets=[
35                targets.LambdaFunction(wayback_function),
36                ],
37            )

line 30 indicates that you will be using the CDK's version of cron notation. Note that the timezone is UTC and everything here is represented by a string. In addition, parameters that are not stated default to *. So, for example, not including day= above would mean all/any days of the month.

line 34 is where you can specify what you want to happen when this event occurs. In this case it is referencing the wayback_function named above.

Building and Deploying

The stack can be built, including the packaged lambda (assuming docker is running) and deployed with the following commands:

cdk synth --no-staging
cdk deploy

Once created you will have a function that will archive a list of web pages (that you don't have to manage storage of) without further direction from you.

Modifications to the stack or the application just need to rebuilt and deployed and the old scheme will be replaced with the modifications.

Now, figure out what web-pages you think could use some oversight and start archiving them. You never know when an external record of there content at a given time will bear fruit!

Lessons in AWS Python CDK: 2-Parameterizing Your Stack

Jakob Ondrey — Sun, 20 Mar 2022 13:40:23 +0000

TLDR
I don't like using the cdk.context.json file. I use a config/<env>.yaml file that holds environment specific values and omegaconf to parse it.

Say you have dev and prod VPCs that already exist and you want to use the same stack to deploy to both.

Files:

config/dev.yaml

vpc:
  vpc_id: vpc-1234567890

config/prod.yaml

vpc:
  vpc_id: vpc-9876543210

In the stack:

from omegaconf import Omegaconf

deploy_env = "dev" # <-- This could/should be set by your CICD pipeline
conf = OmegaConf.load("config/{0}.yaml".format(deploy_env))

# load pre-existing vpc into variable
vpc = ec2.Vpc.from_lookup(self, conf.vpc.vpc_id, vpc_id=conf.vpc.vpc_id)

In typical yaml fashion everything is a dictionary or a list and you can, therefore, navigate and loop. (details below)

The Problem

One thing that I found some difficulty with when I started using the AWS-CDK was how to handle deploying into multiple pre-existing environments. Of course, the CDK makes it easy to create a stand-alone stack with a new VPC, subnets, buckets, and certificates, but sometimes we have a pre-existing environment we need to deploy into. Or perhaps we want to use variations on the same stack to deploy into multiple environments, eg. smaller instances instances for a development environment.

In Terraform, we might pass in a dev.tfvars or prod.tfvars file. With the CDK you can use the recommended cdk.context.json file and pass context dependent parameters into the stack. But after scanning over the documentation for how to add values to the context file, I decided it was too annoying and I wanted a better way.

My Solution

I have settled on pairing a yaml parsing library called omegaconf to make my own .tfvars-like parameter file. Let me show you how it works.

The Setup

First, create yourself come configuration files. I made a folder at the root of the project called /config where I created yaml files named for each environment, eg. dev.yaml, uat.yaml, prod.yaml. Lets also add a couple things to dev.yaml

aws:
  account: "12345678910"
  region: us-east-1
env: Dev

Now, throw omegaconf into your requirements.txt file and pip install it. (you are using a virtual environment right?)

Import it into both the app.py file at the root of your project and any stack files you plan on using the parameters.

from omegaconf import Omegaconf

Implementation

Depending on how you are going to manage deploying to your environments, you are going to load a different config file.

For example, if you are just going to deploy from your local computer, you can set an variable the the deploy environment at the top of your file (so that it is visible and easy to change) and then use that variable to load your config.

 1   import aws_cdk as cdk
 2   from uber_for_cats.uber_for_cats_stack import UberForCatsStack
 3   from omegaconf import Omegaconf
 4   
 5   deploy_env = "dev"
 6   
 7   conf = OmegaConf.load("config/{0}.yaml".format(deploy_env))
 8   
 9   app = cdk.App()
10   UberForCatsStack(app, "UberForCats{0}".format(conf.env),
11       env=cdk.Environment(account=conf.aws.account, region=conf.aws.region),
12       )
13   
14   app.synth()

On lines 5-7 the environment is set to dev and the config/dev.yaml file is set to conf.

If you were using a CICD pipeline to automatically deploy, deploy_env could be set based on a pipeline variable and line 5 could look like this.

 5   deploy_env = os.getenv("DEPLOY_ENV")

But the result will be the same the values set in the config/dev.yaml file will be used and the file will be read as if the strings were there.

 1   import aws_cdk as cdk
 2   from uber_for_cats.uber_for_cats_stack import UberForCatsStack
 3   from omegaconf import Omegaconf
 4   
 5   deploy_env = "dev"
 6   
 7   conf = OmegaConf.load("config/dev.yaml")
 8   
 9   app = cdk.App()
10   UberForCatsStack(app, "UberForCatsDev",
11       env=cdk.Environment(account="12345678910", region="us-east-1"),
12       )
13   
14   app.synth()

This allows you to set other values in config/uat.yaml such as a different account, different sized instances/EBS volumes, autoscaling rules, etc. depending on what is required in each environment.

Something to Remember

Yaml is basically a nested dictionary that can also contain lists. When you see a - before something it is a list and therefore iterable. Take, for example, the following.

aws:
  account: "12345678910"
  region: us-east-1
vpc: 
  vpc_id: vpc-aabbccdd
  subnet:
    private:
    - subnet-65asdf651sadf65
    - subnet-c65as1df65f56sa
    - subnet-afas65df1a6sdf5

Those private subnet IDs are a list and you can iterate over them.

private_subnets = []
for i, subnet in enumerate(conf.vpc.subnet.private):
    private_subnets.append(
        ec2.Subnet.from_subnet_id(self, "pri{0}".format(i), subnet_id=subnet)
    )

This would create a list of subnet objects (ISubnet) that you can use for the placement of an autoscaling group or EKS cluster. Neat!

Congrats, Those are the basics! You should be able to get started.

Some Additional Tricks

While using a yaml file for configuration settings, I ran into some situations that might be worth sharing.

Sometimes strings require extra steps

I am using the CDK to make some EKS clusters. Part of this process is creating node-groups (think autoscaling groups for Kubernetes). I wanted to leverage some spot instances for a portion of our development cluster and part of the process of creating these nodegroups in CDK is specifying the compute size and class. Because this could be different between environments I put it in the config file.

node_group:
  spot:
    min: 1
    max: 5
    type:
    - i_class: BURSTABLE3
      i_size: LARGE
    - i_class: BURSTABLE3
      i_size: XLARGE
    - i_class: COMPUTE6_INTEL
      i_size: LARGE
    - i_class: COMPUTE6_INTEL
      i_size: XLARGE

But that won't work for setting instance types:

instance_type = ec2.InstanceType.of(
    ec2.InstanceClass.conf.node_group.spot.type[0].i_class,
    ec2.InstanceSize.conf.node_group.spot.type[0].i_size,
    )

And, honestly, it isn't very readable either. My workaround was making dictionaries of instance classes and sizes. then using the value in the yaml as the key the the appropriate class/size.

node_group:
  spot:
    min: 1
    max: 5
    type:
    - i_class: t3
      i_size: large
    - i_class: t3
      i_size: xl
    - i_class: c6i
      i_size: large
    - i_class: c6i
      i_size: xl

ec2_class = {
    "t3": ec2.InstanceClass.BURSTABLE3,  # max 2xl
    "c6i": ec2.InstanceClass.COMPUTE6_INTEL,  # min large
}

ec2_size = {
    "large": ec2.InstanceSize.LARGE,
    "xl": ec2.InstanceSize.XLARGE,
}

With the combination of the above, we can now use the string in the yaml as the key to pull in the ec2 objects in the format that the CDK requires. Below I make a list of the specified instance combinations and pass it into the EKS cluster as nodegroup capacity, but you could just as easily specify a list of classes and types and use a python library like itertools to make ALL of the combinations in one line. That might sacrifice readability though. So actually, don't do that. But you could...

spot_instance_types = []
for instance_type in node_group.spot.type:
    this_type = ec2.InstanceType.of(
        ec2_class[instance_type.i_class], 
        ec2_size[instance_type.i_size],
    )
    spot_instance_types.append(this_type)

cluster.add_nodegroup_capacity(
    "{0}-spot-nodegroup".format(conf.env),
    nodegroup_name="{0}-spot-ng".format(conf.env),
    capacity_type=eks.CapacityType.SPOT,
    min_size=node_group.spot.min,
    max_size=node_group.spot.max,
    instance_types=spot_instance_types, # <-- list of instance types
    disk_size=250,
    subnets=ec2.SubnetSelection(subnets=private_subnets), # <-- those subnets from before!
)

Booleans are Useful

So we are promoting this project out to production and someone doesn't think spot instances are a good idea even though you have diversified your spot pools. Throw a trigger into the config.

node_group:
  spot:
    enabled: False
    min: 1
    max: 5

Then you can run your node creation based on it!

if conf.node_group.spot.enabled:
    spot_instance_types = []
    for instance_type in node_group.spot.type:
        this_type = ec2.InstanceType.of(
            ec2_class[instance_type.i_class], 
            ec2_size[instance_type.i_size],
        )
        spot_instance_types.append(this_type)

    cluster.add_nodegroup_capacity(
        "{0}-spot-nodegroup".format(conf.env),
        nodegroup_name="{0}-spot-ng".format(conf.env),
        capacity_type=eks.CapacityType.SPOT,
        min_size=node_group.spot.min,
        max_size=node_group.spot.max,
        instance_types=spot_instance_types, # <-- list of instance types
        disk_size=250,
        subnets=ec2.SubnetSelection(subnets=private_subnets), # <-- those subnets from before!
    )

Beautiful!

Lessons in AWS Python CDK: 1-Getting Started

Jakob Ondrey — Sat, 05 Mar 2022 20:07:02 +0000

TLDR
Install dependencies:

Definitely - brew install aws-cdk
Maybe: - node version manager

Bootstrap account:

cdk bootstrap aws://<Account_ID><AWS_Region>

Make Project:

cdk init app --language python

Install Requirements:

pip install -r requirements.txt

Deploy Project:

cdk deploy

Profit

Install The CDK and it's dependencies

The AWS CDK

Step one is to install the AWS CDK. I am going to be installing AWS CDK v2 and not v1. Newer is better right? Right?? The official docs are here.

Using npm

AWS suggests installing with npm (Node Package Manager). Thats fine. Just go to your terminal and make sure you have it installed by checking for the version. (almost always the best way to see if you have something installed)



npm --version

If you have npm installed you can go ahead and install the cdk.



npm install -g aws-cdk

This will globally install the latest version of the aws-cdk.

Assuming you didn't get any errors, the CDK CLI should now be properly installed and you can check.



cdk --version

If your output is a version number you are good to go. If it looks like THIS, you will need to change your node version. I will show you how to do that after I discuss the other ways to install the CDK.

Using Homebrew

If you are using Homebrew for everything else, you might as well use it to install the CDK as well. You can also use Homebrew to install the CDK because you feel like it. Sometimes it is nice to have choices! Of course verify that you have Homebrew installed by checking the version.



brew --version

Then go ahead and install the CDK with Homebrew.



brew install aws-cdk

Assuming you didn't get any errors, the CDK CLI should now be properly installed and you can check.



cdk --version

Other Ways

I am betting you can use a number of other package managers to install the AWS-CDK. Since you are using Python to write your CDK stacks, I bet you can install it with pip install aws-cdk or something like that. apt install aws-cdk or yum install aws-cdk might get you there too. Feel free to try!

Changing Node Versions ##

Did you get an error about the CDK not being tested on your current done version. Yeah, me too. I don't use node for anything but the CDK (that I am aware of) so to fix this issue I opted to just change my node version to the most recent supported version: 16.3.0. You can do it using a node version manager. That link has more detailed instructions that you can read to install nvm (for OSX / *nix) or nodist (for Windows). I will summarize the nvm instructions because that is what im using.

CAUTION: It is risky to run remote scripts on your machine. When you see something like curl <url> | bash your alarm bells should go off. The command will take the script that you haven't read and execute it on your computer, often with root permissions. Use common sense. Do you trust the source? Have you read the source?

With that.. To install nvm you can follow their instructions here or you can run the two commands below. The remote script you will execute can be viewed first here.

Install:



curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash

Export the directory and Load:



export NVM_DIR="$([ -z "${XDG_CONFIG_HOME-}" ] && printf %s "${HOME}/.nvm" || printf %s "${XDG_CONFIG_HOME}/nvm")"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm

Verify that nvm is installed with vnm --version and use the above links to troubleshoot further if you run into issues.

Now that nvm is working you can switch node versions and get rid of that super annoying warning. I am using node version 16.3.0 because that is the newest one that is supported.



nvm install 16.3
nvm use 16.3
node -v

This should now show that your node version is one that is fully tested with the AWS CDK. Rechecking your CDK version should no longer show the error.

Bootstrapping your account/region for CDK deployments

As I mentioned, CDK is just a great way to use a programming language to build and deploy Cloudformation templates. This means that the CDK will need some infrastructure in place in your account deployment region to work. It will make a bucket for storing your CFn template files and other assets like zipped lambda functions. Bootstrapping will also create a number of roles that the CDK needs to operate.

Get your AWS account Number

If you have your account number handy you don't need to do anything. But if you are unsure or are using a Cloud Playground (you have added your Access Key ID and Secret Access Key to your .aws/credentials file already right?) the easiest way to get your account number is the below sts command.



aws sts get-caller-identity

The output should have the account number of the default account saved at .aws/credentials

Hint: hit q to leave screens like this.

Then you use that account number and your deployment region in the bootstrap command.



cdk bootstrap aws://<AccountID>/<region>

You should see the following satisfying output as the CDK gets busy making things for you.

Once the bootstrapping process is completed you are ready to make your first project. Just be aware that if you update your CDK version (and it is being developed and versioned at a rapid pace) you may need to bootstrap the account/region again.

Also, a special note for those who may be sharing an AWS account, if another person upgrades their CDK version and then updates the bootstrap version, you may be forced to update your CDK version to match. Keep your eyes open for strange errors.

Make a project

Wahoo!! We can finally get started making a project!

To make a CDK project you need to start with an empty directory. Note that the name of this directory will be the default name of your stack. Don't worry, you will be able to change things after the cdk project is first created, but you can do yourself a favor name the directory something meaningful.

So, to repeat, you will make an empty directory and then initialize the cdk project within that empty directory.



mkdir <project-name>
cd <project-name>
cdk init app --language python

This is what it would look like to make my groundbreaking disruptor project I have in the works.

Once the init is done, you will find that a virtual environment has been created and activated for you (.venv) and a git repository has also been initialized. Additionally, the previously empty folder will have a number of assets in it.

The most important ones for you to note will be requirements.txt, app.py, and uber_for_cats/uber_for_cats_stack.py. we will address them in that order.

`requirements.txt`

If you are working in python, you should use a virtual environment for your development. Seriously. Especially if you plan on deploying things to the cloud and having them work there. When you first init your project, the cdk creates and activates a virtual environment for you. It uses .venv I'm sure to keep it at the top of the file structure.

Using a virtual environment allows you to install and import python packages in different versions and varieties for use in your project while not impacting the rest of your system. One project uses version 1 and another uses version 2.2? no problem. The versions that you want of your project are specified in the requirements.txt file. Open up the file and you will see that there are two requirements already! Install them with pip.



pip install -r requirements.txt

Note that the above command can be repeated any time you have an additional dependency or version change.

Now that the requirements are installed, lets move on to the next important file.

`app.py`

The app.py file in the root of your project directory is where everything starts.

There are a number of things to talk about with this file.

os is imported on line 2. This is so that, if you were to comment out line 18 you would be able to pull some environmental variables from whatever is building this stack.
aws_cdk is imported as cdk (remember that is one of the things installed via pip install -r requirements.txt). This is so that the app object can be created in line 9 and the cloudformation template can be created in line 28.
UberForCats class is imported from the other important file we are going to talk about next. The class is scoped the the cdk.App construct and named.
You can change the names of your folders/files and classes if you would like, but make sure the change is reflected in line 6 and maybe line 10. Think of it this way:



from directory/file import class_name

There are a number of commented out arguments in this class that we aren't really going to touch on, but you can read the comments and get the gist of it.

`uber_for_cats.uber_for_cats_stack.py`

This file is the meat and potatoes of the cdk magic.

I have uncommented the sqs queue example that comes with the template so that we can see it easier in the IDE. It contains a couple of things.

Standard imports at the top for the file.
The UberForCastStack class. Now, I am going to be honest with you, I don't mess with the code from line 8 to line 11. I just leave it there and start building the stack below. maybe I'm a coward.
Resources! Line 16 to line 19 create an SQS Queue that will be named UberForCatsQueue with the parameter visibility_timeout set to 300 seconds.
- Documentation for the other parameters is at https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_sqs/Queue.html
- While the queue will have "UberForCatsQueue" in its name in AWS, it is called "queue" in this stack. This will allow us to reference it for other things like if we were to use it for a lambda trigger.

Deploy a Project

Remember that the CDK is used to make a CloudFormation template which CloudFormation then deploys. To view the the CFn template execute:



cdk synth

This will output the CFn template in yaml format on your terminal as well as save it formatted in json at cdk.out/UberForCatsStack.template.json. There is a lot more than what is included on the screenshot below, but that is what the important part is.

You can also run a command that will compare your desired state (the result of a cdk synth) and what is currently saved as the desired state in aws. Think of it like terraform plan.



cdk diff

This one is pretty busy because I don't have ANYTHING in the stack yet.

To deploy the resources you will run trhe following command:



cdk deploy

It may ask you to approve it there are any IAM permissions increases, but otherwise it will synthesize your CFn template, upload it to S3, and then cloudformation will begin deploying it.

After it is complete, we will have the stack and an SQS Queue in AWS.

But Wait!! That isn't the name I wanted to give the queue! Lets add a parameter to the stack, safe the file and run cdk diff

You can see that it is telling is that if we make the changes we are adding a QueueName "UberForCatsQueue" and that will require a replacement of the queue. Note that not all changes would require a replacement.

Thats, better. Now it is easier to read if you are in the console, but this is definitely not needed if you just care about your stack working with itself.

Okay, now that we have had our fun, and because who needs an UberForCats, Lets get rid of all those resources.



cdk destroy

Conclusion

Alright, those are the bones of setting up, deploying, and destroying a CDK project in AWS. There are a number of neater things that I have learned in the course of deploying resources with the CDK and I cant wait to share them with you. But this is already too long. Until next time!