DEV Community: Aaron Brighton

Pandemic-driven bubble tea availability w/ Amazon Pinpoint, Location Service, Lambda, EventBridge, Step Functions, and more…

Aaron Brighton — Wed, 15 Sep 2021 20:42:19 +0000

In the Toronto area when the COVID-19 pandemic hit last year (spring 2020) one of the consequences (benefits?) was that all businesses where employees could reasonably work from home ended up being forced to do so.

This had an interesting effect on restaurants and the food service industry. In the suburbs food establishments saw a huge increase in sales, people were buying lunch near their homes instead of in downtown Toronto. Unfortunately, this also meant long lines for take out that would normally have taken a mere 5 minutes.

Some food stores went purely to online-ordering only, one of those was a Bubble Tea franchise named “Chatime”, you’ve probably heard of it, it’s apparently the biggest in the world. This actually caused a really frustrating problem for those wanting to order, my wife and I included.

Given the increased volume of orders, the Chatime near us would “turn-on” their online ordering system for a few minutes, wait for it to fill up with orders, and then turn it off again, in a desperate effort for them to keep up (orders apparently still took upwards of an hour to be made). For a week my wife and I tried many times to order, and each time we got to the order submission page… Uber Eats would give up and say the store is no longer open and would happily wipe out our cart at the same time (thanks). Frustrating puts it mildly.

So what do we do? Keep hitting F5 until it’s balled hoping to catch the store when it opens it’s online ordering again? Thankfully no.

Enter AWS.

Situations like this are one of the reasons I love the cloud, and specifically Serverless technologies. In roughly a day, I was able to put together a quick and dirty serverless app that would (effectively) hit F5 for us and then send an SMS message to let us know the moment online ordering opened back up.

It worked! First time we got the SMS message, we succeeded at placing an order!

Enter 2021.

It’s time I clean-up what I had built and put it out there for others to learn from, repurpose, etc…

For those that aren’t interested in a wordy walk-through and explanation, feel free to skip the rest and jump right into the code (a clap and follow would be nice though before you leave, if you’re also feeling generous :)

cdk-serverless-chatime-ordering-helper — https://github.com/aaronbrighton/cdk-serverless-chatime-ordering-helper

For the rest of you, let’s dive into it!

Amazon Pinpoint for Two-way SMS

Last spring, for whatever reason (maybe the feature actually wasn’t available yet, someone please correct me in the comments) I couldn’t find a way to cost-effectively handle in-bound SMS on AWS. So the initial implementation actually leveraged Twilio’s SMS w/ Webhooks and API for the SMS interfacing.

As I was cleaning up the code this past week, I realized I can cut Twilio and use Amazon Pinpoint directly. Which drastically simplifies the setup and explanation.

Amazon Pinpoint is… a flexible and scalable outbound and inbound marketing communications service. You can connect with customers over channels like email, SMS, push, or voice.

Pinpoint is the one piece I wasn’t able to automate using IaC (Infrastructure as Code), so if you’re following along, you’ll have to provision an SMS enabled long code phone number manually. This is pretty straight forward in the the AWS Management Console.

Subscriber Lambda and Amazon Location Service

Once an SMS message is received by the phone number in Amazon Pinpoint, it is relayed to some custom logic by way of an Amazon SNS (Simple Notification Service) topic. This is how Two-way SMS is integrated with downstream services in Pinpoint.

Amazon Location Service

Another external service that I ended up replacing as part of this cleanup was the Google Maps API. One of the things the app needs to do is convert a users Postal Code (Canadian) to LAT/LONG Coordinates. In the initial implementation I used Google Maps for this. At re:Invent 2020, Amazon announced Amazon Location service — so time to leverage this as well.

The first SMS message the end-user sends to this app will be their Postal Code. As you can see from the above diagram we then hit an external API hosted by Chatime. Their API expects a user to provide LAT/LONG Coordinates, so first we do the translation with Amazon Location service and then POST to Chatime’s Location API. We get back a list of Chatime locations near the user’s coordinates, as well as information like the Uber Eats online ordering URL. Code excerpt below:

Registering the store for monitoring

There’s a second exchange to be had between the app and the end-user before we start monitoring. We return them a list of the 3 closest Chatime locations that have an Uber Eats online store. They then have to respond back with the Store ID for the store they want to monitor / order from.

Once the selected Store ID is received by the app (it will get thrown at the Subscriber Lambda as well). We create an Amazon SNS topic for that Chatime store, with the Uber Eats URL as a tag (for “safe keeping”). Lastly, we subscribe the end-user to this SNS topic using “sms” as the protocol.

Probing Part 1: Amazon EventBridge and Populator Lambda

The next part of our application is the first half of the monitoring piece. We want to probe Uber Eats Chatime store pages at least once a minute. Therefore, we use our trusty “serverless cron” AKA Amazon EventBridge (formerly CloudWatch Rules).

Populator Lambda

The populator Lambda will scan the previously created SNS topics representing Chatime stores to be monitored, and will populate an Amazon SQS (Simple Queuing Service) queue with messages containing the Uber Eats URL (from the SNS Topic tags), and the SNS Topic ARN itself.

Probing Part 2: Amazon SQS and Worker Lambda

In the final steps, our Amazon SQS is configured to “fan-out” messages to a Worker Lambda with a batch size of 1 (meaning only 1 SQS message at a time will be sent to the Lambda — we don’t want to overload ;) the external Uber Eats store pages.

Our Worker Lambda throws a GET for the store page, and checks to see if there are any occurrences of the word “Currently unavailable” — this is what happens when a store shuts off their online ordering capability.

If the occurrence is found, the Lambda simply exits to try again ~60 seconds later. If the occurrence is not* found then we publish to the SNS topic we created earlier — which will result in an SMS message being sent to anyone who wanted to be notified if this store’s online order came online.

Probing Part 3: AWS Step Functions and Unsubscriber Lambda

Lastly, we want to remove the end-user’s subscription as we’ve now done our part. One of the “gotchas” of SNS Topics is that if you publish to them and then immediately try to delete those topics, there’s a chance not all of the notifications will be sent-out.

Deleting a topic might prevent some messages previously sent to the topic from being delivered to subscribers. — AWS SDK for Javascript (“deleteTopic”)

So we have a couple options:

Sleep the Worker Lambda for ~30 seconds
AWS Step Functions

Sleeping the Lambda is probably the first thought someone would have, but it’s not great for a number of reasons. One of those is you’re paying every ms the Lambda is running… so that’s just financially wasteful.

With Step Functions we can define a “Wait” step, say 30 seconds, before executing additional custom logic (our unsubscriber Lambda).

Unsubscriber Lambda

Our final step of this app is to remove the SNS topic completely along with all of it’s subscriptions, that’s what this Lambda does, the code is very simple.

How do I deploy this myself?

I won’t go into too much detail, as it’s largely already documented in the cdk-serverless-chatime-ordering-helper repo’s README.md.

If you have experience working with CDK (AWS Cloud Development Kit), then it should be fairly straightforward. If you don’t have experience with CDK, I highly recommend you familiar yourself with it! It’s a game changer for cloud development, especially serverless architectures.

There’s a really awesome low time-investment CDK workshop here: https://cdkworkshop.com/

To conclude… what does it look like for the end-user?

Thanks for reading!

If you found the content valuable, please press that heart button and follow, comment what you liked or didn’t like and let me know what I should cover next!

Is it possible to PASS an AWS Exam without studying?

Aaron Brighton — Tue, 15 Jun 2021 19:19:58 +0000

TL;DR: Yes. Do I recommend it? No. Do you need actual experience in the subject matter area? Of course. Do the exams still validate knowledge? Yes!

Even with our best of intentions, life sometimes gets in the way and things don’t go according to plan. If you’re like me, there’s a million things you’d rather be doing than studying, and so I tell myself, tomorrow, I’ll study tomorrow… and so on, procrastination. Next thing you know, your exam starts in 5 minutes, and you’ve suddenly become the creature in the picture above.

That creature was me yesterday at 9:25 AM, and this is my story of prepping for re-certification of my AWS Certified Solutions Architect — Professional certification.

What‘s on the AWS-PSA exam?

Like many professional certification exams, you’re required to agree to an NDA before starting the exam. Essentially dis-allowing you from sharing specifics about the exam questions with others, for obvious reasons. Therefore, I’ll be purposefully vague…

Abilities Validated by the Certification

(copied from AWS Certification Site)

Design and deploy dynamically scalable, highly available, fault-tolerant, and reliable applications on AWS
Select appropriate AWS services to design and deploy an application based on given requirements
Migrate complex, multi-tier applications on AWS
Design and deploy enterprise-wide scalable operations on AWS
Implement cost-control strategies

I’ve highlighted a couple of the points above, as the questions that stood out to me on the exam seemed to be heavily skewed towards these two areas.

Everyone who takes the exam gets a set of questions randomly chosen from a large pool, and therefore, this was my experience, YMMV.

Migrate complex, multi-tier applications on AWS

There was a significant number of questions focused on strategies for migrating on-premise workloads to AWS.

Focus was on how to migrate workloads in the most cost-effective way and/or with the little to-no impact to end-users of the applications/workloads.

Design and deploy enterprise-wide scalable operations on AWS

A good number of questions were about large (enterprise?) AWS environments, with multi-account strategies with 100s or 1000s of AWS accounts.

Often these questions were fixated around services like AWS Organizations, AWS Control Tower, and surprisingly AWS Direct Connect.

Overall content and comparison to prior exams / versions

Coming out of the exam I was surprised by how much of it was focused on large enterprise and organizational strategies and data migrations.

In the past I’ve certified under AWS Certified Solutions Architect — Associate, AWS Certified Security — Specialty. I’ve also wrote this AWS-PSA exam previously (older version), three years ago.

Comparing this experience to those prior exams:

While it’s been a while since I wrote the previous AWS-PSA and AWS-ASA exams, if my memory serves (which it probably doesn’t), it seems that AWS may be differentiating the AWS-ASA and AWS-PSA exams not necessarily based on more advanced expertise but rather on a different set of problems.

This exam was almost assuredly more focused on big enterprise problems than the exams in the past. It’s possible AWS-ASA is more focused on smaller/medium sized company problems and AWS-PSA is for larger enterprises. If you disagree with this assessment I’d love to hear it in the comments!

My exam experience

One of the nice things to have become more common due to the reality of the pandemic is being able to write certification exams from your home instead of a stressful exam center.

This necessitates a proctor “investigation” of your room via webcam before they release the exam to you.

Prior to the exam I made sure to remove any questionable items from my desk, and thought this would be good enough:

Turns out I was wrong, the first 30 minutes prior to the exam was spent dismantling my desk.

Plants had to go.
Keys had to go.
Cables and USB adapter for my keyboard & mouse had to go.
Monitors had to be completely unplugged.
Even my water had to go.

After the exam, I double checked the list of rules for the environment where you write the exam, and sure enough… Beverages are listed as prohibited.

In a prior exam I had wrote from home, I remember being interrupted middle of the exam and being told not to cover my mouth with my hand (I think they use facial recognition tech to flag whether you’ve left view of the webcam during the exam).

Time limit

One major part of the experience that if I could warn others about is the time limit! I was given 75 questions, and 3 hours (180 minutes) to answer them, which sounds like a lot of time, but it can disappear quickly! I ended up only getting through 73 questions, and no time left to review the questions I flagged throughout.

In the past I’ve heard others recommend using time strategically, and I probably should have followed the advice:

Read the answers first.
Read the “ask” part of the question second.
If you still need additional context, read the actual scenario third.

On the AWS-PSA exam, majority of the questions are scenario based (similar to the practice exams), and you can easily waste time overthinking and trying to visualize/comprehend the scenario when the ask/options don’t require you to.

How did I pass without studying, and why does this still make the exams valuable?

Certification exams validate one thing: knowledge.

There are two ways to acquire knowledge, studying and experience.

I had a couple things going for me heading into this exam:

I passed this exam previously (earlier version).
I’ve been working closely with AWS for more than 6 years.

Value of the exams

If you think about it fundamentally, the exam is there to validate your knowledge and mindset.

If you’ve worked closely with AWS across a range of services successfully for years, it’s likely you’d have soaked up the knowledge and experience to put you in the right mindset to answer architectural questions correctly.

I can prove this point thanks to my own past failings. In 2015, I rushed myself into my very first AWS exam (AWS-ASA), I had a month or two of limited hands-on experience, and I studied no-where near enough. Not surprisingly, I failed this exam. After more hands-on experience with AWS, I retried the next year, the result was much better!

Should you still study for an AWS exam, anyways?

Regardless of your proficiency with AWS and their service offering, don’t do what I did. Instead, study!

It’s nearly impossible (unless someone can prove me wrong) to have experience with every AWS service (there are over 200 as of this writing) not to mention the broad set of features each offer. New services pop-up so frequently which means keeping up to date with the entire AWS ecosystem is darn near impossible.

The end-goal of studying should not be to achieve a certification, but rather to expand the knowledge you have. In the tech industry this is especially critical, as we’re focused on optimization problems and some of the best optimization opportunities are thanks to services we aren’t yet aware of — until you study.

So yes, study.

If you’re planning to take an AWS exam, I’d encourage you to checkout a past article I wrote about resources I used to prep for the AWS Security — Specialty exam: 2020 Preparation Guide for AWS Certified Security — Specialty

In closing

If you’re planning to maintain your AWS certifications and you’ve been working in the industry with AWS services on a daily basis, don’t stress too much.

Definitely set aside time to study, but know this — AWS isn’t testing whether you know how to study effectively. They are testing to see if you know what it is you already do, and that you’re keeping yourself familiar with what’s new.

I encourage you to post your feedback in the comments below! If you found this article insightful or helpful in anyway, feel free to reach out directly, I love hearing from people who find value in my content!