DEV Community

Applying the Well-Architected Framework, Small Edition

Mark Nunnikhoven on June 27, 2020

Do you ever tackle a problem and know that you’ve just spent way too much time on it? But you also know that it was worth it? This post (which is a...
Collapse
 
scgrk profile image
Stephen Gerkin

Excellent post! I'm looking at designing something similar myself (a completely serverless CMS), using S3, DynamoDB, and Lambda, so this was helpful for considering the design.

One recommendation I would make is that your use case seems perfect for AWS CodeBuild. I have used it a little bit for syncing a repo with S3 using webhooks, but I believe you can also set it up to run on a schedule. There is a free tier offering for 100 build minutes per month. It might save you some (minimal) cost, but considering you've fully automated the pipeline already, it's probably worth it to keep what you have.

I haven't had a chance yet to play around with Lambda and EFS, but it is definitely something that piqued my curiosity when I heard about it. Does Lambda require a NAT, or can it be routed through your IGW directly? For enterprise solutions, I imagine this is not a big deal (especially if you don't need to access the internet), but for small personal projects, obviously, this is a huge cost increase and a bit disappointing.

While writing the above, it also occurred to me that CodeCommit might be a good solution to the NAT problem, assuming your repo does not need to be hosted on GitHub. I'm not sure, but I imagine, that you could put up a VPC endpoint for your Lambda and access the CodeCommit repo without traversing the public internet to clone and do your build.

Collapse
 
marknca profile image
Mark Nunnikhoven

Glad the post could help a bit. That's the whole reason for sharing!

Lambdas in a VPC require a NAT to reach the internet. That was the snag as soon as I integrated EFS (which has to be in a VPN).

CodeCommit and CodeBuild are definitely possibilities here as well. @esh pointed that out to me as well. It's been a while since I used CodeBuild, so I have to circle back on it.

I would caution you in building your own CMS unless it's as a project to learn about building in the cloud. There are already a ton fo great options out there (most free and open source) that could save you a ton of time.

That said, as a learning project, it's a fun activity to take on!

Collapse
 
scgrk profile image
Stephen Gerkin

Good to know about the Lambdas and NAT. A bit disappointed, but not entirely surprised. But, because it's a brand new offering, maybe this will change in the future as the service evolves. For an enterprise solution, this is probably not a deal-breaker, but for a student like me, it's definitely cost-prohibitive for portfolio projects I want to keep going on a long-term basis.

The project is definitely just a learning project! I did a few projects in the past with Lambda and DynamoDB and did not completely understand how they work. I have a much better understanding now, but definitely want to learn more. I am also considering playing around with Aurora serverless and a CMS is a fairly simple use case to integrate all of the above and get some experience with them.

Thread Thread
 
marknca profile image
Mark Nunnikhoven

Yes, for work production accounts the NAT gateway cost is usually a drop in the bucket. Though it would be nice to see some sort of slicker solution there for purely serverless setups.

Your CMS efforts sound like a lot of fun. Aurora serverless is very cool and also pretty straight forward. For the DynamoDB piece, have you read Alex Debrie's book dynamodbbook.com/?

Thread Thread
 
scgrk profile image
Stephen Gerkin

I have not, I'll definitely check it out. Thanks for the recommendation!

Collapse
 
rosswilliams profile image
rosswilliams

Code build does not compete well on price, especially on smaller instances. The author would pay at least 10x cost for code build.

Collapse
 
marknca profile image
Mark Nunnikhoven

After a couple people suggested it, I checked into the latest updates around it. You're 100% spot on.

Even if I could drop build time by 50% to 2m, that's $0.01/build * 750 builds a month for a total of $7.50/mth or 10x!

Collapse
 
softprops profile image
Doug Tangren

Nice post! I've been thinking alot recently about the Well-Architected framework and realized it can often be accomplished without needing aws services.

For your goal

Run a python script on-demand and a set schedule

You mentioned a few points about GitHub actions

doesn’t support running on a schedule.

I wanted to share for other readers that it actually is possible to run a python script in GitHub actions on a schedule, for free even!

help.github.com/en/actions/referen...

Have a look. I've used this to replace main cloudwatch scheduled event lambdas for free

Besides, trying to run that code in the action is going to require another event to hook into or it’ll get stuck in an update loop as the new feed items are committed to the repo.

I wanted to clarify for other readers that you can associate a separate action for pushes. A scheduled action to run your script and a push trigger for what ever else you need, filtering on paths or branches of needed.

If this is just static content you could also just host on gh pages for free

To tie these back to the AWS Well-Architected Framework am alternative might be

  • it’s highly performent

Assuming this is a static website, check. A python script will run as fast as a python script can run where ever it runs ;)

  • Low cost

Both GitHub actions and GitHub pages are free.

  • has minimal operational burden

There's no infra to maintain. No lambda lambdas to write, no IAM credentialing to manage. You check a workflow file into your repo and GitHub manages the operations of executing it.

There's also many moving parts which can sometimes remove operational burden.

  • a strong security posture

A secret store is built into GitHub actions if you need it

  • Is very reliable

Same story here. GitHub scheduling system is pretty reliable. If a script fails it gets rerun on the next pass.

Collapse
 
marknca profile image
Mark Nunnikhoven

Sweet! Excellent tips. I'll check it out and see if how I can make that work as well.

As expected (and loved) there are always a bunch of ways to solve the problem.

This one might even come in less expensive than what I've laid out. Not because it's actually cheaper computationally but thanks to GitHub absorbing the cost!

Collapse
 
softprops profile image
Doug Tangren

Agreed. Love the detail and spirit in this post

Collapse
 
aronjohnson profile image
Aron Johnson

Thanks for sharing this! I use the heck out of cloud-init for doing initial config of servers, but never thought to co-opt it to make an ec2 instance into something Lambda-ish. Might come in handy in the future.

Collapse
 
frankdilo profile image
Francesco Di Lorenzo

Great post! I am the developer of Mailbrew and love when people hack on the product.

Collapse
 
marknca profile image
Mark Nunnikhoven

Thanks Francesco, I'm loving Mailbrew and signed up immediately when I saw. Super clean and amazingly useful.

Btw, I completely understand why you wouldn't support OPML in the product as it would be a nightmare on the backend. I had originally included that call out in the post but things were getting way, way too long.

Besides, this was a fun thing to solve!

Collapse
 
rosswilliams profile image
rosswilliams

If we are playing AWS cost golf, why not switch to parameter store and save 40p, 50% cost reduction. I don’t think your solution uses any feature of secret manager that isn’t part of parameter store.

Collapse
 
marknca profile image
Mark Nunnikhoven

I actually get that one a lot and there's a reason not to use Parameter Store and that's intention.

Yes, Secrets Manager is more expensive but if you clearly delineate between sensitive data (Secrets Manager) and configuration options (Parameter Store), you're less likely to run into accidental leaks down the road. It's a nice bit of operational security to help show you and the team what needs a high level of care.

...but yes, Parameter Store is significantly less expensive

Collapse
 
anthonygrear profile image
anthony-grear

Just getting started into AWS and this is great content for me. I like how you explained your decisions in terms of the well-architected framework. Cheers!