DEV Community: Alex

Setup Atom/RSS feed as event source for AWS Lambda invocation with CDK

Alex — Fri, 08 May 2020 14:54:41 +0000

Now everyone has a license to speak, it’s a question of who gets heard.

― Aaron Swartz

Recently I've found myself blogging extensively about my findings when developing simple serverless event-driven applications with the help of AWS Cloud Development Kit (CDK).

In a previous blog post for example I shared the outcomes of developing publish-to-social, a simple app that gives me a convenient and extensible way to update (almost all...) my social media feeds whenever there's a new blog post published and ready to be shared.

In that article I observed that the 100% automation goal was not met though and left the completion for the future, hinting at one possible solution like polling the RSS (actually Atom) feed of this blog using Python Feedparser library.

Filling the gap

As a start I've thrown the feedparser Python module into the bunch creating a dedicated Lambda Layer for that. In CDK is very convenient to add a Lambda Function dependency via Lambda Layers: with just a bit of configuration by convention (keeping all the Layers content under lib/layers/<layer_name>) all it takes now is a single line of code when instantiating a Lambda Function in the CDK stack:

lambda_poll = get_lambda(
    # ...other_parameters,
    layers=[get_layer(self, "feedparser", id)],
    # ...
)

Feedparser is the main actor in the new feed_poller Lambda function: it takes care of sending the HTTP GET request to fetch the XML feed content, parse and validate it and finally provide an handy API for extracting useful information such as time sorted blog entries with title, link, timestamps and so on.

Optimizing RSS/Atom feed polling

Both RSS and Atom feeds specification provide bandwidth optimization features, such as ETag and Last-Modified HTTP headers, which remove the need to fetch the whole data set at every poll event in case there are no new entries.

Unfortunately though the current setup of this blog (statically generated by Jekyll v4 and hosted by Netlify) does not support any of those out of the box.

This blog and its feed are low traffic and currently don't require any kind of optimizations so investing effort on saving very little bandwidth would have been pure over engineering (or, more specifically, premature optimization).

Netlify provides custom HTTP headers support though so it should be possible (I'm not sure, I haven't done any research) to update the Jekyll Feed plugin to better integrate with the current setup and I don't exclude I'll have a look at this at some point just for the fun of it: it's been long since the last time I had the chance to work with Ruby and I always found it a (mostly) pleasant experience.

A stateless option

At first I thought there was the need to manage some kind of state to ensure only new posts were considered for publishing. Instead of jumping at any stateful kind of solution though I thought a bit deeper about the actual requirement and decided that I was more than happy to trade some latency for... statelessness 🥰!

As already discussed in the previous chapter, currently there's no support for bandwidth-optimization headers anyway (i.e. neither ETag nor Last-Updated data) so no need to store the last updated timestamp (not for that reason at least).

Beside that, RSS/Atom feeds are generally meant to be used in an eventually consumed kind of fashion anyway: I definitely have no need for updating my social feeds in real-time and I'm comfortable with even one day of delay between publishing something new on the blog and notifying the various socials about the new article.

These specifications help in simplifying the implementation sensibly as you can see from the current workflow summary:

poll the blog Atom feed once per day (currently at 06:00 UTC)
check for entries that have publishing timestamp matching any timestamp from yesterday
for each new entry (if any), trigger the publish Lambda (with a little delay between each invocation to avoid being throttled by the destination API services)

This workflow implies there's no need to keep any state whatsoever, which helps in keeping it as simple and stupid as possible.

I also make sure in the CDK stack that feed_poller Lambda is executed only once (per day) even in case of errors to avoid double postings: the default AWS Lambda behavior is to retry execution two times in case of previous non-zero exit.

This is not meant to be a mission critical service anyway so, in the unlikely case of errors, the "Lambda errors" email report I receive daily should be more than enough to let me deal with them.

For the records, the above automation workflow has been in place for the last few weeks and it did work as expected when publishing the previous blog post too.

Python datetime module (included in the Python Standard Library) comes handy when working with timestamps and time deltas, it should also deal properly with those nasty edge cases that can surface during leap seconds/days/years (many thanks to Gregory for ridding us of Leap Months at least 😆):

from datetime import (datetime, timedelta)

def _midnightify(date: datetime) -> datetime:
    """Return midnightified datetime."""
    return date.replace(hour=0, minute=0, second=0, microsecond=0)

now = datetime.utcnow()
yesterday = now - timedelta(days=1)

today_noon = _midnightify(now)
yesterday_noon = _midnightify(yesterday)

new_posts = [entry.link for entry in entries  # entries: Feedparser iterable object
             if yesterday_noon <= entry.published_datetime < today_noon]

The above snippet is just a redacted excerpt that's supposed to give you the general idea, please refer to the actual implementation for more details.

To close the 100% automation loop there's now a new CloudWatch Cron Event that fires every day at 06:00 UTC and triggers feed_poller Lambda, so this is the updated publish-to-social architecture diagram:

Feedback welcome

Please feel free to tell me what you think leaving a comment here or contacting me directly (various options on the Contact page). And, of course, don't forget to subscribe to this blog feed 😜

Originally published at Alexander Fortin's tech blog

Automate router WiFi bridge setup with Raspberry Pi, Node.js, Puppeteer and Ansible

Alex — Tue, 28 Apr 2020 06:00:15 +0000

Originally published at Alexander Fortin's tech blog

One does not accumulate but eliminate. It is not daily increase but daily decrease. The height of cultivation always runs to simplicity.

― Bruce Lee

I've been a frequent traveller for the last few years and I learnt along the way that reducing the clutter that comes with me to the bare minimum is a very good trend to follow.

For example I drastically reduced the amount of clothes that I own to the point they all fit into a cabin luggage, and I'm generally mindful when I buy any new object because I know that either has to fit in my luggages or has to be sold/gifted/trashed next time I'll relocate.

I confess though that the geeky side of me still requires substantial room in the non-cabin luggage for the following:

an old MacBook Air dated fall of 2013 (if I'm not wrong)
Sennheiser Bluetooth headphones with noise cancellation (which of course I take to the cabin when boarding...)
Sony PlayStation 3 Super Slim with one gamepad and a few original DVD games
Raspberry Pi (armv6l)
1TB USB3 external 2.5" hard drive
various Ethernet/USB/HDMI/miniJACK cables and plugs
unbranded ADSL/4G/WiFi router, which runs a very crappy administrative software but once it's configured it does its job pretty well and comes with a few built-in handy features, like:
- 4g/LTE connectivity
- WiFi-to-WiFi bridging
- persistent DHCP configurable table mapping (MAC <-> IP)
- 4-ports Ethernet switch
- etc.

Nomad multimedia lounge setup

With the above items I'm able to very quickly setup a comfortable yet powerful gaming/media environment whenever I move into a new place where there's some connectivity (at least some mobile network coverage) and/or a TV color.

The only thing I have to do once settled in the new place is to bring power to router + Ps3 + Raspberry Pi, update the router configuration to connect in "bridge mode" to the available WiFi (or buy a SIM card with a LTE data plan if WiFi is not an option), plug both the Raspberry Pi (with the USB drive attached) and the PlayStation to the router switch with Ethernet cables, PlayStation to the TV with (hopefully) HDMI cable and... voilà!

Everything is connected to the Internet and ready to serve its purpose, including streaming media (Netflix/Amazon Prime/YouTube) and music (Spotify) directly from the relative PlayStation apps. For all that's not available online I make use of Serviio media server, which lets me consume from the Playstation (via DLNA protocol) the audio/video/photo media files stored in the external USB hard drive.

Another side benefit I get with this setup is that with a single shot all my portable devices (MacBook, iPhone and Kindle) get Internet connectivity too: the router WiFi password is already saved and there's no need to configure yet another WiFi and type the relative password for each device.

The Problem

One very annoying problem I face with this setup is that every single time there's a power outage and the router reboots, or for whatever other reason the WiFi bridge is broken, the router doesn't automatically reconnect to the source WiFi.

It might sound like something infrequent to you if you're used to live in developed countries but for example in east Africa, where I've been living the last few years, brief power outages are quite common even in the main urban areas.

In order to restore the bridge I have to manually log into the router admin interface from my laptop's browser, find the right admin web page from the unintuitive navigation bar, perform a few selects and clicks, retype the source WiFi password and, finally, click to apply and persist the configuration.

{: .img-fluid .col}

The above could be classified as a first-world problem indeed but, as you might already know if you've been following me for a while, I'm a long time advocate of the lazy software engineer attitude (our motto? life is short!): automate all the tedious and boring tasks so to be able to focus on the most important ones, those that are not so easy to automate and where I believe our true value as engineers is.

The Solution

I was initially tempted to approach this using a tool I already know and use for some other projects (Cypress). Recently though I've been reading good things about Google's Puppeteer so I decided to take this chance to see what all the fuss is about.

This is the tech stack needed to implement the solution:

a relatively old Raspberry Pi mounting Raspbian 10 (Buster)
a Node.js runtime (in my case v11.15.0 which is the last available version officially distributed as binary that supports armv6l architecture)
puppeteer-core v3.0.0
Chromium browser v78
Ansible v2.9.6 (not strictly needed but that's how I drop things into my Raspberry Pi by default anyway)

These 80 lines of Node.js automate the above process thanks to the awesome puppeteer-core library. It runs both on MacOS and Linux:

macOS: tested on my MacBook Air running Node.js v13.13.0 and puppeteer-core v3.0.0
Linux: tested on my Raspberry Pi armv6l, Node.js v11.15.0 and puppeteer-core v3.0.0

To complete the setup, I add this simple cronjob that verifies connectivity every 2 minutes and, in case of failure, runs the Node.js script:

*/2 * * * * ping -nq -c 3 1.1.1.1 | grep -q " 100\% packet loss" && node /opt/routerfix/repo/fix-router.js 2>&1 | logger -t ROUTER_FIX

I split the above one liner command to better explain how it works:

*/2 * * * * tells CRON daemon to run the command every 2 minutes, it could have been run at 1 minute frequency but the Node.js script takes around 40 seconds to complete execution on my Raspberry Pi so this is meant as a safeguard to avoid running two processes at the same time. The script also commits suicide 90 seconds after execution anyway (refer to the terminate function at line 17 for more details) so this race condition should never happen
ping -nq -c 3 1.1.1.1 sends 3 ICMP ECHO packets to a highly available IP destination (Cloudflare public DNS service), and in case these packets are all lost it will output to stdout something like "3 packets transmitted, 3 packets lost, 100% packet loss"
output from ping is piped to grep, the -q option suppress output and just returns 0 or non-zero exit code based on the string match. This means that the command after && will be executed if and only if all the 3 ICMP packets are lost
node /opt/routerfix/repo/fix-router.js 2>&1 executes the Node.js script which runs Chromium browser events in the background, it also combines stdin and stdout into a single stream (the 2>&1 bit)
by default, crontab commands output is sent to the UNIX user mailbox that owns the process; piping Node.js output to logger -t ROUTER_FIX sends it to the syslog service instead (which in my case is then eventually forwarded to Papertrail where I can read them comfortably from my browser as you can see from the following screenshot)

{: .img-fluid .col}

You can find all the code involved in the solution in this GitHub repository including the Ansible playbook.

Pitfalls and Caveats

You know when they say you never truly know anyone? that applies to software too:

after ages of CRON usage I just discovered that the percentage char (%) has a special meaning in a crontab so needs to be properly escaped with a backslash char (\)
in this pitfall I fell so many times that I think it's always a good idea to refresh our memories: CRON's PATH environment variable is usually not the same one of the default UNIX shell, so remember to either provide full path for each command or explicitly set it in the crontab (I opted for the latter solution as you can see at line 58 of the Ansible playbook)

About Puppeteer, I barely scratched the surface of what the APIs offer so I can't say much about it beside that's well documented and easy to get started with. If you are already familiar with JavaScript and its callback-driven nature, Puppeteer should pose no real issue and you might be up and running in minutes.

Personally I find Cypress' promises-inspired style more elegant: chaining promises commands in one or more sequences is usually the way I use to think about interacting programmatically with the browser. As far as I understand they are targeting different users and use cases though so, as is usually the case, choose the right tool for the task, they're both solid projects anyway.

Conclusion

The above automation has been active for the last few days and it's working like a charm, to the point I actually notice there's been an outage only when the microwave clock gets zeroed. I hope this simple example might help you in solving some more complex issue you're facing or give you some more ideas on how to approach it.

As usual, a reminder to not forget to drop a comment here below... and keep in mind the most important of things: life is short! ⚡

An AWS Lambda execution report webpage built with Lambda Destinations, CDK and Svelte

Alex — Thu, 27 Feb 2020 12:18:12 +0000

Liquid syntax error: Variable '{{%20" alt="Architecture Diagram">

Frontend

For the frontend side of things I decided to just add a new page to this very Jekyll blog and have it publicly available at this URL.

As a side note, I already have plans to migrate the blog from Jekyll to Hugo so I didn't invest too much on the page layout design at this time.

For the dynamic part I could have added some good old jQuery given that it's already been used for the {% raw %}' was not properly terminated with regexp: /\}\}/

Setup AWS Lambda-to-Lambda invocations between CDK Stacks

Alex — Thu, 27 Feb 2020 07:56:52 +0000

Liquid syntax error: Variable '{{' was not properly terminated with regexp: /\}\}/

Automate social media status updates with AWS Lambda, SNS and CDK

Alex — Mon, 17 Feb 2020 09:41:31 +0000

Originally published on Alexander Fortin's tech blog

As some of you have noticed, recently I've been blessed with some extra time and decided to take the chance to finally come up with some fresh new content for this blog.

How to choose a new pet project in the IT world where the choice of things to be learnt and practiced to stay relevant is simply overwhelming?

With a green field pet project I generally move from fresh itches that I'd like to scratch because, well... there's never a shortage of them, no matter how serious you are in trying to automate them all away.

I try then to come up with a feasible solution that won't need any wheel to be reinvented, spend too much money to keep it up and running or require too much time and effort to actually make sense.

Once the goal and its scope are defined I select the tech stack but, instead of being strict in choosing the right tool for the job (as I'd do while working under contract), I let my curiosity for the new interesting tech lead at least some of my choices.

To be clear, I don't want to jump on the first buzz at hand, I still try to pick the best tools for the job but not being yet confident with those tools I can only make educated guesses.

Beside the curiosity for leveraging abstractions writing CDK code as mentioned in the previous article, it's been a while that I wanted to better understand the OAuth flows that today are very common for authenticate/authorize public APIs.

Let's scratch

Especially now that I'm publishing content more frequently (and planning to keep doing so) I want to inform the ones who follow me when there's a new article available, otherwise what's the point of producing any content?

In time though the social media accounts and (re)publishing platforms proliferated almost out of control. The good old days when basically everybody in the Internet owned a personal website (GeoCities anyone?) are long gone and apparently for good.

To be more specific, here is a list of my current digital presence so far:

... and no need to mention how the Instant Messaging situation is even worse, right?

I suspect I'm forgetting one or two but you get the point: they are already too many to even remember without browser bookmarks hints and it's safe to say the list in time is likely going to grow.

This means that every time I publish a new article, unless I'm relying only on the RSS feed to spread the news, for each of the above accounts I should:

open a new tab in the browser
perform a log in (with two-factor authentication of course...)
pass the occasional Voight-Kampff recaptcha test
finally, copy&paste in a textarea some text that might look like this:

New blog post:
[a brief description of the content...]
https://link.to.new.article

a snippet that will need slightly different content and formatting depending on the targeted platform (e.g. in Twitter prefer title over description to fit into the 140 characters limit, etc)

I know there are multiple professional services like Hootsuite that provide this kind of feature (and many more) but, unless you're willing to pay for the premium service, usually they put strict limitations on how many integrations you can use, how many posts per month, and so on.

Actually I'm even ok paying a monthly fee for something I deem valuable but for such a simple use case a fully featured SaaS is definitely overkill. On top of that I'm even happier if I can avoid sharing any sensitive data like secret tokens with any 3rd party service.

As you already have guessed from the spoilering title, this looks like a perfect use case for a serverless event-driven application on AWS.

The tech stack

For the case I'm presenting here, the choice fell on the following technology stack:

AWS Lambda + SNS to build the infrastructure
AWS CDK to model the infra plus create/update and wire together its building blocks
Python 3.8 + Boto3, requests_oauthlib and BeautifulSoup4 3rd party libraries as runtime for the application

What's the story

The hypothetical user story for this project might have looked something like this:

As a blog writer, I want to automatically update all my social feeds with a link to the new article as soon as it's published.

and here the features request:

hard requirements

fully managed infrastructure, near-zero maintenance needed
cheap to the point of being free (until I learn how to publish new posts at sub-second frequency. Just kidding 😁)
simple to manage
easy to add new output services for different social medias

soft requirements

have the whole process 100% automated
delivery to all the social networks mentioned above
highly available
fun to build

Show me the code

This time instead of creating a dedicated repository with some example code I decided to clean up the actual codebase I was already using to deploy my personal APIs and make the repository public.

So, speaking of abstractions, I added to the main CDK app (lib/cdk.py) two more stacks that join the previous single one, api.

The first new stack is called lambda-layers and its only reason to exist is to keep the Lambda Layers isolated from the other stacks unless those stacks need one or more Layer as dependency: in that case stacks.lambda-layers.layers object will be passed to the needing stack as parameter, making that part of the code DRY and pluggable as needed (more on this later).

If you don't know already Lambda Layers, it's a powerful AWS Lambda feature that lets you provide additional code to your Lambda runtime, removing for example the need for bundling external libraries together with your actual Lambda code. You can read more about how Lambda Layers work here.

Speaking of CDK, I've restructured a bit the codebase compared to the one you find in the mini-tutorial from the previous article. Instead of the default scaffolding provided by cdk init I preferred a files and folders structure that I find easier to understand and reason about. There are now no redundant folders and everything that relates directly to CDK is either in a cdk.py file or in stacks/<stack_name>/__init__.py.

Another improvement to the DRYness is that now default values and helpers for CDK live in the utils.cdk library that's shared by all the stacks.

The actual code for each individual application (api, publish-to-social, lambda-layers) can be found under stacks/<stack_name> folder, packaged according to the specific application needs.

The api-l3x-in project is basically a container for different applications, the new one I'm going to focus on now is publish-to-social.

publish-to-social

Here a diagram that shows how the building blocks of publish-to-social are connected:

{: .img-fluid }

It's a simple and fairly typical pub-sub (and stateless) serverless application with one main Lambda publisher that sends messages to an SNS topic and multiple Lambda subscribers that are executed every time there's a new message in the topic.

One of the main benefits of decoupling services communication via a messaging bus like SNS is that publishers can be oblivious of subscribers existence, hence we can add as many Lambda subscribers as we need without making any change to the publisher(s) code (another main benefit is increased scalability which is obviously not a requirement in this case, I just mention for the sake of completeness).

I started with the idea of having a small prototype to share but somehow I quickly came up with what I think is a decent solution and something that I might want to maintain for a while, so I'll soon freeze the API at v1.0.0 after adding some unit tests (I know, I know...) and maybe a couple of more features and/or support for other social services.

Speaking of services, a few adapters for the social networks mentioned above are already implemented, you can find them all in the publish-to-social.lambda.services module with links to APIs documentation. All the heavy lifting for logging, handling exceptions, parsing events and implementing proper response interface are shared by all the lambdas so the code needed for each service is very small, you can have a look at utils.handlers and utils.helpers modules for more details.

Probably so far the most interesting implemented one is devto. It's slightly more complex compared to the others given that instead of a simple text message it publishes the whole article (with the original link set as canonical url); this is how the flow looks like:

publish-to-social main lambda is triggered via CLI, then:
1. scrapes the URL given as input in search for content
2. if point 1.1 is successful, publishes a message to the SNS topic
devto lambda is triggered (simultaneously with all the other services) by the SNS subscription, then:
1. authenticates to GitHub APIs and fetch some Markdown content from my private blog repository
2. finally, publishes the article using dev.to/articles/ endpoint

User Interface

The soft requirement of 100% automation is not met yet, ideally the process should be initiated by some kind of cronjob action using the blog RSS feed as source for new articles data. It should be fairly easy to implement but I felt there's already enough beef to share right now.

Currently to publish a new article link on (some of) my feeds I just need to trigger a Lambda execution from the terminal (I'm actually using a wrapper script to just provide the url as argument, here I show the internals):

$ aws --profile api-l3x-in \
    lambda invoke \
    --function-name publish-to-social-publishtosocial17FB96F7-LXWRTUFARTKU \
    --invocation-type RequestResponse  \
    --payload '{ "url": "https://a.l3x.in/link/to/the/article.html" }'

# Response that implements Api Gateway interface
{"isBase64Encoded": false,
"headers": {"Access-Control-Allow-Origin": "*"},
"body": {
    "message": "messageId '5ad1368b-fda5-5c89-8b9e-604bad6433f9' with content scraped from source https://a.l3x.in/link/to/the/article.html delivered successfully"
},
"statusCode": 200}

For now I'm ok with the UX bit and I don't have to deal with any (inconsistent) state nor double posting issues but it's definitely something I might add eventually and might end up in a dedicated article.

As a side note, Python Feedparser external library is powerful and easy to use, I had the pleasure to work with it back in the days when I was actively contributing to the awesome Home Assistant Open Source project. Now I just have to figure out how to handle the last submission state, for example keeping a record updated in a DynamoDB table. For the cronjob part, I already know where to get inspiration thanks to this great article by Blake Green.

404 Social not found

Unfortunately I discovered exploring the various APIs' documentation that some of the most popular social networks don't allow this kind of workflow. For example:

LinkedIn support answered me they don't let developers use OAuth 2-legged authorization anymore:

"All of our API calls run through 3-legged authorization. We are not assigning 2-legged auth for any of our partners"

Hacker News is read-only
Facebook posting via the /feed endpoint is disabled since April 24, 2018

As a side note, I honestly find this lack of faith in their users... disturbing. In a world eaten by software like the one we live in today I'd consider this a basic feature and its unavailability a strong factor for deciding which platform is less worth investing in (professionally speaking).

Pitfalls

I list here some of the missteps I made along the way in the hope they might help someone avoid time waste (and frustration):

making changes to lambda-layers stack after the initial deploy leads to CDK inconsistent states because of an open CloudFormation bug. As a workaround you could think of bundling external libraries' code in the stack or use the Layer resource name (instead of the stacks.lambda-layers.layers object) as input for other stacks. This obviously clashes with the premise of CDK abstractions in the first place. I'm ok with what I have right now but I might change my mind in the future if I see my need for Layers increases. If you end up in an inconsistent state for whatever reason, you might need to either destroy/recreate all the stacks one by one in reverse hierarchical order (lambda-layers at last) or, in case you can't sustain any downtime, deploy to a different target region and/or account, migrate the service and finally destroy the original setup
as already mentioned above, many platforms don't give write access via APIs and you might end up spending quite some time implementing what you found in the documentation... before discovering that they won't work as expected because of missing support for Oauth 2-legged flow for example.
in the official example repositories you might find outdated CDK code, nothing major popped up so far but at least some deprecated code here and there (for example the use of core.Code.Asset instead of core.Code.fromAsset), so better to double check the docs all the time after the occasional cargo cult rushes
just increase memory when lambdas are dying early without producing logs even if logs REPORT from CloudWatch Logs say max memory limit was not reached (in my experiments I never went over 80Mb but a few Lambdas were dying silently when running with the 128Mb default ram size, just doubling the allotted memory solved the problem)
deleting CloudWatch LogGroups manually lose their default retention period when automatically recreated by Lambda's execution
prefer requests over the more complex standard urlilib Python library if you don't mind adding an external dependency, it will make your life much easier

Caveats

Lambdas defined in the same stack are always redeployed when there's any change in the stack folder (excluding the CDK code) or in the utils library. This is by design:

given the tiny blob size (64Kb unzipped, less than 8Kb after compression) the code is the same for all the lambdas anyway and in this way I ensure they're always running the latest version
the blob zip archive is uploaded only once per deploy so only a new UpdateFunctionCode call per Lambda with a link to the new uploaded code archive in S3 is triggered, i.e. there's no bandwidth waste anyway

If that doesn't fit your needs you might want to tinker with symlinks, CDK core.Code.fromAsset.exclude glob patterns, make a different use of Lambda Layers (the last one at your own peril!) or rethink the folders layout.

What's Next

I think what I have is already in a decent enough shape to be shared but I won't consider this production ready yet. To close the gap some important pieces are missing, for example:

add unit tests for the core (especially the utils lib)
add monitoring/alerting
run some security audit (e.g. review the IAM roles and policies autogenerated by CDK)
add a local testing environment to easily test Lambdas, and maybe a remote staging one too
add support for automatic deploys with cdk deploy run by a CI/CD pipeline
introduce some state management to:
- save the latest status update to avoid double submissions
- keep a well formatted log of the publish events (to be displayed via a web dashboard for example)
add meta tags to the blog with a small size preview image to increase clickability
... let me know about what you think might be a must to have for production

Why Python

One might ask why not to choose some other runtime given that CDK offers a wide option of programming languages for our Infra as Code. Here I list few of the reasons why I think Python v3.8 fits really well in this context:

the standard library is fully featured so usually very little dependencies are needed, in this case only Boto3 (which is already bundled with the default Python Lambda runtime), BeautifulSoup4 and requests_oauthlib. Less dependencies mean smaller memory footprint and code size, ease of maintenance and reduced risk of hidden (security) bugs
the inheritance model is neat, for example the OO pattern I followed in the utils.handlers library helps me to reason about that particular piece of logic in a way that I feel natural
Python has been a first class citizen in AWS Lambda land since it was initially released, the latest stable version (3.8.1) of the runtime is supported, and generally I bet it's a language that's going to stay relevant for long
I'm used to dynamic typing but I find that the typing built-in library is quite useful for properly documenting the core classes and methods while at the same time lets me free to use it only when and where it makes sense to me (to be more specific, it's not an all or nothing choice like the one between JavaScript and TypeScript for example). Disclaimer: differently from statically typed languages it won't help you with correctness at runtime though.

Conclusion

This was a really fun project to work on, it reminded me many times of the good old days when I used to spend massive amount of time just playing with those little LEGO© bricks.

Back to now, the more I get to know CDK (and AWS in general) the more I understand the huge potential for quickly developing robust, secure and massively scalable applications while having some fun along the way. Adding new AWS resources and dependencies with CDK is very straightforward so you don't have to waste time defining resources' details and can focus almost completely on the actual business logic of your application.

Speaking of the different building blocks offered by AWS nowadays, they are so many that I find it hard to believe anything to cover all the possible compute needs is missing; I'm sure there are such cases, I just haven't found them yet personally. I know for example lots of developers are asking for some kind of serverless ElasticSearch offer, I won't be surprised if that's announced at one of the next AWS re:Invent events: the rate at which new AWS features and services are released is truly astounding and it makes it harder and harder for who like me wants to keep up with the available options.

Finally, speaking of the CDK abstractions considered harmful argument I can only say that, at least with a simple project like this, they deliver exactly what I'd expect: eliminate code duplication and seamlessly add dependencies between stacks. It's true there are still a few rough edges to be softened here and there (for example the Lambda Layers dependency bug mentioned above, or this about CloudWatch Alarms that I recently stumbled upon) but I see the project is quite active and I'm confident they'll be all fixed as the project becomes more mature.

As usual I'd like to close the write up reminding you that I'd love to hear some feedback (of ANY kind) from you, including suggestions on how to improve the codebase. Drop a message here below or contact me directly if you prefer, I'm looking forward to hear your thoughts.

Introducing AWS CDK with a real life Lambda and API gateway example

Alex — Tue, 04 Feb 2020 10:12:29 +0000

Originally published on Alexander Fortin's tech blog

Amazon Web Services recently announced a new tool meant to ease the management of AWS resources in a fully programmatic way: AWS Cloud Development Kit (CDK for short).

As stated in the project homepage:

[CDK is] an open source software development framework to model and provision your cloud application resources using familiar programming languages.

To me it represents a welcome addition to the likes of AWS SAM and HashiCorp's Terraform, extending the realm of what's possible today with Infrastructure as Code (IaC) on AWS.

It's also introducing a new paradigm compared to other DSL-based tools, i.e. write Infra as Code in your favorite popular OO language, together with some interesting new features that weren't available until now (more on these later).

CDK is available in different popular programming languages like TypeScript (used as default for the examples in the official developer guide), JavaScript, Python (the one I use in my mini tutorial), C#/.NET and Java.

What is CDK

From a practical standpoint, CDK is a Node.js application coupled with a bunch of packaged libraries that let you design, deploy and manage your AWS resources in a programmatic way.

As you might expect from such a tool, you can define resources and interconnect them in dependency chains letting you leverage powerful abstractions (I'll discuss briefly about pros and cons of using abstractions in another chapter) and enforce various best practices out of the box.

For example one of the most welcome features for me is that it removes almost completely the tedious task of properly defining (and attaching) IAM resources.

Behind the scenes, when a CDK app is executed, it creates (synthesizes in CDK parlance) a model of the infrastructure the user has defined in code leveraging CDK primitives.

If no errors are found (wrong syntax, missing imported libraries, etc.) it generates an artifact (synth phase) that is used as input by an AWS CloudFormation CLI process. CloudFormation will then be responsible for provisioning (deploy phase) the actual resources in AWS.

The latter point doesn't mean you need to run CloudFormation CLI manually (even if you could just stop at the synth phase and use the CFN artifacts as you wish), just that CDK main process doesn't interact directly in the creation/destruction of resources and it's basically oblivious to what's happening while waiting for CloudFormation to be done.

For example you can't add any if this fails, than kind of logics but I don't think this is a unhealthy limitation to have, nor the other mentioned tools provide this kind of possibility as far as I know.

Why CDK

CDK is not only the CLI tool and libraries, it's a new take on the whole idea of defining AWS resources all-together. The tool is so powerful it makes you truly feel the sky is the only limit.

For example, one idea that I'm already toying around is to have a reusable stack that makes it dumb simple to have Lambda outputted errors delivered to some (easily replaceable) notification service of choice, together with proper alarms in place, and so on. If done right in future CDK projects should be possible to just pass this hypothetical errors delivery stack as a property to pass to new stacks and have the functionality automatically applied in their scope.

If I get it right, the most ambitious goal of CDK is to make these kind of abstractions the common way of thinking about your AWS-based projects, all while transparently enforcing best practices like modularity, security, and so on.

One of the common troubles when defining infrastructure as code in AWS has been the tedious task of writing CloudFormation templates by hand. Tools like Terraform and SAM ease that burden but they generally lack the unlimited flexibility provided by CDK.

It's true, since long we could use libraries like Boto to interact programmatically with AWS resources. So far though we were lacking a general purpose framework that saved us from the burden of implementing a dependency graph, or writing a lot of IAM boilerplate, or enforcing least privilege principle all the time, and so on.

Getting started

General availability for TS and Python has been announced not long ago but to my surprise I found there's already a good amount of (official and not) resources to quickly get up and running with CDK.

As a side note, I got to know about CDK in the first place via this YouTube feed. I then moved my first steps from the official getting started documentation and subsequently landed on the excellent workshop at https://cdkworkshop.com.

Core concepts

In the following paragraphs I list the basic CDK concepts needed to get started with a simple project like the one used in the tutorial:

constructs

Constructs are the main building blocks you're going to define with CDK. From the official docs:

Constructs are the basic building blocks of AWS CDK apps. A construct represents a "cloud component" and encapsulates everything AWS CloudFormation needs to create the component.

[...]

For example, a central team can define a construct that implements the company's best practice for a DynamoDB table with backup, global replication, auto-scaling, and monitoring, and share it with teams across a company or publicly

[...]

AWS constructs make least-privilege permissions easy to achieve by offering simple, intent-based APIs to express permission requirements.

[...]

AWS constructs are designed around the concept of "sensible defaults."

Follows a code example from my tutorial.

src/stack.py:

from os import environ
from aws_cdk import (
    [...]
    aws_certificatemanager,
)

[...]

cert = aws_certificatemanager.Certificate(
    self,
    '{}-certificate'.format(environ['CDK_APP_NAME']),
    domain_name=environ['CDK_BASE_DOMAIN'],
)

stacks

A Stack is an abstraction that helps in grouping resources:

All AWS resources defined within the scope of a stack, either directly or indirectly, are provisioned [by CloudFormation] as a single unit.

In our example we make use of a single stack but we could define more in our application. Refer to the docs if you want to know more.

A code example from the tutorial:

src/stack.py:

class ApiStack(core.Stack):  ## Stack class definition

    def __init__(self, scope: core.Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)

        ### the constructs belonging to the scope of ApiStack follow below...

identifiers

Identifiers are the labels we give to our resources:

Identifiers must be unique within the scope in which they are created; they do not need to be globally unique in your AWS CDK application

The list above is by no any means exhaustive, I suggest you to read the documentation to get acquainted with all the other CDK core concepts.

Best practices

CDK is relatively new so it's too early to discuss thoroughly about best practices, as usual they should emerge in time (assuming of course that the tool will gain adoption and stay relevant).

I found this interesting one in the docs though:

Generally, we recommend that you perform validation as soon as possible (usually as soon as you get some input) and throw exceptions as early as possible.

I also dare adding a related recommendation: add constructs iteratively and frequently test syntax and semantics running cdk diff. This will help you identify problems early on sparing you from debugging complex mixed code exceptions (both Node.js and Python in my case).

Tutorial: migrate Lambda and API Gateway from Terraform to CDK

Enough talk, about time to see something practical.

In this mini tutorial we see how to make use of CDK to connect both a Lambda and an ACM SSL certificate to a (CORS enabled) API Gateway, so to be able to trigger a synchronous Lambda execution via an unauthenticated POST request to an HTTPS endpoint like https://api.mydomain.com/.

The code in the tutorial is currently used for a real backend that receives POST data from an HTML form and (in absence of errors) delivers the message content to a 3rd party API messaging service. You can find more details about both architecture and implementation in the README.

As a side note, the migration that gave birth to this mini-tutorial was relatively simple and went very smoothly, with all the functionalities replicated in a new AWS region without any downtime for the API service.

It also brought significant reduction in the amount of code required for the exact same AWS setup (almost exact to be precise: CDK introduced a few new CloudFormation and S3 resources that were not there):

# OLD SETUP
$ wc -l infra/* src/* Makefile
  121 infra/api-gateway.tf
   58 infra/lambda.tf
   24 infra/main.tf
    4 infra/versions.tf
   97 src/lambda.py
   19 Makefile
  323 total

# NEW SETUP
$ wc -l lambda/* src/* Makefile
   92 lambda/contact_us.py
   20 src/app.py
   54 src/stack.py
   51 Makefile
  217 total

If we consider only Infra as Code LoC the simplification is even more evident:

# OLD SETUP
$ wc -l infra/*
  121 infra/api-gateway.tf
   58 infra/lambda.tf
   24 infra/main.tf
    4 infra/versions.tf
  207 total

# NEW SETUP
$ wc -l src/*
  20 src/app.py
  54 src/stack.py
  74 total

I know, Lines of Code is a rough indicator that doesn't tell the full story about complexity but it's common belief that less code means less surface for introducing bugs and less data to have stacked in your mind while coding. Personally I find the CDK code very easy to read but I'm a long time Python developer so it's hard for me to tell how much my bias is involved in this judgement.

Pros and Cons

I haven't had enough time yet to properly experiment with CDK, adopting it for new projects (where it makes sense) will help me grow a better understanding of the tool and give a better informed opinion about it.

That said, I list here some of the pros and cons that I noticed so far:

pros

very powerful and flexible
sane and secure defaults
DRY-friendly
built-in support for multi-region, multi-account and multi-environment (dev, prod, etc) setups
support for popular languages may get you (and/or other team members) quickly up to speed
physical resource names (e.g. an actual S3 bucket name) are name-spaced to avoid name clashes, making it much easier to manage global resources (e.g. S3) in multi-region/multi-account setups
custom resources enhanced reusability: > "[...] wrap them up into a regular construct interface, so from another user's perspective the feature feels native".
AWS has grown a reputation for maintaining products and services for long (actually I'm not aware of any AWS product that has been discontinued) which should make investing in CDK a safer bet compared to other Open Source tools that have no powerful company on their back and might disappear from the radar if not properly financed
nice to have features like apply tags recursively to all children constructs

cons

not cloud agnostic: works only on AWS
it feels like it's still in early stage of development, e.g.:
- CloudFormation APIs are not 100% covered yet
- links to examples are disappearing and/or changing URL
- documentation could definitely be improved adding more examples on common scenarios and best practices
inflexible (and AFAIK unavoidable) way of setting up new projects
it's reasonable to believe at least some of CDK metadata files (kept in the <app_name>/cdk.out folder) should be managed in the way terraform.tfstate files need to be. Currently there's no built-in support for decentralization (for example in the way Terraform is doing supporting S3 buckets and other backends), making it harder to fit in a CI/CD pipeline for example
minor: cdk destroy leaves some leftovers in CFN and CloudWatch Logs that need to be deleted manually

TF vs CDK... FIGHT!

I'm a HashiCorp's products user since the early Vagrant days. Terraform in particular has been one of my favorite tools and I've been using it for quite some time now.

To clear up any doubt, I'm not planning to drop TF any soon nor to replace it with CDK in any new project.

Even if I focus mainly on AWS, now and then I need to interact with other service providers which is obviously not doable with CDK, and this blocker is already enough to keep Terraform safe and warm in the toolbox.

That said, I like very much the way CDK is doing a lot of heavy lifting that TF does not when setting up VPC or IAM resources for example. The boilerplate reduction is consistent as it is the reduction of security risks not having to write IAM policies by hand.

Disclaimer: I trust AWS developers here, personally I haven't dug much into the actual soundness of any of the IAM setup CDK automatically generated for me. IAM correctness is described multiple times in the docs as a strong selling point though and I think it's reasonable to believe it. Double checking IAM policies and roles before going to production is always a good idea though, regardless of how IAM resources were defined.

Finally, I'm undecided yet how the object oriented approach maps the domain compared to something more declarative like the Terraform DSL. In the past (before the Serverless epiphany struck me...) I made extensive daily use of Puppet and generally thought that using declarative code for defining system/infrastructure setups helped in maintaining and reasoning about the codebase in general, about resources' interdependencies in particular.

In practice though declaring resources in hierarchical order doesn't feel much of a limitation given that's how I'm used to think about the infrastructure anyway (i.e. first I create an Ec2 VPC, then I add an ECS cluster to it, then I add a tasks definition to the ECS cluster, etc.. ).

I briefly discussed the topic with a good friend of mine who's working daily on AWS at (very big) scale, he rises the argument of how mature those declarative DSL approaches are today compared to other battle tested old technologies like Python. He's right when he says there has been not enough time yet to squash all the most annoying bugs, reduce the workarounds needed in some common scenarios like multi-region/multi-account setups and to consolidate best practices.

Honestly it's a hard call and I don't feel like I have enough hands-on experience yet with CDK and the other tools to make it.

General reception

Apparently CDK approach is not very welcome in the Serverless world. For example in the latest "Serverless Chats" podcast episode (around minute 34') CDK doesn't get much appreciation compared to other tools like Serverless Framework or SAM.

I understand the argument validity, that those tools might be a better fit compared to CDK in specific serverless scenarios, like the ones involving mostly Lambdas and API Gateways. So far though I still believe CDK might fit decently even in those cases. If I got it right from the start, sane defaults and built in secure IAM setup are the biggest key selling points of CDK, not the wide choice of languages available.

On top of that, being an AWS general purpose framework means that you might use it in any kind of projects involving AWS and not only when following a strict serverless approach.

Talking about the possibility for abstractions, they are definitely a selling point too and I agree they are a double edged sword that can get out of hand if used improperly. That said, CDK is not very opinionated on that, leaving you the freedom to choose if and when to make use of abstractions, as well as to proceed in a more procedural way if preferred. You can still choose your poison.

My recommendation, in this case as in many others, is to try it out and decide by yourself if CDK gives you better results compared to other approaches. On one thing I believe we all agree though: don't reinvent the wheel trying to write your own IaC deployment system, just choose the one between the popular ones that better fits to your context.

Feature requests for CDK devs

I guess these will never make it to the top of their roadmap but it's honestly what I'd like to have changed with the CLI tool at this phase: being able to bootstrap a new project with an empty template (i.e. no automatic scaffolding) with decentralized handling of CDK local metadata (cdk.out folder).

This would fit well with my usual workflow that goes similar to this:

initialize the project repository
add commands to Makefile as I go
iterate until the project is up and running
in the future, when I'll have most probably forgotten the implementation details, just run git clone && make all to have the project up and running on a different workstation or CI/CD setup, or deployed on a different target AWS account/region.

Currently the last CDK CLI stable version (1.22.0) only provides an option (--output) to place cdk.out folder in a different PATH which might help when in a CI/CD but it doesn't when working in a team for example.

Coupled with the feature above it would be nice to have some kind of cdk sync command that reads cdk.out URI from a local config file and fetches whatever needed (locally) to be able to apply new functionalities (remotely).

Lastly, a nice to have: some kind of --force switch for cdk destroy to remove every resource created, including the CloudFormation CDK meta stack and CloudWatch Logs (if any).

Final words

I hope this article and the mini tutorial give you a good introduction to CDK, the external links above should also point you in the right direction if willing to try CDK out or know more about it.

It's definitely a powerful tool that has just entered the landscape joining other projects that are at a more mature phase of development.

My recommendation is to spare some time and have some fun with it so to make a better informed decision next time you bootstrap a project backed by AWS.

I'm curious to know what you think about it! If you have anything to say or errors and imprecisions to correct please leave a message here or contact me directly.

Remaking my curriculum vitæ with modern web technologies

Alex — Fri, 31 Jan 2020 12:15:26 +0000

As I wrote in my previous blog article, recently I became quite hooked by the web and all that's orbiting around it.

This time I'm going to show you a simple project developed following Jamstack guidelines:

HTML5 for responsive and accessible markup
CSS3 to lay out the presentation
Hugo to statically generate the content
GitHub for source hosting
Netlify for content hosting and (fast) delivery

The project

In the past I've been relying on a free offer from VisualCV to host my curriculum vitæ online. Recently though (end of 2019) they changed their terms and removed the free offer, so no more CV online for free.

For a while I wanted to manage my CV in a more flexible way than VisualCV or any other job platform like LinkedIn offer. Almost at the same time of VisualCV move I stumbled upon this great article on CSS-Tricks by Ali Churcher, I liked it quite and made me think I had no more excuses to procrastinate.

I thought a while about how to best approach the issue and then put in practice my new acquired skills as a web developer (while having quite some fun in the process).

Feature request

CSS-Tricks' article is showing flawlessly how to setup the layout, and that's exactly what I'd expect from an article by CSS-Tricks, but to me it is missing a few key features to make it really shine.

One feature I definitely want to have is a comfortable way of managing the CV content. In the end what's the point of having it online if I can't keep it up to date with ease?

In practice I just want to open my IDE, edit some Markdown, run git commit && git push and have the new changes deployed online asap.

Implementation

To add my features I pair the cited HTML5+CSS3 tricks with Hugo and its powerful content management features.

A couple of more clicks in the Netlify app (plus a few new DNS records in my l3x.in domain) and everything is live at https://cv.l3x.in, replacing the old HTTP redirect to VisualCV.

You can find the actual source in the GitHub repository with all the details in README.md, together with a mini tutorial on how to use it to build your own CV.

Highlights

I take the chance to highlight a couple of more things:

differently from the layout proposed by CSS-Tricks, I opted for keeping the header (name, title and photo) outside the main grid. To me it makes more sense to have it decoupled from the content (i.e. from the grid) because the likelihood I'll put it anywhere else than the top is close to zero, and having picture and text box swap relative position with display: flex is trivial
in layouts/partials/head.html there are a few tricks to make the site load faster, thanks to Google Insights. At the time writing the live site scores above 99% for both mobile and desktop
Hugo makes it fun to add all sort of functionalities, for example the copyright and last modified info in the footer
another cool thing when using Hugo, you might think of partials as sort of web components, that might help structuring your markup in a way clean and easy to maintain

Possible enhancements

I'm quite happy with the results and I don't think I'll change the layout for a while (I always need to improve the content but that's out of scope eheh).

If you have any suggestions or recommendations please let me know either by leaving a comment below, contacting me directly or (even better) opening a pull request.

I list here a few of the possible improvements that came to my mind so far:

I've used SCSS (in assets/ folder) to keep the style DRY but I'm pretty sure there is a lot of room for improvement
The scroll to top icon should appear only after some scrolling, I don't know if that's possible in pure CSS alone and I'd like to keep the project JS free. If you know how to do it in CSS please let me know, thanks!
The footer could appear at the very end of the viewport even when content is not filling up the page completely (e.g. https://cv.l3x.in/programming/). It should be easily done with flexbox, I'm ok as it is for now

Feedback always welcome

I hope you like it and hopefully find it useful too.

As usual I'm more than happy to hear what you think, especially if you have something critic to say about it: for sure it will hurt a bit but so far I don't know any better way to improve my skills ;)

Originally published on Alexander Fortin's tech blog

My quest for identity in Software Engineering

Alex — Wed, 29 Jan 2020 10:17:04 +0000

Keep walking. If I look back I am lost.

― George R.R. Martin, A Dance with Dragons

They say that the sum of all our past choices and experiences makes up most of who we are. Despite that knowledge I never liked to invest much of my free time thinking about the past.

I always thought there's such a thing as the right to forget and to be forgotten, that the opposite (to look forward, to imagine how the future could be, some might say to dream) is one choice that more consistently brings better outcomes.

If only I could perfectly stabilize between those two opposites... but that's maybe for another time.

Faulty Random Access Memory

I have to believe that my actions still have meaning, even if I can't remember them.

― Memento

There's more. It turns out that our brain is not very accurate at remembering.

Contrary to the common belief, it doesn't store a perfect snapshot of reality as a calculator might do: immutable, pixel perfect and ready to be retrieved at will. Multiple loads, same exact results.

The human brain apparently doesn't care much about von Neumann architecture and goes its own way as it did for the latest three hundred thousand years or so.

When remembering, i.e. creating a representation of a past experience in our mind, the hippocampus coordinates the decoding of various media chunks, loading the bits and pieces scattered (should I say sharded?) between different areas of the neocortex into a coherent entity.

There's consensus between neuroscientists on how incredibly plastic our brain actually is, how it's constantly and frenetically rewiring itself, creating new pathways and dropping established ones to make room for new models of reality. All of that while running countless maintenance jobs in the background. It is hard to say to what extent this frantic synaptic rewiring activity is messing with the memories of what actually happened.

That's not the point I want to make with this article though, neither looking forward is a dogma for me. I always liked to keep flexible, leave doors open and go out of the ordinary when it makes sense.

The more I get older, the more I recognize circumstances where it does make sense so, with my 40th birthday knocking at the door, it feels safe and fair enough to look back and indulge in some wrap-ups and where to go from here kind of reflections.

Special Circumstances

Just remember that the things you put into your head are there forever, he said.

― Cormac McCarthy, The Road

If I do look back now I can't think anything other than "it has been a great ride". Sometimes harsh. Sometimes exhilarating. Never boring.

The truth is, I was lucky. Not because of some mysterious invisible force keeping me pointed in the right direction, neither because of the average quality of the choices I made.

I was born lucky.

Lucky enough to spend the formative years in a rich economy that gave me, between other things that today for many are still mere luxuries, a solid education paired with early access to new fancy technologies.

On top of that, I had a very timely chance to give a closer look at what back then was the new rising star in the global sky: the "World Wide Web".

How deep is your learning?

I’m floating around the universe on this giant sphere, suspended here by gravity and going for a ride.

― Anita Harris, Conscious

With a growing stack of knowledge and experiences coming along wherever I've been, I was allowed to keep building new skills needed to constantly practice a profession that I still love today as much as I did when moving my first steps, if not more.

Since then I could travel far and abroad, always to find a more than decent occupation virtually everywhere without much of a sweat. Titles have been ranging ever since from Sysadmin to System Engineer to DevOps/SRE Engineer but those labels never truly mattered to me: I just wanted to deliver value doing what the business needed and it turned out their IT infrastructures needed a lot of care.

I met a lot of professionals on the way. Some of them brilliant and ingenuous. Some flamboyant and inspiring. Some pretentious and annoying. Many though, if not all, joined by a common passion for what we crafted.

That multicolored culture within the culture, made up of uncountable people and ideas and cool new technologies and funny but visually ugly memes (...and on and on), soon became fuel for virtuous loops of discover => learn => practice => discover that, almost mindlessly, kept me moving ahead doing what I liked and getting a pretty good living out of it.

Eventually inconsistent

Do not confuse duty with what other people expect of you.

― Robert A. Heinlein, Time Enough for Love

I'm increasingly unsure about how to answer the question "Would you do any different?" though.

I dropped out in the middle of my engineering studies and still forget big O magnitudes 10 minutes after a coding interview is over. I never worked for a BIG nor visited Silicon Valley. I didn't write a kernel module nor create any widely used (Free and Open Source) software. I was never fully in charge of any product beside when freelancing or working on a personal one.

I could go on indefinitely.

Heck, let's be honest, I was never the best in anything I did, neither while studying the theory nor putting it into practice. For sure I missed out interesting but invisible (to me) opportunities, and a few times I even dropped already taken good ones while pursuing other more important goals.

Finally, I lost the count of the many pet projects I fell in love with and forgot soon after, a behavior that at this point we can safely call chronic. It's true that some of those projects left a mark on me, increasing my knowledge and experience, but none of them ever truly conquered my heart.

Maybe it's true what they say then: the first love you can't ever forget.

Contacts[0]

Common misconception that; that fun is relaxing. If it is, you’re not doing it right.

― Iain M. Banks, The Player of Games

I first got interested in the web around the late Nineties, when I connected to the Internet with a noisy 28.800 baud modem and upgraded to a PPPoE ADSL soon after.

Broadband connectivity in 1999... that was luxury!

The initial heat soon dissipated though. I started to believe I missed a key natural skill required to become a decent (web) designer, and if you see my hand writing you'd know instantly what I mean.

The first attempts to set up from scratch a decently looking template for a website put me off as an impossible feat and made me wish for something... more ergonomic.

To be truly honest with you, the main reason I gave up is because it wasn't fun.

Got ROOTs?

Control can sometimes be an illusion. But sometimes you need illusions to gain control.

— Mr. Robot

Sending cryptic commands to an IRC bot while waiting for a recompiling (2.x) Linux kernel. Configuring MTAs to securely relay and deliver root@localhost emails to some mbox. All while jumping between one (hopefully secure) shell to another, palms sweating before hitting enter when commands ended in something like -j DROP.

All that was way too much fun to go in search of any other digital adventure. Even better, I discovered almost without realizing it that businesses were already willing to pay me to do that, full time.

I soon lost interest in the new world of web development while I dived deeper and deeper in the one of networked computer systems. The back, the infrastructure, the invisible end felt like the right place to be: abstract, loosely constrained, wild and unexplored. So there I decided to put my roots (pun intended).

The web, the frontend, was out of my way for good. Or so I thought.

Breaking (out of) DevOps

"Evolution [is] a random walk across a minefield, not a pre-ordained trajectory, onward and upward toward perfection."

― Greg Egan, Permutation City

Jump forward a couple of decades and here I am, microblogging about Vue.js, discussing about Serverless every chance I get and subscribing to (excellent) web design courses and CSS RSS feeds.

Wait... WHAT?

It looks suspiciously like the classical mid-career/mid-life crisis, right? Just a feral impulse to an easy change: let's get rid of the past with one clean stroke, reject altogether the idea of self-managing backend systems for good. ssh is dead. Get over it.

I'm afraid the truth is far from that, even though my opinion is that today there's generally no real need to orchestrate any fleet of servers running SSHd... unless you are deep in legacy territory, a (ti)tty fetishist or some kind of (cloud) service provider.

SSH is dead. Long live SSH

Humans are meant to work and sweat to earn a living. That's the lesson.

― Cowboy Bebop, Jupiter Jazz (part 1)

I'm not saying server administration is dead. Heck no, long life to Linux & Co. and all the community around it, and most importantly, my sincere compliments to all the System / DevOps / SRE Engineers out there for keeping the Internet alive and kicking, 24/7 365 days per year.

Taming the operational complexities in this era of Global Scales, Spot Instances, Chaos Monkeys and Multi Regions is a very difficult task and one that most of the time gets only little appreciation (to the point of publicly asking for some every new year) because it's not on the front of IT.

A smarter move might just be to let those brilliant and hard working professionals take care of that, those who have been running the (invisible to the end user) Internet infrastructure for the last decades or so and today most probably work at AWS, Digital Ocean, Netlify and so on. They are the best suited to take care of that anyway, so everybody else can focus on tasks more directly tight to the actual business at hand.

I'm not saying that you absolutely must avoid old style operations involving servers either. I'm just stating that if you're starting a new software project today, adopting one or more servers (because that's what it is going to be: you'll need to nurture them for as long as your business runs) might not be the best way to go.

Simply put, today system administration is most probably a skill you don't want to internalize unless it's a core pillar of what you're building.

Free lunch? Maybe just a cheaper one

The task of a craftsman is not to generate meaning, but rather to cultivate in himself the skill of discerning the meanings that are already there.

― Cal Newport, Deep Work

Unless you operate in some specific and/or constrained context (or if you're operating at Netflix kind of scale and it might economically make sense to self host), today there are simpler and more effective methodologies that help you build and deliver highly performant (and massively scalable) software products without the need to own nor manage any (virtual or otherwise) server.

Even if you are a strong old school advocate or worry about cloud lock-in, you should at least agree with me that the benefits (auto-management, built-in autoscaling, reduced attack surface, cheaper bills, faster Time To Market... to name just a few) are too many and too appealing for discarding those new trends as just a buzz.

Serverless Computing, Jamstack, Static Site Generators... today all of those technologies are mature and my opinion is not mine alone: many successful enterprises are adopting them as we speak and at an increasingly faster pace. A few of those are kind enough to openly discuss about them so we can make better informed decisions when evaluating adoption of those technologies and adapt them to our context.

To embrace the new a small but sensible paradigm shift is needed and letting go of old familiar practices always comes with some friction. In my opinion though there's nothing truly dreadful as you might soon realize by yourself once you break the ice with those (only relatively new) concepts.

Sir Chaos, please have a sit

I must not fear. Fear is the mind-killer.

― Frank Herbert, Dune

We've been using web services to build our applications for long now. We even survived the global Micro-services invasion that, even though they massively increased the complexity burden on operations, forced us to think about our applications more in terms of fault tolerant, self-contained and independent blocks, orchestrated by asynchronous communication delivered via remote APIs.

I'm oversimplifying here, but I guess you get the point. AWS Lambda today is only 5 years old, Fargate 2, but if you (try to) keep up with the tech landscape it truly feels like those kind technologies have already become a commodity. In time even BOFH kind of sysadmins are learning how not to worry about it... and even love it.

Chaos, unpredictability, is not to be feared.

Maybe it's even your best ally, the one who keeps your edge sharp: listen to relevant events only, embrace eventual consistency, expect peers unavailability. These are the new kinds of mantras.

At this point should be obvious that personally I bet the new dudes and dudettes are here to stay, so if you have read so far and you're not thinking yet I'm completely delusional, try to follow me a little longer while I look forward and keep walking.

join(past, present, future)

“Who are you?” he would ask her every day.

“No one”, she would answer.

― George R.R. Martin, A Feast for Crows

I don't feel there's any discontinuity, neither practical nor ethical (professionally speaking), between being in operations and developing applications following Jamstack principles.

At this point we've been sharing tools and practices between devs and ops for so long that they're overlapping to the point of fading almost completely. Take this example of a common daily workflow of one of your colleagues:

pick an assigned ticket
analyze its content in search of meaning
identify the relevant repositories and relative file(s)
fire up an IDE to edit these files(s)
implement the requested functionality in a feature branch enforcing best practices
add (unit/E2E/whatever makes sense) tests (move this one up if you prefer)
iterate until all tests are green
push the patch and wait for code review
wait for thumbs up, go back to #2 until done
squash/rebase/push the new commit to master
wait for the CI/CD system to run tests, build artifacts implementing the requested change and deploy them to production
keep an eye on telemetry and alerts to ensure everything is fine
set the ticket state to closed

It might be a bit contrived but it shouldn't be that far from the real thing. Can you tell his/her job title from this list alone? That old fence line has become quite thin, hasn't it?

And it might be a very good thing too. We're left with only engineers working on some task at hand regardless of age or gender or (previous) job titles, everything redundant like labeled hats dropped for good.

Open Change, last revision

Law 25: Re-Create yourself

― Robert Greene, The 48 Laws of Power

With that said, whether you consider yourself a dev or an ops or just a wildcard engineer, it seems to me that the foundations of software development and operations did not change much from my first days as a junior.

Performance, maintainability, availability, data consistency and confidentiality... all those needs are still with us and relatively unchanged. Some might be new (I might be wrong but for example accessibility didn't feel that important back then), most of them are not.

To sum up with a one-liner, the need to get the work done is with us today as much as it was in the past, virtually untouched. Stakeholders needed to get it done then as much as they need it today, they never cared much how fancy were the names filling up the tech stack as long as they were functional in hitting the mark.

Real Matter

No fear. No distractions. The ability to let that which does not matter truly slide.

― Chuck Palahniuk, Fight Club

I'm sure some of you grins at the idea of a Perl runtime serving traffic via Apache and CGI today. Try to explain why that's funny to amazon.com or booking.com shareholders, to name just a couple.

Trust me, I don't reject old and battle tested boring technologies at all, quite the opposite.

I work with the terminal every day and I couldn't get anything done without the repeated use of grep and Unix pipes. I still love to code in Python even if I feel like everybody else is jumping on Go or Rust or... put your favorite modern language here.

I just say, let's use the best tools at hand today for the job avoiding all that's unnecessary.

Sure, the tech landscape evolved in the meanwhile: tools, frameworks, methodologies got new names and restyling. Practices to better manage complexity evolved along with them but nothing ever truly got rid of it completely.

For each new approach that gave us a step forward in simplicity, soon we got more pressure in the opposite direction, pushed by needs like bigger scales of operations, more integrations, more delightful user experiences, smaller time to market... and the list goes on.

Once the two opposed forces have stabilized, new balances emerge making room for bigger amount of business value, delivered faster and to more (hopefully delighted) users.

SQEmpathy injection

Anyone got time for a code review?

― anonymous colleague

Most importantly, the need for empathy is also unchanged. We might write in computer languages, but the ones who read and write in those languages all the time are, as far as I know, still humans.

Empathy for the end user AND for the developer both should drive our efforts, and if you're reading this article there's a good chance you are one of the latter... so do yourself a favor today and remember to be empathetic to your future self too.

I don't refer to the kind of empathy you might feel for someone recently friend-zoned but the kind you feel when you see a colleague struggling with something unnecessarily complex.

Care for user experience is an act of empathy. Why should only the end user benefit from it? Software engineers are software users too and have the same right for a good experience.

That's at least what I always tried to keep in mind while trying to write maintainable (Configuration as) Code automation. (If you are a former colleague, trust me when I say I truly hope you didn't have to swear too much when dealing with what I left behind).

I believe that maximizing value delivery to the business should not be the only deciding factor when choosing a technology stack. As the developer tools and practices evolve and become more ergonomic, it's reasonable to wish for adopting the new and leave the old where it belongs.

In the end people make the product so is reasonable to believe happier developers lead to better products.

Left in the middle. Down to Earth. Up to something?

There are no two words in the English language more harmful than good job.

― Whiplash

So here I find myself today: unable to define home more precisely than planet Earth, neither to define a job title any better than an all encompassing Full Stack Software Engineer, in an evolving polyamory relationship with browser dev tools, APIs, FaaS runtimes and container orchestrators.

In balanced tension between back and front, the old and the new, the boring and the bleeding edge, the server and the (perceived) absence of it.

In time, who knows, I might finally meet that redhead, the kind who keeps telling "you know nothing!" and follow her in yet unexplored wild(lings) territories, leaving all my (techie) legacy behind for good.

Until then I guess I'll keep telling myself the same thing I've been telling myself for the latest 20 or so years: there's never been a better time to be in IT.

Originally published on Alexander Fortin's tech blog

References

I link here some of the media content cited above. The list represents part of what I have read/watched/listened to (either very recently or long ago but still stuck in my head) that inspired me and I find worth recommending:

the world of A Song of Ice and Fire - https://awoiaf.westeros.org/index.php/A_Song_of_Ice_and_Fire
Memento, the movie - https://www.imdb.com/title/tt0209144/
The Road, by Cormac McCarthy - https://www.goodreads.com/book/show/6288.The_Road
Conscious: A Brief Guide to the Fundamental Mystery of the Mind, by Anita Harris - https://www.goodreads.com/book/show/41571759-conscious
an immortal classic: Time Enough for Love, by Robert A. Heinlein - https://www.goodreads.com/book/show/353.Time_Enough_for_Love
The Culture saga, by Iain Banks - https://www.goodreads.com/series/49118-culture
Mr Robot, the tv show - https://www.imdb.com/title/tt4158110/
Subjective Cosmology series, by Greg Egan - https://www.goodreads.com/series/160715-subjective-cosmology
Cowboy Bebop, in my opinion still the best anime ever made - https://www.imdb.com/title/tt0213338/
the excellent Deep Work: Rules for Focused Success in a Distracted World, by Cal Newport - https://www.goodreads.com/book/show/25744928-deep-work
another immortal favorite of mine: Dune saga, by Frank Herbert - https://www.goodreads.com/series/45935-dune
The 48 Laws of Power, by Robert Greene - https://www.goodreads.com/book/show/1303.The_48_Laws_of_Power
a modern classic: Fight Club, by Chuck Palahniuk - https://www.goodreads.com/book/show/36236124-fight-club
Whiplash, the movie - https://www.imdb.com/title/tt2582802/