Well, I'm not breaking headline news when saying that data is the name of the game in technology and sits left, right and centre of most business decision-making processes these days.
So, with that mindset, I throroughly enjoyed coming back to the annual AWS Summit London event once again which, as expected, was a day full of data-centric sessions and tech solutions. All very inspiring!
Here are some of my personal takeaways of the day:
- The AWS Well-Architected tool is a thing of beauty, and the fact that AWS has recently thrown Sustainability into the pillars of the Well-Architected framework is very interesting too.
- Implementing Infrastructure-as-Code (IaC) using the AWS Cloud Development Kit (CDK) to model your infrastructure seems like a very good option for those like me that are still not much versed on CloudFormation (which the CDK uses under the hood) or Terraform. Watching the live demo expressing some sample infrastructure with just a few lines of Python code was great fun, and I'd love to try that out.
- Automating data pipelines with Apache Airflow was the session of the day for me. To programmatically create workflows in Python that help you run, schedule, monitor and manage data engineering pipelines could not be any more up my alley right now. You know the guys at Airbnb were onto something great when they decided to create Airflow, and then open source it to get a great community behind it. The session had loads of code samples and cool demos too, so it was all a real treat in the end. I'll be diving more into Airflow - and also Cloud Composer for comparison, which is GCP's own SaaS offering of the AirFlow OpenSource product - for sure!
- Amazon Redshift still remains a bit of a mystery to me, even after a whole session on it unpacking loads of its features, possibilities and use cases. Trying to draw some parallels in my head with BigQuery - Google Cloud Platform's own cloud data wharehouse service, which I know well - also didn't help much. So it remains one of those things that now I know a little bit more about than yesterday, but still feels like I haven't even begun to scratch its surface. So one to watch for me, and learn more about. The use case presented by National Rail was great and full of detail too.
- The built-in Machine Learning capabilities in many of AWS services is something I knew nothing about until today, and I thought it was all preeeety cool. Ideally suited to allow various personas (devs, data analysts, data scientists, BI analysts) to perform ML tasks without having to build, train and manage your own models, it seemed almost too good to be true. Just simply the demo on Aurora, using just SQL and the built-in Aurora ML to run some queries, then perform some sentiment analysis to get some predictions and finally updating and linking the result data back into your queries and tables there and then was nothing short of awesome.
And away from the AWS world, it was also good luck and great timing that I had the chance to catch up with the folks of Elastic on their stand and ask them some few questions I had on my mind around the current Elasticsearch work we have going on, which was quite helpful.
Other than that, I have to confess that socks are still my favourite swag of choice (got a cool pair this time) and, for the fellow sticker lovers out there, NewRelic have come up with a very cool little sticker with a simple design and concept which I loved: peace, love and cloud.
Till next year!