May 11th - Instalment #19
Episode 19, and this week's edition is bursting at the seams with probably the biggest amount of news and updates to share (hence the later posting). Looks like everyone has been busy, and there were some things I had to leave out for another time.
Before I get started though, I think it is worth reflecting on this blog post by Matt Asay, The most important part of an open source project, where he talks about the value of all contributions in open source projects, not just the developers cutting code. It is well worth 10 minutes of your day to reflect upon this post, as well as think about this data point from a study from the World Bank on open source contributions, that highlighted:
57% of the invested resources went to writing code. The rest, 43%, was spent on things like marketing, docs, meet-ups and other activities that aided the collaboration effort.
So, without further delay, let's dive into this weeks latest open sources news and updates.
Celebrate open source contributors
The articles posted in this series are only possible thanks to contributors and project maintainers and so I would like to shout out and thank those folks who really do power open source and enable us all to build on top of what they have created.
So thank you to Ben Kehoe (@ben11kehoe), Adrian Horn (@adhorn), Joseph Maney (@josephbmanley), Luca Bianchi(@bianchiluca), Michael Garcia, Anish Yadav, Josh Kahn, Julie Lerman, Fernando Dingler, Jared Short, Cosmin Sanda, Nader Dabit, Nico Vibert, Dario La Porta, Mikhail Shapirov, Harish Shenoy, Letu Yang and Bernd Mathiske.
Latest from open source projects
AWS SSO credential process
This project from Ben Kehoe helps you get AWS SSO working with all the AWS SDK's that you might be using that do not currently understand it. Here is a link to the GitHub repository for aws sso credential process. Thank you Ben!
AWS Chaos Scripts
This week from Adrian Hornsby, (Operational Excellence and Chaos engineering writer and general all round good guy), comes this collection of scripts to help you inject a bit of failure into your life. Make sure you read the disclaimer, but if you have heard or read his posts on operational excellence and building resilience, then you can check these scripts out and get started on your own journey. Thank you Adrian.
Thanks Adrian for helping me with my own operational excellence by spotting the mistakes. I would love to say these were part of a deliberate plan to embed Chaos into my workflow, but alas.....
AWS Billing Alerts
Setting up billing alerts for your AWS environments is general good practice, so here is a sample architecture complete with open sourced templates to get you started from Jospeh Manley.
You can find the details of how to set this up following Joseph's instructions on his GitHub repository here. Be sure to check out his other repositories too. Thank you Jospeh!
IAM Policy Generator
Luca Bianchi (@bianchiluca) shared this week a nice little library you can find on his GitHub repository, IAM policy generator.
It helps you generate a valid IAM PolicyStatement object without having to “guess” the right Action (and to avoid human error and typos in general), thanks to proper validation and code completion.
Thank you Luca!
Security - Cloudsplaining
The lovely folk at Salesforce bring you this open source project to help improve the security of your deployments on AWS. Cloudsplaining is an AWS IAM Security Assessment tool that identifies violations of least privilege and generates a risk-prioritised HTML report with a triage worksheet.
Check out the GitHub repo at https://github.com/salesforce/cloudsplaining
VMware Cloud on AWS - pyVMC
Are you a python developer that has been looking for a way to automate your VMWare Cloud on AWS Software defined data centre environments? If so, then you are in luck as Nico Vibert has put together this project, pyVMC on GitHub and written up a supporting blog post, Introducing pyVMC. and you can also read this post too.
Check out Nico's project on GitHub here.
Blog posts you should check out
There have been lots of great posts this week, as well as new releases and updates so check these out.
DynamoDB Workbench for Linux
NoSQL Workbench for Amazon DynamoDB is a client-side application that helps developers build scalable, high-performance data models, and simplifies query development and testing. NoSQL Workbench is now available for Ubuntu 12.04, Fedora 21, Debian 8, and any newer versions of these Linux distributions, in addition to Windows and macOS.
Read the full announcement here, and if you just want the download link, here it is for Linux, Windows and MacOS installs.
Mosquitto mqtt to AWS IoT
Mosquitto mqtt is an open source package I have used extensively to connect all sorts of devices and sensors to the Cloud. In this updated blog post, How to Bridge Mosquitto MQTT broker to AWS IoT, Michael Garcia and Anish Yadav show you how to implement the Mosquitto broker ‘Bridge’ capability to setup bi-directional exchange of data with AWS IoT Core through MQTT messages. This will enable your devices to communicate locally with the Mosquitto broker and with AWS IoT Core to benefit from the power of the AWS Cloud.
Now let me dust off one of my many Raspberry Pi's and give this a go.
AWS AppSync and Amazon QLDB
Amazon Quantum Ledger Database (QLDB) is a purpose-built database for use cases that require an authoritative data source. Amazon QLDB maintains a complete, immutable history of all changes committed to the database (referred to as a ledger). Amazon QLDB fits well in finance, eCommerce, inventory, government, and numerous other applications.
Pairing Amazon QLDB with services such as AWS AppSync allows you to safely expose data and that data’s history for mobile applications, websites, or a data lake. In this two part post, part one and part two Josh Kahn explores a reusable approach for integrating Amazon QLDB with AWS AppSync to power an example government use case.
Amazon CloudWatch powers Prometheus metrics
You can use Amazon CloudWatch to monitor Prometheus metrics from Amazon Elastic Kubernetes Service (EKS) and Kubernetes clusters, now available in beta. With this new feature, DevOps teams can automatically discover services for containerised workloads such as AWS App Mesh, NGINX, and Java/JMX. They can also expose custom metrics on those services, and ingest the Prometheus metrics in CloudWatch.
Read the full announcement here.
.NET Core
Julie Lerman writes about AWS for .NET developers in Discovering AWS for .NET Developers. If you a .NET developer and are new or just getting started with AWS, then this will be a useful starting point for you. From getting started to actually writing some code, you will soon be on your way to running open source .NET workloads on AWS.
Thank you Julie.
Amplify DataStore
Fernando Dingler, shares a reference application and architecture that demonstrates how to build a modern mobile application with built-in offline and cloud synchronisation capabilities using React Native with the Amplify DataStore in the front end and AWS AppSync with Amazon DynamoDB in the backend.
AWS AppSync is a GraphQL serverless backend for mobile and web applications. It provides a flexible and reliable API with a built-in compute layer to run queries, mutations, and subscriptions to securely access, manipulate, and combine data from one or more data sources for any application. On the client device, the Amplify DataStore provides a familiar programming model for leveraging shared and distributed data without writing additional code for offline, real-time and online scenarios, which makes working with distributed, cross-user data just as simple as working with local-only data.
You can check out the docs and links to the GitHub repository for Amplify here, the quickest way to get started.
HPC
AWS ParallelCluster is an AWS supported Open Source cluster management tool that makes it easy for you to deploy and manage High Performance Computing (HPC) clusters in the AWS cloud. This week, we have a great post covering this area.
Using AWS ParallelCluster with a serverless API
In Using AWS ParallelCluster with a serverless API, Dario La Porta, AWS Senior Consultant for HPC writes about creating a serverless API of the AWS ParallelCluster command line interface. With this API, you can create, monitor, and destroy your clusters. This makes it possible to integrate AWS ParallelCluster programmatically with other applications you may have running on-premises or in the AWS Cloud.
Thank you Dario La Porta!
Amazon Corretto - introducing Heapothesys
Load testing and performance tuning Java stacks are going to be a lot easier with this open source project, Heapotheys. Created by the Amazon Corretto team, the Heapothesys benchmark is a synthetic workload that simulates fundamental application characteristics that affect garbage collector (GC) latency. The benchmark creates and tests GC load scenarios defined by object allocation rates, heap occupancy, and JVM flags, then reports the resulting JVM pauses. OpenJDK developers can thus produce reference points to investigate capability boundaries of the technologies they are implementing.
Read the post by Letu Yang and Bernd Mathiske, Introducing Heapothesys, an open source Java GC latency benchmark with predictable allocation rates and then go to the GitHub repository for Heapothesys, here.
Thank you Letu Yang and Bernd Mathiske!
Deep Java Library (DJL) and Spring Boot
The java programming language is still very popular and dominant in the enterprise space. Organisations looking to adopt machine learning and deep learning in their existing applications are looking for solutions that enable existing talent and resources to rapidly get started using the programming languages and tools they know.
Recognising this challenge, AWS has created a few open source projects to facilitate the adoption of ML for Java and microservices, and ultimately to help our customers, partners and the open source community as a whole. The Deep Java Library is one such open source project.
In this post, Adopting machine learning in your microservices with DJL (Deep Java Library) and Spring Boot, Mikhail Shapirov walks you through building a Spring Boot application that uses the Deep Java Library to create a simple object detection application using Apache MXNet and DJL.
Thank you Mikhail!
German readers will find this post on the DJL interesting but you can also use various translation services to go through this introduction into Deep Java Library.
Other blog posts
SaaS and open source
In this detailed blog post, Jared Short shares his advice for what you should think about when building your SaaS products. Check out #2 Build as if you might open source any time. Thanks Jared.
Experiment tracking with MLflow inside Amazon SageMaker
Fantastic post by Cosmin Sanda, Experiment tracking with MLflow inside Amazon SageMaker, walking you through how to support Apache MXNet using MLFlow and Amazon SageMaker. Thank you Cosmin!
Amplify - a play list
The modern equivalent to an 80's mix cassette, Nader Dabit shares with you a curated set of resources to get you started with Amplify. Head over to Up and Running with Amplify Hosting and CI/CD. Thank you Nader.
Manage changes to your databases using Liquibase and Jenkins
In his post Deploy, track, and roll back RDS database code changes using open source tools Liquibase and Jenkins, Harish Shenoy walks you through building a cost effective, database-independent solution in AWS using open source tools Liquibase and Jenkins that help DBAs to deploy, track, and perhaps roll back the database changes as and when developers need them to.
Thank you Harish Shenoy!
Blockchain - deploying HyperLedger Fabric on AWS
In this post, find out how Nix Solutions deploy HyperLedger Fabric 1.4 on AWS. This is a detailed walkthrough, and you will finish up with a sample application using smart contracts.
Do you use AWS and love open source?
Do you have some content you want to share with a broader audience? We are always looking for guest content for the AWS Open blog. Please get in touch (via comments below) and I would love to speak with you about what you are doing in open source. We are always looking for interesting new content.
We are hiring
We are hiring, so if you love open source, then why not check out the current openings. We would love to hear from you and if you want to chat and ask me anything, connect to me via LinkedIn or Twitter and always happy to talk.
Open Source jobs at Amazon and AWS - scroll to bottom, on left.
Stay in touch with open source at AWS
I hope this summary has been useful. I have looked for all the session videos that have been uploaded to date, but if I have missed anything, please get in touch and I will update this summary. Remember to check out the Open Source homepage to keep up to date with all our activity in open source by following us on @AWSOpen.
Oldest comments (0)