DEV Community: Ari 🪐

Creating Chaos and a Giveaway ⚒ 🎁

Ari 🪐 — Sun, 22 Nov 2020 07:04:50 +0000

Introducing Chaos

"Chaos engineering is the discipline of experimenting on a software system in production in order to build confidence in the system's capability to withstand turbulent and unexpected conditions."

I think when most of us hear 'Chaos' we automatically get a little anxious. The idea of presenting failure into anything let alone a system can sound scary, however, by running these controlled experiments we can gain insight and reduce disasters. This is important because inevitably a system WILL fail.

So let's break some stuff and see how it all works!

Below is a repo with step by step instructions to setup Kubernetes clusters and use a few different platforms to run experiments on them.

A lot of tutorials I have found around running chaos experiments with K8s skip the cluster setup bit - making this repo a good complete getting started guide if you're new to K8s and Chaos Engineering. 🙂 🎉

ari-hacks / kubernetes-chaos-sandbox

🧪 Tutorials for running chaos experiments with litmus chaos, chaos mesh, and gremlin (includes k8s setup)

Kubernetes Chaos Sandbox

A sandbox repo of chaos experiment tutorials with Gremlin, Litmus Chaos, and Chaos Mesh

Local Setup

OS: MacOs Catalina
Kubernetes: Docker Desktop

Resource settings

  - CPUs:6
  - Memory: 6GB
  - Swap: 3GB
  - Disk Size: 59.6GB

Quick Links to Tutorials ☝️

Quick Link to Kubernetes setup

Additional tutorials

Setting up Prometheus and Grafana monitoring on K8s

View on GitHub

There are several organizations making waves in the chaos experimenting world today. I choose to learn a bit more about Gremlin, Litmus, and Chaos Mesh.

Each provides various interesting experiments and testing experiences. Gremlin is chaos engineering as a service. This is perfect if you like a good GUI and want to get up and running quickly. Litmus Chaos is a good choice if you enjoy the command line (which I do 😉 ). I had the opportunity to contribute to their project and the team is great! Chaos Mesh is similar to Litmus Chaos and they have some awesome documentation I was able to contribute to this Hacktoberfest as well. Chaos Mesh offers the option of using an interactive GUI or the command line (or both!) making it very developer friendly.

While the purpose of Chaos experimenting is introducing failure the main goal in testing is really to fix and overcome any weaknesses found in a system. By using these tools we are making software and infrastructure more reliable and sustainable.

"The purpose of Chaos Engineering is NOT to 'Break Things on Purpose'. If anything it is to try and 'Fix Them on Purpose.'"

🚨 Before you go.....

I am giving away a $12 voucher to the dev shop! Leave a comment below if interested and I'll pick someone at random this weekend to send it to! 🎁 ✌🏾

Building and Testing Infrastructure with Github Actions 🏗

Ari 🪐 — Fri, 18 Sep 2020 05:35:59 +0000

My Workflow

I've been learning a bit about Terraform this past month and thought it would be cool to leverage IaC in a CI/CD environment with GitHub Actions!

My workflow runs terratest to test the infrastructure then initializes, formats, plans, and applies changes based on the Terraform configuration. Changes are applied after a pull request has been merged.

To ensure Terraform runs in the correct directory the commit and PR message should be the directory name.

Submission Category:

DIY Deployments

Link to Code

ari-hacks / infra-github-actions

Testing and provisioning infrastructure with Terraform and Github Actions

infra-github-actions

View on GitHub

Actions used in my workflow

5 things Terraform can automate in AWS ⚙️

Ari 🪐 — Sun, 26 Jul 2020 16:25:41 +0000

Terraform is an orchestration tool that can provision infrastructure with code. Instead of opening the AWS Management Console to create our resources, we can do that directly in our editor with HCL (HashiCorp's Configuration Langauge).🎉

Let's look at some things we can automate in AWS.

Prerequisites

Note: A step-by-step guide for setup can be found in the Github README.md

TODOs

Launch an EC2 Instance
Create an S3 Bucket
Create a backend state with S3
Create IAM group and policy
Add IAM users to a group

Launch an EC2 Instance

#variables.tf
variable "region" {
  description = "AWS region"
  default     = "us-east-2"
}

variable "ami" {
  default     =  "ami-0c8110836d05ad7bd"
}

#main.tf
provider "aws" {
  region  = var.region
}

terraform {
  required_version = ">= 0.12"
}

resource "aws_instance" "ec2_example" {
  ami           = var.ami
  instance_type = "t2.micro"
}

In this example, we are creating an AWS resource of type aws_intstance with a unique name ec2_example. This resource spins up an ec2 server by launching a copy of the AMI(Amazon Machine Image).

The AMI is a dynamic value and needs to be updated when creating instances. You can find these on the AWS console or search for ubuntu images here (For t2.micro select hvm:ebs-ssd).

EC2 directory

Create an S3 Bucket

#variables.tf
variable "region" {
  description = "AWS region"
  default     = "us-east-2"
}

#outputs.tf
output "s3-bucket" {
  value = aws_s3_bucket.b_example.bucket
}

#main.tf
provider "aws" {
  region  = var.region
}

resource "aws_s3_bucket" "b_example" {
  bucket = "bucket-example-tmed232323"
  force_destroy = true
  acl    = "private"

  versioning {
    enabled = true
  }
}

An s3 bucket is used for storage. To create a bucket a unique name is required. If you do not add one AWS will create one for you. To grant access to a bucket acl can be used. By default it is set to private. force_destory=true is set for testing purposes. This will allow us to delete the bucket with terraform destroy even if it is not empty.

versioning keeps different variants of an object in the bucket and outputs.tf will output the results of the file to the console after an apply.

S3 Bucket directory

Create a backend state with S3

#main.tf
terraform {
  backend "s3" {
    bucket = "bucket-example-tmed232323"
    key    = "terraform-aws-automation/create-s3-backend-state/terraform.tfstate"
  }
}

In this example, remote state is being stored with a terraform.tfstate file. This is common practice when using Terraform with more than one person so state does not get locked and only one person can make changes at a time on the latest copy. Since the bucket we created is using versioning there will be a history of changes stored as well.

To add an object to a bucket the unique bucket name is required along with the key which defines the path of the file that's created in AWS.

S3 backend state directory

Create IAM group and policy

#variables.tf
variable "region" {
  description = "AWS region"
  default     = "us-east-2"
}
#main.tf
provider "aws" {
  region  = var.region
}

resource "aws_iam_group" "admin_example" {
  name = "admin_example"
}

resource "aws_iam_policy_attachment" "admin-attachement" {
  name       = "admin-attachement"
  groups     = [aws_iam_group.admin_example.name]
  policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess"
}

Groups are collections of IAM users that share specific privileges or policies.

Two resources are created here, one for the IAM group and another to attach a policy to that group. The attribute policy_arn is set to attach the Administrator Access policy to the IAM group.

IAM Group and Policy directory

Add IAM users to a group

#variables.tf
variable "region" {
  description = "AWS region"
  default     = "us-east-2"
}

#main.tf
provider "aws" {
  region  = var.region
}

resource "aws_iam_user" "admin_example_1" {
  name = "admin_example_1"
}

resource "aws_iam_user" "admin_example_2" {
  name = "admin_example_2"
}

resource "aws_iam_group_membership" "admin-user-group-example" {
  name = "admin-user-group-example"
  users = [
    aws_iam_user.admin_example_1.name,
    aws_iam_user.admin_example_2.name,
  ]
  group = "admin_example"
}

After an IAM group is created we can add new users to it. This is down with the resource type aws_iam_group_membership. Users are passed to this resource along with the name of the group.

Add users to group directory

To run these examples locally, clone the repo and navigate to the root directory. In your terminal cd into one of the above directories and follow these steps:

Initialize Terraform: terraform init
Check the plan to make sure the configuration will do what we expect: terraform plan
Apply the execution plan and build the stack: terraform apply
Check the resource is up: terraform state list or terraform state show 'type.name' ex. aws_instance.ec2_example
Tear down all provisions: terraform destroy

If you found this article useful give the repo a ⭐️ and check back later for more examples of automating AWS with Terraform. 🙂✌🏾

ari-hacks / terraform-aws-automation

⚙️ Examples of provisioning AWS resources with Terraform

Automating AWS with Terraform

External Resources

Amazon EC2 AMI Locator

Pre-requisite Setup

AWS Account - Free Tier

Create an IAM user

Log into your root AWS account
Select Services > IAM
On the left nav bar select 'Users'
Select 'Add user'
Create a username 'terraform-admin'
Select AWS access type as Programmatic access (and AWS management console access to view resources on the dashboard)
Select Next: Permissions
Select Create group
Add a group name 'admins'
Check AdministratorsAccess and Create group, check group
Select Next: Tags
Select Next: Review
Select Create User
You will need the generated access key Id, and the secret access key (Download the csv provided or store these values)

…

View on GitHub

Explain IaC like I'm Five

Ari 🪐 — Sun, 19 Jul 2020 03:35:03 +0000

IaC like I'm Five
What is Infrastructure as Code
Why should we use it
How do we use it
Terraform Example
Other IaC Tools

What is Infrastructure as Code?

Historically, creating infrastructure involved manually setting up servers and configuring systems to run applications, operating systems, databases...etc. Infrastructure was imperatively configured and maintained, meaning this was done with step by step instructions and executed by the admin. This process becomes unsustainable with large & complex systems leaving room for issues with cost, availability, and consistency.

Infrastructure as Code solves these problems by automating setup and management by writing code in a high-level descriptive language.

With Iac we can produce secure,reliable, and robust infrastructure

HashiCorp's Founder, Armon Dadgar explains IaC simply and eloquently in this short video

Why should we use it?

Accidents happen, human error happens. By automating infrastructure with code we reduce error and produce more reliable code.

Complexity, with the emergence of cloud-based infrastructure IaC is paramount for scaling distributed systems, cloud-native applications, and service-based architecture.

How do we use it?

Sai Vennam with IBM Cloud discusses approaches to IaC with a quick example

Terraform Example

Terraform is a cloud provider agnostic orchestration tool. This means we can build, version, and destroy resources across multiple providers at the same time. We can also easily look up modules to use directly from the Terraform Registry saving time on creating common configurations. These features make it a leader in IaC. Now let's take a look at the power of Terraform with an example.

This is a Terraform module which creates a private S3 bucket resource on AWS with versioning enabled in HCL(HashiCorps's configuration language)

Other IaC Tools

Another Orchestration tool is AWS CloudFormation, this tool is similar to Terraform but it is only AWS focused.

Along with orchestration tools, there are configuration management tools. They are used to install and manage software on existing servers.

Common configuration management tools:

** Some configuration management tools can be used for orchestration. It all depends on the best fit for your needs or the task.

Conferences in the Time of COVID-19: Cloud and Infrastructure

Ari 🪐 — Sat, 11 Jul 2020 06:03:11 +0000

As you all know due to the pandemic traditional conferences have been canceled this year. Alternatively, a new era of virtual conferences has emerged and I was very surprised by the efforts and content available out there for free.

Here is a list of events focusing on DevOps, Reliability Engineering, and Cloud Development. You can watch all these conferences for FREE right now. Also listed are some upcoming events.

Virtual Events
Failover Conf
DevSecCon24
Camp Cloud Native
Dockercon2020
SRE SKILup Day by DevOps Institute
INS1GHTS2020
Future Events

Failover Conf

Failover Conf. focused on Reliability and Resilience Engineering. It was created by Gremlin, one of the leaders in the Failure as a Service Industry. If you are interested in Chaos Engineering this is a great watch.

All the sessions can be found here on youtube.

DevSecCon24

DevSecCon24 was geared towards DevSecOps and features 24 hours worth of content!

Check out all the presentations here on youtube.

Camp Cloud Native

Camp Cloud Native was a 12-hour event featuring sessions on the cloud native ecosystem with a focus on education accompanied by a slack discussion forum. With some camp stories and demos, this event made for a fun watch.

All the presentations can be found here on youtube.

Dockercon2020

Dockercon covered the container ecosystem. There are some great interviews, talks, and Q&A sessions that come with this virtual event.

All the sessions can be found on demand and viewed after registering.

SRE SKILup Day by DevOps Institute

Once a month the DevOps Institue host MicroConferences with the aim of highlighting the people, process, and technology within a selected practice. For June the topic was Site Reliability Engineering. To view these events you can sign up for free with DevOps Institue and watch the sessions on demand at no cost. SRE SKILup day is not available on demand just yet but previous content includes Value Stream Management and Enterprise Kubernetes.

In addition to the excellent speakers, these one-day events include chat rooms, interactive sponsor booths, raffles from the sponsors, an attendee rewards leaderboard (winners get some cool prizes), scavenger hunts, wellness videos, and a "challenge" feature that is interactive with the community.

INS1GHTS2020

INS1GHTS2020 was a one-day event hosted by NS1. Sessions were delivered by industry leaders in NetOps, DevOps, and application delivery.

Much like some of the other events, INS1GHTS2020 offered an active slack channel that you can still gain access to.

Also much like SKILup, this event was packed with opportunities to get some swag and sponsor prizes.

All the presentations can be found on demand and viewed after registration.

Future Events

SKILup Days Future MicroConferences
- July 16th Continuous Delivery Ecosystem
- August 20th Observability
- September 17th DevSecOps
- October 15 AIOps & MLOps
- November 12th Continuous Testing
Septemember 22 - 24 DevOps World (Note: the
event is free but some workshops are not)

Even though these events are not like your average conference experience they are a great quarantine getaway!

Hope you enjoy 🙂✌🏾

freeCodeCamp has Python Certifications

Ari 🪐 — Sat, 04 Jul 2020 08:59:09 +0000

In addition to the JavaScript certifications offered, freeCodeCamp recently rolled out some Python certs to join in!

These are the 4 new certifications and the material they cover:

1. Scientific Computing with Python

Python for Everybody
An introduction to Python with some regEx, networking, web, database, and even data visualization material.

Scientific Computing with Python Projects
Arithmetic Formatter, Time Calculator, Budget App, Polygon Area Calculator, and Probability Calculator

2. Data Analysis with Python

Data Analysis with Python Course
An introduction to data analysis, Jupyter Notebook, Numpy, Pandas, data cleaning, reading data, parsing data, and some more core Python.

Numpy
An overview of Numpy and its applications.

Data Analysis with Python Projects
Mean-Variance-Standard Deviation Calculator, Demographic Data Analyzer, Medical Data Visualizer, Page View Time Series Visualizer, and Sea Level Predictor

3. Information Security

Information Security with HelmetJS
This is a bit of JS in the mix covering web security and hashing.

Python for Penetration Testing
Material on Sockets, TCP, and Developing an Nmap Scanner, Banner Grabber, and Port Scanner.

Information Security Projects
Stock Price Checker, Anonymous Message Board, Port Scanner, SHA-1 Password Cracker, and a Secure Real-Time Multiplayer Game

4. Machine Learning with Python

TensorFlow
Material on TensorFlow, Machine Learning, Algorithms, Neural Networks, NLP, and Reinforcement Learning.

How Neural Networks Work
Material on Neural Networks, Deep Learning, and Convolutional Neural Networks.

Machine Learning with Python Projects
Rock Paper Scissors, Cat and Dog Image Classifier, Book Recommendation Engine using KNN, Linear Regression Health Costs Calculator, and a Neural Network SMS Text Classifier.

Note: A lot of this material is video-based while freeCodeCamp is building out the interactive content.

You can read more about the certifications here, Quincy Larson talks about the update as well as some more info on the next release. 🙂✌🏾

Python...in the browser!? 🤯 : A Web Monetization x Web Assembly Experiment

Ari 🪐 — Wed, 20 May 2020 04:38:34 +0000

What I built

I built a demo hub where users can experiment with data science by graphing data sets in the browser then downloading them. Web Monetized browsers won't see the Ads on the page and can also experiment with additional graphing tools.

Note: This project is in the demo phase. In the future, I’d like to build out the graphing tools to handle more sophisticated data sets. As well as add a login/dashboard feature. 🙂

Why I built it

While researching what to build for the Grant For the Web Hackathon I initially thought of creating something with a JavaScript framework since I'd be interacting with the browser API. However, my journey here on Dev.to is to learn more about Python and what we can do with it. So I thought about how I could incorporate Web Monetization with Python. After some digging, I found this cool project created by Mozilla called Pyodide!

"Pyodide is an experimental project from Mozilla to create a full Python data science stack that runs entirely in the browser."
~ Michael Droettboom

Some of you may be thinking what's wrong with just using JavaScript? JavaScript is great but can you imagine a world where you can use mature data science libraries in the browser to quickly create data science tools and machine learning models? Not to mention being able to share those resources over the web? AND being able to make some money in the process?! 🤯🙌🏾

Pyodide is currently in its infancy and there are caveats and performance drawbacks but I'm excited to see where it goes!

Submission Category:

Creative Catalyst
Exciting Experiments

Demo

Link to Code

ari-hacks / the-hub

📈📊 A hub where users can experiment with graphing and Python in the browser (https://pyodide-experiment.herokuapp.com/)

Data Hub 📈

About

A demo hub where users can experiment with data science by graphing data sets in the browser then downloading them. Web Monetized browsers won't see the ads on the page and can also experiment with additional graphing tools.

This Project is a Grant For the Web x DEV Hackathon Runner-Up Winner 🏆🎉

✍️ Blog Post

Features

csv upload
Enhanced Plots with Plotly
Web Monetization enabled
Image download

Testing Data

Plot	Input File
Pie	Comparison of trending operating systems
Scatter/Line	Trends of funny zoom backgrounds
Time Series	Comparison of trending noodles
Geo Map	International earthquake data
Heat Map	Random Data Set
3-D	Random Data set

Set up

Requirements

Python 3.7
Pipenv
Browser with coil wallet (No Sign in required for testing)
~~Get Latest Pyodide Downloaded (pyodide.js)~~ Using CDN
Bookmarklet : For testing coil without an account

Local development

After the above requirements have been met:

Clone this…

View on GitHub

How I built it

Stack

Web Monetization Api - Exclusive Content
Web Monetization Api - Removing Ads
Pyodide
Python | JavaScript | HTML | CSS

Additional Resources/Info

Mozilla
Test Web Monetization
Webassembly.org
UI Template
Data Sets for Testing
More on Pyodide

#212 Python in Web Assembly with Pyodide

Talk Python

Your browser does not support the audio element.
1x initializing... ×

A Reading List & Repo List 📚 for Learning DevOps, SRE, and Automation(w/Python)

Ari 🪐 — Thu, 07 May 2020 18:59:00 +0000

When learning new things I find that I fall into the same practice of adding countless bookmarks to my browser, keeping wayyyyy more tabs open than necessary, and admittedly searching the web trying to find the perfect tutorial or resource to get started. So I decided to put some of the resources I'm finding helpful in this post to help anyone else who finds themselves in an endless google search. 😅

My main goal is to learn more about DevOps/SRE best practices and role responsibilities, AWS, and automation with python.

If you are on a similar learning path take a look at this list and PLEASE add more in the comments if you've found some great resources or projects/tools that helped you! 🙂✌🏾

Githubs

dastergon / awesome-sre

A curated list of Site Reliability and Production Engineering resources.

Jump to DevOps Tools in awesome-python

vinta / awesome-python

An opinionated list of awesome Python frameworks, libraries, software and resources.

gremlin / sre-tools

A collection of SRE tools

Books

Additional Resources

A Cloud Guru - A great place to learn about cloud technology and to prep for AWS certs.
Gremlin - Testing all the things in production!
Slack Channels:

A Smart-ish Covid ChatBot 🤖

Ari 🪐 — Thu, 30 Apr 2020 18:27:40 +0000

What I built

I built a ChatBot that communicates via Twilio's API on WhatsApp. The Bot gives Covid-19 statistics from the US & the UK. It also provides some positive news articles to read in light of recent times.

Category Submission:

COVID-19 Communications
Interesting Integrations

ari-hacks / covid-chatbot

🤖 A Twilio Hackathon Project

Covid-19 ChatBot 🤖

About

This is a simple ChatBot that communicates via Twilio's API on WhatsApp. The Bot gives Covid statistics from the US & the UK. It also provides some positive news articles to read in light of recent times.

How it works

Users interact with WhatsApp to communicate with the ChatBot. The Bot replies to greetings along with different variations of the following questions.

Sample Questions:

Input	Output
Greeting!	Hi
Hi, How is it going?	Good
uk confirmed cases?	Confirmed Uk Cases 165221
How many uk confirmed cases?	Confirmed Uk Cases 165221
What are the us covid stats?	United States Stats: confirmed: 1055303 recovered: 144423 critical: 18665 deaths: 61112
uk stats	United Kingdom Stats: confirmed: 165221 recovered: 1918 critical: 1559 deaths: 26097
what are the uk covid stats?	United Kingdom Stats: confirmed: 165221 recovered: 1918 critical: 1559 deaths: 26097
how many us recovered cases are there?	Recovered Us

…

View on GitHub

How I built it

Stack

Python web framework using FastApi
Twilio API for WhatsApp
Automated responses and bot training with Chatterbot

Some Issues

Setting up ChatterBot produced issues with the nltk package.
It took some time to find the right way to set up deployment in Heroku, FastApi is a relatively new Python framework so there are not as many resources for it as Django or Flask.
Querying news articles based on keywords did not always produce 'good' news, however, I selected The Guardian API because you can also query on basic logic operators.
The Chatbot doesn't always return the desired results.

Things I learned

Since this is my first Hackathon and I am quarantined in another country I decided to use the time and opportunity to learn all new things! This is the first API I've created with python. I decided on FastApi because of its simplicity and great features (especially the built-in docs 🙌🏾).

I also wanted to add a machine learning component so after some research, I landed on ChatterBot. I think it is a great introduction for someone wanting to learn some of the basics of ML.

Additional Resources/Info

I used the Guardian API to get news articles and a Covid-19 API from rapidapi.com to get coronavirus statistics.

DEV Community: Ari 🪐

Creating Chaos and a Giveaway ⚒ 🎁

ari-hacks / kubernetes-chaos-sandbox

🧪 Tutorials for running chaos experiments with litmus chaos, chaos mesh, and gremlin (includes k8s setup)

Kubernetes Chaos Sandbox

Local Setup

Quick Links to Tutorials ☝️

Quick Link to Kubernetes setup

Additional tutorials

🚨 Before you go.....

Building and Testing Infrastructure with Github Actions 🏗

My Workflow

Submission Category:

Link to Code

ari-hacks / infra-github-actions

Testing and provisioning infrastructure with Terraform and Github Actions

infra-github-actions

5 things Terraform can automate in AWS ⚙️

Prerequisites

TODOs

Launch an EC2 Instance

Create an S3 Bucket

Create a backend state with S3

Create IAM group and policy

Add IAM users to a group

ari-hacks / terraform-aws-automation

⚙️ Examples of provisioning AWS resources with Terraform

Automating AWS with Terraform

External Resources

Pre-requisite Setup

AWS Account - Free Tier

Create an IAM user

Explain IaC like I'm Five

What is Infrastructure as Code?

Why should we use it?

How do we use it?

Terraform Example

Other IaC Tools

Conferences in the Time of COVID-19: Cloud and Infrastructure

Failover Conf

DevSecCon24

Camp Cloud Native

Dockercon2020

SRE SKILup Day by DevOps Institute

INS1GHTS2020

Future Events

freeCodeCamp has Python Certifications

These are the 4 new certifications and the material they cover:

1. Scientific Computing with Python

2. Data Analysis with Python

3. Information Security

4. Machine Learning with Python

Python...in the browser!? 🤯 : A Web Monetization x Web Assembly Experiment

What I built

Why I built it

Submission Category:

Demo

Link to Code

ari-hacks / the-hub

📈📊 A hub where users can experiment with graphing and Python in the browser (https://pyodide-experiment.herokuapp.com/)

Data Hub 📈

About

This Project is a Grant For the Web x DEV Hackathon Runner-Up Winner 🏆🎉

Features

Testing Data

Set up

Requirements

Local development

How I built it

Stack

Additional Resources/Info

#212 Python in Web Assembly with Pyodide

Talk Python

A Reading List & Repo List 📚 for Learning DevOps, SRE, and Automation(w/Python)

Githubs

dastergon / awesome-sre

A curated list of Site Reliability and Production Engineering resources.

vinta / awesome-python

An opinionated list of awesome Python frameworks, libraries, software and resources.

gremlin / sre-tools