DEV Community: Rohit

How IaC can streamline the Infrastructure & Configuration

Rohit — Wed, 05 Jun 2024 17:18:17 +0000

In today's fast-paced digital landscape, the ability to quickly and reliably deploy infrastructure is critical for product's success. As a Technical Project Manager, I've got a chance to transformative power of Infrastructure as Code (IaC) tools in streamlining infrastructure management and subsequently helped the organization in healthy infra setup and successful project delivery. In this article, I’ll explore the concept of IaC, discuss popular IaC tools, and share insights from a recent project where we successfully implemented IaC.

What is Infrastructure as Code (IaC)?

First lets understand some basics about IaC. Infrastructure as Code (IaC) is a practice where infrastructure configurations are written and managed as code. In overall, this approach allows for the automation of infrastructure provisioning & management, enabling consistency, scalability and automate the repeatability. By using infrastructure as code, engineers & designers can version control their configurations changes, enhance collaboration & communication, and deploy infrastructure changes with greater confidence & security.

Popular IaC Tools:

In the market, there are several tools that have emerged as leaders in the IaC space. Each of them offering unique features and benefits. Here are few widely used IaC tools in the market:

Terraform: This is an open-source tool developed by HashiCorp, Terraform enables infrastructure provisioning across various cloud providers. It uses a declarative language to define infrastructure resources and supports modular configurations.
AWS CloudFormation: Its a service by Amazon Web Services cloud platform. AWS CloudFormation allows users to define and provision AWS infrastructure using JSON or YAML templates. It can easily integrates seamlessly with other AWS services.
Ansible: One of the most popular configuration management tool that also supports infrastructure provisioning. Ansible uses a simple, agentless approach, making it easy to set up and use.
Pulumi: Another IaC tool that supports multiple cloud providers. This tool allows users to write infrastructure code in familiar programming languages like TypeScript, Python, and Go.

IaC in Implementation: A Project Case Study

In a recent project, team faced the challenge of managing complex, multi-environment infrastructure. We decided to adopt Terraform due to its provider-agnostic nature, easily integration with cloud platform and strong community support. Here’s how we implemented IaC:

Overall Architecture Diagram (in text format)

IaC Implementation Steps

Requirement gathering & Analysis: Firstly team has started gathering requirements from business, did the feasibility study and strategize how we can implement the infrastructure for each development, staging, production environment. Identified the resources needed (such as VMs, databases, networks). List down the overall scope, divide into smaller breakdown, calculate the required talents (engineers and designers) and finally the budget required. Presented to the management for the project initiation and budget approval.
Defining the Infrastructure: Based on the agreed scope, we started writing the Terraform configuration files to define our infrastructure. Each environment had its own set of configuration files, allowing for isolation and customization. Worked with the team to design and implement within the agreed timelines.
Version Control: One of the critical activity to control the various versions of configuration setup. We stored our Terraform code in a Git repository, enabling version control and collaboration among team members.
Continuous Integration/Continuous Deployment (CI/CD): Most important steps to automate the integration and deployment cycles for every new version release. We integrated Terraform with our CI/CD pipeline using Jenkins. Every change to the infrastructure code triggered a pipeline that validated and applied the changes.
Monitoring: Once the deployment is done, it is always important to monitor the setup and address immediately in case of any issue. Post-deployment, we used Terraform's state management to keep track of the infrastructure, configuration and made use of monitoring tools like Prometheus to ensure everything was running smoothly.

Benefits After IaC Implementation:

Reduced Errors: With the help to automation, we minimized the manual intervention and therefore reduced the likelihood of errors.
Scalability: This IaC implementation allowed us to scale our infrastructure effortlessly based on the demands and business needs.
Improved Consistency: By defining infrastructure as code setup, we ensured that every environment was consistent and reproducible.
Collaboration: With the help of version control, we noticed far better co-ordination within the team, which subsequently facilitated better collaboration and code reviews. This leads to higher quality infrastructure code with less errors.
Reduced Operation Cost - With automation, we have received significant reduction in operation cost (15%) and shoot-up the overall profit for the organization.

Conclusion!

I must say implementing Infrastructure as Code has been a game-changer in my tech project management toolkit. The ability to define, provision, and manage infrastructure through code has brought about unprecedented efficiency and reliability. I encourage my fellow project managers to explore IaC tools and consider integrating them into their projects for a more streamlined and scalable infrastructure management process.

Happy Learning!

Host web application on EC2 instance with Ansible Playbook

Rohit — Mon, 03 Jun 2024 10:35:48 +0000

Ansible is an open-source automation tool used for configuration management, application deployment, orchestration, and task automation. It is designed to simplify complex IT workflows and streamline repetitive tasks across large infrastructures.

Project Summary

In this project, lets learn about launching an application with configuration management tool Ansible. Launching a web application using an Ansible playbook involves several steps, including setting up the infrastructure, installing necessary software, and deploying the application. Here is a simplified example of an Ansible playbook that can help you deploy a web application

Prerequisites
1. AWS Account - you should have a valid AWS account to launch EC2 instances for installing ansible and hosting an application.
2. Linux Background - should have some background on Linux as we will be running various commands.

Step 1 - Launch AWS EC2 instance using Ubuntu & CenOS

Launch Ubuntu Instance for Ansible - Use t2.micro instance type (Free tier) with name as 'Control'. This is the instance where we will install the Ansible and use it as control machine for other webservices or application (installed on another EC2 instance).

Key Pair - Create new key pair with control-key.pem file

Security Group - Create new security group with inbound rules to allow port 22 from my IP address and launch the instance.

Launch CentOs Instance for web services - Use t2.micro instance type (Free tier) with name as 'web01'. This is the instance where we will deploy web application. For OS, go to 'Browse more AMI'-> AWS Marketplace AMIs and search for centos9.

Key Pair - Create new key pair with client-key.pem file.

Security Group - Create new security group with inbound rules to allow port 22 from my IP address & from Ansible security group too. Also allow port 80 (http) from my IP address. After this launch the instance.

Step 2 - Installing Ansible on Ubuntu OS

SHH into Control Instance - SSH into the control EC2 instance and to login.

Install Ansible - Using below mentioned commands, Install Ansible software in your control EC2 instance.



  $ sudo apt update
  $ sudo apt install software-properties-common
  $ sudo add-apt-repository --yes --update ppa:ansible/ansible
  $ sudo apt install ansible

Step 3 - Create project directory, Inventory and playbook files

Create new directory - After Ansible installation, create a new directory 'project1' where we will create the Ansible playbook for web appl.

Inventory file - Create a inventory file inside the 'project1' directory. The purpose of this file is to defines the hosts and groups of hosts upon which commands, modules, and tasks in a playbook operate. Sample inventory file code will looks like below. Make sure the spacing or indentation is correct while using inventory file.



all:
  hosts:
    web01:
      ansible_host: 172.31.17.59
      ansible_user: ec2-user
      ansible_ssh_private_key_file: clientkey.pem



**explanation:-**
hosts = web01 (give you web instance
ansible_host = should be the private IP address of your web01 instance
ansible_ssh_private_key_file = should be the private key that you use to authenticate with web01 instance

Client Key - Since we are using clientkey.pem in our inventory file, we need to copy the private key of client EC2 instance to 'project1' directory with the same name that we have given in inventory file i.e clientkey.pem. This is to authenticate the web01 server.

Key Permission - After copying the private key to clientkey.pem and place it into 'project1' directory, you need to modify the permission of the key.

Current Permission:

Modify the permission using below command:-



chmod 400 clientkey.pem

After modification, it should be as per below:-



ubuntu@control:~/project1$ ls -l
total 16
-r-------- 1 ubuntu ubuntu 1675 Jun  3 09:18 clientkey.pem

Ansible Playbook - An Ansible playbook is a file written in YAML (Yet Another Markup Language) that defines a series of automation tasks to be executed on managed nodes. Playbooks are the heart of Ansible's configuration management, allowing users to describe configurations, deployments, and orchestrations in a straightforward, human-readable format. Lets create Ansible playbook with name 'web01.yaml'.



---
- name: Webserver setup
  hosts: web01
  become: yes
  tasks:
    - name: Install httpd
      ansible.builtin.yum:
        name: httpd
        state: present

    - name: Start service
      ansible.builtin.service:
        name: httpd
        state: started
        enabled: yes

    - name: Copy the index file or template
      copy:
        src: myfile/index.html
        dest: /var/www/html/index.html



**explanation:-**
name - This is the name of the ansible playbook or task, you can give based on your choice.
hosts - This is the host name which you have given in the inventory file
tasks - It stores the list of tasks that you want to execute automatically

Step 4 - Run and Test the application

Run - Finally its time to run the command as per below to execute the playbook and deploy the application to web server (which is web01 EC2 instance)



 ansible-playbook -i inventory web01.yaml

Output should be as per below:-

See Application on Browser - Lets take the public Ip address of the web01 instance and paste into the browser and a simple web application should be launched:- Note:- Make sure that security group of web01 instance should allow inbound rule for port 80 through control (ansible) instance from my IP or anywhere. If you are not doing this then it will timeout.

Conclusion!
Deploying a web application using Ansible on an AWS EC2 instance offers a powerful and automated solution to streamline the process of provisioning infrastructure, installing necessary software, and managing application deployments. By leveraging Ansible's playbooks, you can ensure consistent, repeatable, and scalable deployments, significantly reducing manual intervention and potential errors. This approach not only enhances operational efficiency but also allows for rapid iteration and deployment of web applications. Whether you are managing a single server or a complex infrastructure, using Ansible to automate the deployment process on AWS EC2 instances empowers you to focus more on developing and improving your application rather than dealing with infrastructure complexities.

Happy Learning!

Integrate Slack with Jenkins to receive CICD pipeline notification

Rohit — Sat, 01 Jun 2024 08:44:02 +0000

Integrating Slack with Jenkins allows you to receive build notifications directly in your Slack channels, which helps in monitoring the status of your CI/CD pipeline. Here’s a step-by-step guide to set up this integration:

1) Create Slack Account - First create a Slack account and then subsequently the slack workspace and channel:-

Give your workspace name and some more details:-

Select your team mates to include them. If you are individual then you can skip this. Select the free trial:-

Once you selected, Dashboard will look like:-

Create a channel (I have given name as 'cicd-implementation')

2) Create Token - Now we need to create a token for Jenkins to authenticate slack and for that we need to add an app into our slack account. Refer below steps:-

Do a google search 'slack app'

Open the first link ‘Add app to Slack’ and search for Jenkins:-

Click on ‘Add to Slack’

Select the channel and ‘Add Jenkins to CI Integration’. For me it would be 'cicd-implementation'

In the same page, refer the steps to integrate slack with Jenkins. Also copy the token given in step 3 and save it somewhere. Now come down and click on ‘Save Settings’

3) Integrate Jenkins & Slack - Now we will go to Jenkins and integrate the slack by using plugins and slack token.

Go to Jenkins (Manage Jenkins-> Plugins-> Available Plugins) and look for slack notification plugins and install it.

Now, lets integrate the slack with Jenkins. Go to Manage Jenkins-> Systems and look out for Slack plugin.

Get your Slack workspace name correctly as this could cause issue when you are going to test connection in Jenkins. Refer below to get your slack workspace name:-

Provide workspace name based on slack setup and click add credential

Select 'Secret text', add the token which you have copied earlier and then give some ID & description. Later click on Add button.

Select the 'slackpass', give the channel id and then click on test connection.

A 'success' message should show over here to make sure there is proper integration between Jenkins and Slack.

Note:- It might be possible that you may get Failure message here, this could be because of some setup or plugins issue. So you can just logout from your slack account, login back and do the whole setup again. You should be able to see success message.

Lets add these things in our pipeline code. Add below set of codes to your pipelines. Here define a function to do color mapping in slack. In slack 'good' means green color and 'danger' means red.

Its time to create a new pipeline and paste your Pipeline as a code to Jenkins and test.

Go to Slack channel and see if there any message coming in with 'Success'?

If the pipeline failed then you will see the red color in your slack channel with 'Failed' message

Conclusion!

This configuration sends notifications to Slack on build success and failure with links to the Jenkins build.

By following these steps, you should be able to integrate Slack with Jenkins and receive notifications in your specified Slack channels.

Happy Learnings!

Security Protocol to connect AWS SES

Rohit — Thu, 30 May 2024 18:25:51 +0000

Lets take a look on another AWS services know as Amazon SES and understand the security protocol around that. This could be useful when you send email to receiver and how we can securely deliver it.

Lets learn about Amazon SES service first with some basic details:-

What is Amazon SES

Amazon SES stands for Simple Email Service. It is one of the cost-effective, scalable, reliable email service designed to help the organization, application developers, digital marketers to send notification, and transactional emails. It is one of the powerful service for all kinds of business to integrate with various others AWS services to provide a robust platform for managing and optimizing the organization's email communications.

Integrating Amazon SES into your applications, whether via the API or SMTP interface, requires strict adherence to security protocols to ensure email delivery integrity and protect the data. Here are detailed best practices for securing your Amazon SES integration:-

1) Amazon SES API Integration Security - When using Amazon Simple Email Service (SES) to send emails via the API, it's essential to utilize Transport Layer Security (TLS) to encrypt your data and protect it from potential threats.

Transport Layer Security (TLS) is a well known cryptographic protocol designed to provide secure communication over a computer network. Its a critical protocol that provides and ensures integrity, data security, and authentication between your application and the Amazon SES service.

Steps to Securely Use Amazon SES API with TLS

Use IAM Policies - Use below IAM policies to make sure that IAM user has enough permission to send email using amazon SES.

AWS SDK and HTTPS - We can utilize AWS SDK or HTTPS request to make API request to AWS SES. When you use the Amazon SES API over HTTPS, TLS encryption is automatically applied to all API requests.
HTTPS API Request - We can use HTTPS endpoint to make sure that data is fully encrypted.

Simple diagram created in text editor for TLS Encryption with Amazon SES API

2) Amazon SES SMTP Interface Security - For securing the communication channel, there are two primary methods are required such as STARTTLS and TLS Wrapper.

STARTTLS - It is a command used to upgrade an existing plaintext connection to a secure, encrypted connection using Transport Layer Security (TLS).

Lets understand how STARTTLS Works:

The client connects to SMTP server over plaintext connection.
Send STARTTLS command to the server.
Server send the response and initiate TLS handshake.
Communication is encrypted between client and server once TLS handshake is complete.

TLS Wrapper - This is also known as SMTPS or "SMTP over TLS," establishes an encrypted connection from the outset.

Lets understand how TLS Wrapper Works:

The client connects to the SMTP server using a specific dedicated port 465 for TLS.
The TLS handshake occurs immediately upon connection, before any SMTP commands are exchanged.
Communication from the start is encrypted and provide robust security.

Simple diagram created in text editor with STARTTLS and TLS wrapper with Amazon SES API

Conclusion

Overall, Its crucial to implementing these security protocols when using Amazon SES, whether through the API or SMTP interface to maintaining the integrity, security and confidentiality of your communications.

Happy Learning!!

Services that can integrate with AWS Certificate Manager and their Use Case

Rohit — Sun, 19 May 2024 19:17:03 +0000

Lets see what are the services which can be integrated with ACM (AWS Certificate Manager). This could be one of the interview question so lets learn the services together with some basic details about ACM:-

What is ACM

AWS Certificate Manager (ACM) is a service that simplifies the process of managing SSL/TLS certificates for your AWS-based applications and websites. SSL/TLS certificates are used to secure network communications and establish the identity of websites over the internet as well as resources on private networks.

Key features the ACM provides:-

Certificate Issuance- ACM offers automatic issuance of SSL/TLS certificates and its a free service. You only need to pay for the AWS resources you use for your application.
Auto Renewal - IF you are using SSL/TLS certificates from ACM then it gives you auto renewal facility without any hassle.
Easy Cert Management - ACM provides easy centralized management console to view and manage your certificates.
Import Third Part Cert - You can easily import SSL/TLS certificate from third part too and store in ACM. However, there is no auto-renewal facility for 3rd party certificate in ACM, but you can still automate the renewal process.

ACM can support following services

1) AWS CDN - Also knows as Amazon CloudFront, which is a content delivery network, designed to deliver content with low latency and very high speed.

Use Case - Integrate with ACM to distribute content globally in a secure HTTPS manner. Benefit of this is to utilize SSL/TLS cert for secure content delivery to the worldwide user base.

2) ELB - Know as Elastic Load Balancer. AWS ELB is used to balanced the incoming traffic across multiple targets such as EC2 instance.

Use Case - To distribute secure traffic, load balancer required to install SSL/TLS certificate. ACM integrates with Application load balancer or Network load balancer to encrypt the traffic.

3) Amazon API Gateway - It's a serverless fully managed service provided by AWS that allows us to create, publish, maintain, monitor, and secure APIs at any scale.

Use Case - After API deployment, you can use custom domain name to access it and that domain name can be integrated with SSL/TLS certificates to securely connect your API with back end service and for that we can use ACM service.

4) AWS Elastic Beanstalk - Another AWS fully managed service which makes it easier for anyone to deploy, manage, scale web applications without worrying about the infrastructure. It reduces infrastructure complexity which actually helps the developers to focus on writing code and deploy the application.

Use Case - Elastic Beanstalk use elastic load balancer and there can be easily configure SSL/TLS for your Elastic Beanstalk environments to secure traffic.

5) AWS CloudFormation - Its a Infrastructure as Code (IaC) service provided by Amazon Web Services (AWS) which allows you to define and provision AWS infrastructure using code. It allows to create a template that automatically provisions and configures AWS resources as specified in the template, handling dependencies and order of operations.

Use Case - Automate the provisioning and updating of ACM certificates as part of your CloudFormation templates.

6) AWS CloudWatch - Its a monitoring service which helps us to monitor and manage your applications by collecting and tracking metrics, collecting and monitoring log files, and setting alarms in case of any notification & further action.

Use Case - ACM usually Integrate with CloudWatch for monitoring certificate metrics and setting up alarms for certificate expiry.

7) AWS OpsWorks - It is an configuration management service that provides managed instances of Chef and Puppet, which are automation platforms that allow us to automate how servers are configured, deployed, and managed.

Use Case - To automate operational tasks on AWS resources. Use ACM certificates to secure OpsWorks Stacks and manage SSL/TLS certificates automatically.

Rest of the other services mentioned in the diagram are also using ACM somewhere in a similar fashion.

Conclusion

Overall, AWS Certificate Manager is a key service offered by AWS for managing SSL/TLS certificates effortlessly across AWS services. Whether you’re securing a global content delivery network, API endpoints, load-balanced applications, or any other services, ACM has got you covered

Happy Learning!!

3 weeks Journey with AWS DevOps Engineer Certification: My Experience and Tips

Rohit — Sat, 11 May 2024 04:29:31 +0000

Greetings, my fellow tech enthusiasts! I am thrilled to share with you my journey towards achieving the AWS DevOps Engineer certification. With a passion for technology and a drive to continuously enhance my skills, pursuing this certification was a natural progression in my career journey. Today in this article, I’ll take you through the steps I took to prepare for and pass the AWS DevOps Engineer exam, offering insights, tips, and strategies that I hope will inspire and assist you on your own certification path. So, grab your coffee, buckle up, and let’s dive into the world of AWS DevOps certification together

Study Strategies:

Preparing for the AWS DevOps Engineer certification exam requires a well-structured approach to learning. Here are some key study strategies that proved instrumental in my preparation:

1) Domains — AWS DevOps Engineer professional exam has been divided into various domains and each domain is really important which required detailed knowledge.
Domain 1: SDLC Automation (22% of scored content)
Domain 2: Configuration Management and IaC (17% of scored content)
Domain 3: Resilient Cloud Solutions (15% of scored content)
Domain 4: Monitoring and Logging (15% of scored content)
Domain 5: Incident and Event Response (14% of scored content)
Domain 6: Security and Compliance (17% of scored content)

2) Online Study — Go through the online course from Stephane Maarek on Udemy, which helped me to revise my knowledge about DevOps concept. I took 2 weeks for this. Here is the link for the course.

[(https://www.udemy.com/course/aws-certified-devops-engineer-professional-hands-on/?couponCode=ST20MT50724)]

3) Mock Exam— Taking practice exams played a pivotal role in my exam preparation. These mock exams not only familiarized me with the exam format and question types but also helped me identify areas where I needed to focus additional study efforts. Go through the online practice test from TutorialDojo as follow:

[(https://www.udemy.com/course/aws-solutions-architect-professional-practice-exams-sap-c02/?couponCode=ST20MT50724)]

4) Time Management — This is a tough one and questions were very lengthy. To simulate real exam conditions, I practiced time-bound mock exams to improve my time management skills. This ensured that I could effectively allocate time to each question during the actual exam and complete it within the allotted timeframe.

5) *Recommendation *— There are many key topics which needs to master for AWS DevOps engineer professional exams and based on my experience, some of them are:-

CI/CD Pipelines (CodeCommit, CodeBuild, CodeDeploy, CodeArtifact)
Elastic Beanstalk
AWS CloudFormation
AWS Config
Cloudwatch Event Rules, CloudWatch Logs, CloudTrails
SSM, Lambda, API Gateway
SDLC, Disaster Recovery, High Availability, Deployment Strategy etc.

6) Stay consistent and Hands-on Practice — Encourage readers to start their preparation early and establish a study routine. Focus on hand-on practice with AWS services and mock exams.

In conclusion, my journey to achieving the AWS DevOps Engineer certification has been both challenging and rewarding. With diligent study, practice, and perseverance, I was able to overcome obstacles and emerge victorious on exam day. As I reflect on this experience, I’m reminded of the importance of setting clear goals, staying disciplined in my study routine, and believing in my abilities.

Best of luck on your certification journey, and may your future endeavors be filled with success and fulfillment.

Happy Learning!!