DEV Community: Sagar R Ravkhande

Flask Application Deployment using AWS ECS and AWS DynamoDB with Terraform

Sagar R Ravkhande — Mon, 31 Mar 2025 21:07:54 +0000

Deploying a Flask application can seem daunting, but using AWS makes the process streamlined and scalable. In this blog post, we’ll walk through deploying a simple "Hello World" Flask application using AWS ECS (Elastic Container Service) and AWS DynamoDB.

This is a simple "Hello World" application that exposes two HTTP-based APIs:

PUT /hello/: Saves or updates the given user's name and date of birth in the database.
GET /hello/: Returns a hello message for the given user, including their birthday message if it's today or in the next N days.

Architecture

Requirements

Docker(local)
Terraform
Python 3.8+
Flask 2.0+
SQLite3(local)
AWS account for deployment
AWS DynamoDB
Deployment Architecture
Cloud Platform: AWS (Amazon Web Services)
Github Action: CI/CD Deployment.
AWS ECS: To run the application as a containers in EC2.
AWS DynamoDB: Amazon DynamoDB is a fully managed proprietary NoSQL database.
AWS S3: For storing terraform state file.
AWS CloudWatch: For logging and monitoring.

1. Contents

./applocal.py
contains a Python Flask application, which is integrated with SQLite3
./app.py
contains a Python Flask application, which is integrated with DynamoDB the same applicaion can be used to run locally except you need to have aws dynamodb database deployed already.
terraform/
contains the terraform code necessary to deploy the application into AWS ECS

2. How to run and test locally

Install dependencies: pip install flask sqlite3 boto3
Run the application: python applocal.py
Run test: python3 -m unittest tests/test_app.py
Test the APIs using curl or a tool like Postman

3. How-to Deploy into AWS ECS using github action CI/CD and Terraform for IAC

General workflow:

For the very 1st time, Run the sh setup.sh command which will create ECR repository and build, tag and push the image into it.
As soon as the changes are pushed into git repository branches like main and develop it will trigger the GHA CI/CD to build and deploy into ECS using Terraform
Terraform will output the alb url at the end of the GHA run.

4. File Descriptions

1. provider.tf
This file defines the AWS provider and region where the resources will be created, as well as the backend "s3" configures a remote backend using Amazon S3.

provider "aws" {
  region = "us-east-1"
}
terraform {
  backend "s3" {
    bucket  = "<Bucket-Name>"
    key     = "terraform.tfstate"
    region  = "us-east-1"
    encrypt = true
  }
}

2. variables.tf
This file contains variable definitions used throughout your Terraform configuration. For example:

variable "ecs_cluster_name" {  
  description = "Name of the ECS Cluster"  
  default     = "hello-world-cluster"  
}  

variable "dynamodb_table_name" {  
  description = "Name of the DynamoDB table"  
  default     = "Messages"  
}  

variable "image_repository" {  
  description = "ECR repository for the Docker image"  
  default     = "hello-world-flask"  
}

3. main.tf
This file includes the configurations for the ECS cluster, service, task definitions and dynamo DB configurations.

resource "aws_ecs_cluster" "hello_world_cluster" {  
  name = var.ecs_cluster_name  
}  

resource "aws_ecs_task_definition" "hello_world_task" {  
  family                = "hello-world-task"  
  network_mode         = "awsvpc"  
  requires_compatibilities = ["FARGATE"]  

  container_definitions = jsonencode([{  
    name      = "hello-world-container"  
    image     = "${aws_ecr_repository.hello_world_repo.repository_url}:latest"  
    cpu       = 256  
    memory    = 512  
    essential = true  

    portMappings = [{  
      containerPort = 5000  
      hostPort      = 5000  
      protocol      = "tcp"  
    }]  
  }])  
}  

resource "aws_ecs_service" "hello_world_service" {  
  name            = "hello-world-service"  
  cluster         = aws_ecs_cluster.hello_world_cluster.id  
  task_definition = aws_ecs_task_definition.hello_world_task.arn  
  desired_count   = 1  
  launch_type     = "FARGATE"  

  network_configuration {  
    subnets          = var.subnet_ids  
    security_groups  = [aws_security_group.ecs_sg.id]  
    assign_public_ip = true  
  }  
}

4. outputs.tf
Outputs provide useful information after the Terraform apply, like resource ARNs or URLs.

output "alb_dns_name" {
  description = "The Application Load Balancer DNS name"
  value       = aws_lb.main.*.dns_name[0]
}

5. Deployment Steps

Initialize Terraform: Before applying the configuration, make sure to initialize the Terraform environment.

sh setup.sh # This command which will create ECR repository and build, tag and push the image into it.
cd terraform 
terraform init

Plan the Deployment: Generate and show an execution plan.

terraform plan

Apply the Configuration: Deploy the infrastructure by applying the plan.

terraform apply

Verify Deployment: After the deployment completes, verify that all resources were created successfully.

6. Response Examples

Conclusion

By using Terraform for the infrastructure as code, deploying the AWS resources needed for your Flask application becomes efficient and repeatable. Make sure to replace any default values in the variables to suit your requirements. This infrastructure can be modified or extended to accommodate more complex applications in the future.

For any updates or further assistance, feel free to check the original repository or reach out for clarifications.

Create Azure backup of VM using Automation script in AzCli

Sagar R Ravkhande — Mon, 31 Mar 2025 20:25:30 +0000

echo "Logging In...."
az login --service-principal -u $Clientid -p $Clientsecret --tenant $Tenantid

az account set -s "ba-ib-at23055-neu-dev"

RecoveryServicesVault="at23055vault-test"
resourceGroup="AT23055_DRMDASHBOARD_DEV"
vmName="xd934b23055dev2"

az backup vault show --name $RecoveryServicesVault --resource-group $resourceGroup

retVal=$?
if [ $retVal -eq 0 ]; then
        echo "Vault Exists Already"
else
        echo "Vault doesn't Exist!"
        echo "Creating a new Vault.."
        az backup vault create --resource-group $resourceGroup \
        --name $RecoveryServicesVault \
        --location northeurope
fi

az backup vault backup-properties set \
    --name $RecoveryServicesVault  \
    --resource-group $resourceGroup \
    --backup-storage-redundancy "LocallyRedundant"
#GeoRedundant


az backup protection check-vm --resource-group $resourceGroup --vm $vmName

if [ $retVal -eq 0 ]; then
        echo "Virtual machine is protected Already"
else
        echo "Virtual machine is not protected"
        echo "Creating Virtual machine Protection..."
        az backup protection enable-for-vm \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --vm $vmName \
        --policy-name DefaultPolicy
fi

retention=`date +'%d-%m-%Y' -d "+1 year"`

count=$(az backup job list --resource-group $resourceGroup --vault-name $RecoveryServicesVault --output table | grep -i 'InProgress' | wc -l)

if [ $count -gt 0 ]; then
        echo "Backup is InProgress, Unable to initiate backup as another backup operation is currently in progress."
        echo "Checking the status of backup jobs..."
        az backup job list \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --output table
        exit 0
else
        echo "Initiating backup job.."
        az backup protection backup-now \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --container-name $vmName \
        --item-name $vmName \
        --backup-management-type AzureIaaSVM \
        --retain-until $retention
fi


echo "Checking the status of backup jobs..."
az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table

Please find below the concise explanation of the provided code snippet that is intended to manage Azure backup vaults and virtual machines using the Azure CLI:

Explanation of the Code

echo "Logging In...."  
az login --service-principal -u $Clientid -p $Clientsecret --tenant $Tenantid

This logs into Azure using a service principal (a special Azure account typically used for automated scripts). It uses the client ID, client secret, and tenant ID provided in the variables.

Set the Azure Account:

az account set -s "<Subscription_ID>"

Sets the specific Azure subscription to use for the subsequent commands.

Define Variables:

RecoveryServicesVault="vault-test"  
resourceGroup="RG_DEV"  
vmName="xdvmdev2"

These lines define variables for the recovery services vault name, resource group, and virtual machine name.

Check if the Backup Vault Exists:

az backup vault show --name $RecoveryServicesVault --resource-group $resourceGroup  
retVal=$?  
if [ $retVal -eq 0 ]; then  
    echo "Vault Exists Already"  
else  
    echo "Vault doesn't Exist!"  
    echo "Creating a new Vault.."  
    az backup vault create --resource-group $resourceGroup \
    --name $RecoveryServicesVault \
    --location northeurope  
fi

It checks if the specified backup vault exists.
If it does, it confirms its existence; if not, it creates a new backup vault in the specified location.

Set Backup Properties:

az backup vault backup-properties set \
    --name $RecoveryServicesVault  \
    --resource-group $resourceGroup \
    --backup-storage-redundancy "LocallyRedundant"

This sets the backup storage redundancy to "LocallyRedundant," meaning that the backups will be stored in a way that protects them from local failures.

Check VM Backup Protection Status:

az backup protection check-vm --resource-group $resourceGroup --vm $vmName

This checks if the specified virtual machine is already protected by the backup vault.

Enable VM Protection if Not Already Protected:

if [ $retVal -eq 0 ]; then  
    echo "Virtual machine is protected Already"  
else  
    echo "Virtual machine is not protected"  
    echo "Creating Virtual machine Protection..."  
    az backup protection enable-for-vm \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --vm $vmName \
    --policy-name DefaultPolicy  
fi

If the VM is not protected, it enables backup protection for the VM with the default policy.

Initiate Backup Job:

retention=`date +'%d-%m-%Y' -d "+1 year"`  
count=$(az backup job list --resource-group $resourceGroup --vault-name $RecoveryServicesVault --output table | grep -i 'InProgress' | wc -l)  

if [ $count -gt 0 ]; then  
    echo "Backup is InProgress, Unable to initiate backup as another backup operation is currently in progress."  
    echo "Checking the status of backup jobs..."  
    az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table  
    exit 0  
else  
    echo "Initiating backup job.."  
    az backup protection backup-now \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --container-name $vmName \
    --item-name $vmName \
    --backup-management-type AzureIaaSVM \
    --retain-until $retention  
fi

It defines a retention policy by calculating a date one year from now.
It checks if another backup job is in progress. If so, it displays the active jobs and exits.
If no jobs are in progress, it initiates a new backup job for the VM.

Check Backup Job Status:

echo "Checking the status of backup jobs..."  
az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table

Finally, it retrieves and displays the status of all backup jobs associated with the vault.

Summary

This script automates the process of logging into Azure, checking or creating a backup vault, managing the backup protection of a specified VM, and initiating a backup job, while also checking for existing operations to avoid conflicts.

SonarQube Infrastructure Setup using AWS EC2 and PostgreSQL

Sagar R Ravkhande — Sun, 31 Mar 2024 22:41:30 +0000

Providing a project explanation or documentation for setting up SonarQube infrastructure on EC2, using PostgreSQL as a database, and integrating it with Jenkins to run sonar-scanner on multiple jobs as part of a Groovy pipeline is essential for clarity and future reference. Here's a sample project explanation:

Project: SonarQube Infrastructure Setup

Overview

This project aims to set up a SonarQube environment on Amazon EC2, utilizing PostgreSQL as the database to store code analysis results. Additionally, it integrates SonarQube with Jenkins to automate code quality analysis using the SonarScanner tool in multiple Jenkins jobs.

Project Overview
Infrastructure Setup
- EC2 Instance Setup
- PostgreSQL Database Setup
- SonarQube Installation
Jenkins Integration
- Jenkins Installation
- Jenkins Configuration
- Jenkins SonarQube Plugin
Pipeline Setup
- Pipeline Definition (Jenkinsfile)
- Triggering Code Analysis
Conclusion

Infrastructure Setup

EC2 Instance Setup

Launch an Amazon EC2 instance using the desired Amazon Machine Image (AMI) with necessary security group settings. Ensure that the instance has adequate resources (CPU, RAM, etc.) for SonarQube.
SSH into the EC2 instance and configure it according to the SonarQube installation requirements.

PostgreSQL Database Setup

Set up a PostgreSQL database on a separate EC2 instance or on a managed PostgreSQL service like Amazon RDS.
Create a dedicated database and user for SonarQube with appropriate permissions.
Update the SonarQube configuration to point to the PostgreSQL database.

SonarQube Installation

Download and install SonarQube on the EC2 instance.
Configure SonarQube settings, including database connection details, authentication, and security settings.
Start the SonarQube service and ensure it's running.

Jenkins Integration

Jenkins Installation

Install Jenkins on a separate EC2 instance or use an existing Jenkins installation.
Configure Jenkins to run as a service and access it via its web interface.

Jenkins Configuration

Install and configure necessary plugins for Jenkins, including Git, Pipeline, and others as needed.
Set up authentication and authorization settings for Jenkins.

Jenkins SonarQube Plugin

Install the SonarQube Scanner for Jenkins plugin in Jenkins.
Configure the SonarQube server URL and authentication token within Jenkins.

Pipeline Setup

Pipeline Definition (Jenkinsfile)

Define a Jenkins pipeline using a Jenkinsfile. This pipeline defines how code analysis using SonarScanner should be executed.
The Jenkinsfile should specify the Git repository, branch, and the build steps including sonar-scanner execution.

Triggering Code Analysis

Create multiple Jenkins jobs or pipeline stages for different projects or branches.
Configure these jobs to execute the Jenkins pipeline, which in turn triggers the sonar-scanner for code analysis.
Set up triggers or schedules to periodically run code analysis jobs or integrate them into your CI/CD workflow.

Conclusion

This project provides a comprehensive setup for SonarQube infrastructure on EC2, integrates it with PostgreSQL as a database, and uses Jenkins to automate code quality analysis. By following the steps outlined in this documentation, you can maintain code quality and ensure continuous improvement in your software development process.

This project explanation should serve as a high-level guide for setting up and integrating SonarQube with Jenkins for code analysis in your organization. You can expand on each section with specific details, commands, and configuration options based on your environment and requirements. Additionally, consider providing links to relevant documentation and resources for further reference.

Create MongoDB Atlas Cluster With Terraform and AWS

Sagar R Ravkhande — Sun, 31 Mar 2024 22:39:27 +0000

This project aims to set up an Atlas MongoDB cluster with an AWS network peering to access the resources from AWS EC2, utilizing Terraform as an Infrastructure as a Code(IAC).

Prerequisites Needed:

Basic understanding of Terraform concepts and Terraform CLI installed.
AWS VPC with subnets and route tables.
MongoDB Atlas account.
MongoDB Atlas Organization and a Project under your account with public and private keys with Organization Project Creator API key.

MongoDB Atlas API key for terraform
MongoDB atlas terraform provider
Module for cluster resource
- Cluster Resources
- Users and Roles
- Network peering with atlas
Plan and apply
Resources

MongoDB Atlas API key for terraform

Once you create an organization in Atlas, access to an organization or project can only be managed using the API key, so we need to create an API key.

API keys have two parts: a Public Key and a Private Key. These two parts serve the same function as a username and a personal API key when you make API requests to Atlas.

You must grant roles to API keys as you would for users to ensure the API keys can call API endpoints without errors. Here we will need an API key that will have Organization Project Creator permissions.
e.g.
All API keys belong to the organization. You can give an API key access to a project. Each API key belongs to only one organization, but you can grant an API key access to any number of projects in that organization.

Configuring the MongoDB atlas provider for Terraform

The Terraform MongoDB Atlas Provider is a plugin that allows you to manage MongoDB Atlas resources using Terraform.
Syntax:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.4"
    }
    mongodbatlas = {
      source  = "mongodb/mongodbatlas"
      version = "~> 1.9"
    }
  }
}

provider "mongodbatlas" {
  public_key = "<YOUR PUBLIC KEY HERE>"
  private_key  = "<YOUR PRIVATE KEY HERE>"
}

In our case, all the required variables like Organization ID, Public key, and Private key are created in the AWS Systems Manager Parameter Store and are being referred to respectively.

data "aws_ssm_parameter" "private_key" {
  name            = "/atlas/private-key"
  with_decryption = true
}

data "aws_ssm_parameter" "public_key" {
  name            = "/atlas/public-key"
  with_decryption = true
}

data "aws_ssm_parameter" "atlas_organization_id" {
  name            = "/atlas/org-id"
  with_decryption = true
}

Module for cluster resources

Cluster resources

mongodbatlas_project provides a Project resource. This allows the project to be created.
mongodbatlas_network_container provides a Network Peering Container resource. The resource lets you create, edit, and delete network peering containers. You must delete network peering containers before creating clusters in your project. You can't delete a network peering container if your project contains clusters. The resource requires your Project ID.
mongodbatlas_project_ip_access_list provides an IP Access List entry resource that grants access from IPs, CIDRs, or AWS Security Groups (if VPC Peering is enabled) to clusters within the Project.

resource "mongodbatlas_project" "this" {
  name   = var.atlas_project_name
  org_id = var.atlas_organization_id
}

resource "mongodbatlas_network_container" "this" {
  atlas_cidr_block = var.atlas_cidr_block
  project_id       = mongodbatlas_project.this.id
  provider_name    = "AWS"
  region_name      = local.region_name
}

resource "mongodbatlas_project_ip_access_list" "this" {
  project_id = mongodbatlas_network_peering.this.project_id
  cidr_block = var.vpc_cidr_block
  comment    = "Grant AWS ${var.vpc_cidr_block} environment access to Atlas resources"
}

resource "aws_route" "this" {
  for_each                  = toset(var.private_route_table_ids)
  route_table_id            = each.value
  destination_cidr_block    = mongodbatlas_network_container.this.atlas_cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection_accepter.this.vpc_peering_connection_id
}

Users and Roles

mongodbatlas_custom_db_role allows you to create custom roles in Atlas when the built-in roles don't include your desired set of privileges. Atlas applies each database user's custom roles together with:

Any built-in roles you assign when you add a database user or modify a database user.
Any specific privileges you assign when you add a database user or modify a database user. You can assign multiple custom roles to each database user. E.g.

resource "mongodbatlas_custom_db_role" "roles" {
  for_each   = var.custom_roles
  project_id = module.atlas_project.project_id
  role_name  = each.value.role_name

  dynamic "actions" {
    for_each = each.value.actions
    content {
      action = actions.value.action
      resources {
        collection_name = actions.value.resources.collection_name
        database_name   = actions.value.resources.database_name
      }
    }
  }
}

mongodbatlas_database_user Creates database users to provide clients access to the database deployments in your project. A database user's access is determined by the roles assigned to the user.

resource "mongodbatlas_database_user" "database_users" {
  for_each           = var.database_users
  username           = each.value.username
  password           = jsondecode(data.aws_secretsmanager_secret_version.creds.secret_string)[each.value.username]
  auth_database_name = "admin"
  project_id         = module.atlas_project.project_id

  dynamic "roles" {
    for_each = each.value.roles
    content {
      database_name = roles.value.database_name
      role_name     = roles.value.role_name
    }
  }
  depends_on = [mongodbatlas_custom_db_role.roles]
}

You can create and upload the username and password details as a key pair values in AWS secret manager using,

aws secretsmanager create-secret --name atlas-users --description "Mytest with multiples values" --secret-string file://secretmanagervalues.json

Where,

Which will have below values e.g.
{"Juan":"mykey1","Pedro":"mykey2","Pipe":"mykey3"}

Network peering with atlas

mongodbatlas_network_peering Network peering establishes a private connection between your Atlas VPC and your cloud provider's VPC. The connection isolates traffic from public networks for added security.

resource "mongodbatlas_network_peering" "this" {
  container_id           = mongodbatlas_network_container.this.id
  project_id             = mongodbatlas_project.this.id
  provider_name          = "AWS"
  accepter_region_name   = data.aws_region.current.id
  vpc_id                 = var.vpc_id
  aws_account_id         = data.aws_caller_identity.current.account_id
  route_table_cidr_block = var.vpc_cidr_block
}

resource "aws_vpc_peering_connection_accepter" "this" {
  vpc_peering_connection_id = mongodbatlas_network_peering.this.connection_id
  auto_accept               = true
  tags = {
    Name = var.peering_connection_name
  }
}

E.g.

Plan and Apply your terraform code

Your tfvars file should look like this,

environment.tfvars

vpc_id = "<VPC-ID>"
vpc_cidr_block = "10.0.0.0/16"
peering_connection_name = "tf-mongo-atlas"
atlas_project_name = "<Cluster-Name>"

atlas_cidr_block = "<Atlas-CIDR>"

database_users = {
  user1 = {
    username           = "Juan",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
    ]
  },
  user2 = {
    username           = "Pedro",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
      { database_name = "admin", role_name = "oplogRead" },
    ]
  },
  user3 = {
    username           = "Pipe",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
    ]
  },
  // Add more users as needed
}

custom_roles = {
  oplogRead = {
    role_name = "oplogRead"
    actions = [
      {
        action = "FIND"
        resources = {
          collection_name = ""
          database_name   = "anyDatabase"
        }
      },
      {
        action = "CHANGE_STREAM"
        resources = {
          collection_name = ""
          database_name   = "anyDatabase"
        }
      },
    ]
  }
  # Add more roles if needed
}

Now all you need to do is to plan your terraform code using the terraform plan and verify the changes in the printed plan. You should be seeing one module to be added which will be the atlas cluster with the provided configuration.

After verifying the plan in the above step, run terraform apply to apply your changes. You will see the cluster getting created message on the terminal and your changes will start appearing in your mongoDB console as well. It usually takes about 10 to 15 mins for the cluster to be created.

You will see cluster created like this,

Resources

You can find the project demo code here,
https://github.com/sagary2j/atlas-mongodb-terraform/tree/initial-code
MongoDB atlas: https://registry.terraform.io/providers/mongodb/mongodbatlas/latest
Atlas organizations apiKeys create: https://www.mongodb.com/docs/atlas/cli/stable/command/atlas-organizations-apiKeys-create/#inherited-options

AWS IAM best practices to secure your resources...

Sagar R Ravkhande — Mon, 13 Mar 2023 10:47:38 +0000

AWS IAM (Identity and Access Management) is a web service that enables you to securely control access to AWS resources. IAM enables you to create and manage AWS users and groups and assign policies to control the level of access that each user or group has to specific AWS resources.

Key Components of AWS IAM

User Management: IAM enables you to create and manage individual users within your AWS account. This enables you to control who has access to your AWS resources.

Group Management: IAM enables you to create and manage groups of users. This enables you to assign policies to groups, rather than having to assign policies to individual users.

Access Management: IAM enables you to control access to AWS resources using permissions. Permissions are defined using policies, which specify which actions a user or group can perform on specific AWS resources.

Multi-factor Authentication (MFA): IAM enables you to require MFA for users when they sign into your AWS account. MFA provides an extra layer of security beyond a user's password.

Integration with AWS Services: IAM is integrated with many AWS services, such as Amazon S3, Amazon EC2, and Amazon RDS. This enables you to control access to these services using IAM policies.

Auditing and Logging: IAM provides detailed logging and auditing of all user activity within your AWS account. This enables you to monitor user activity and detect any suspicious behavior.

Overall, AWS IAM provides a robust set of tools for managing access to your AWS resources, helping you maintain security and compliance.

Key features of AWS IAM

• Authentication: AWS IAM lets you create and manage identities such as users, groups, and roles, meaning you can issue and enable authentication for resources, people, services, and apps within your AWS account.

• Authorization: Access management or authorization in IAM is made of two primary components: Policies and Permissions. In the next section, we’ll also look at each of these.

• Fine-grained permissions: Consider this — you want to provide the sales team in your organization access to billing information, but also need to allow the developer team full access to the EC2 service, and the marketing team access to selected S3 buckets. Using IAM, you can configure and tune these permissions as per the needs of your users.

• Shared access to AWS accounts: Most organizations have more than one AWS account, and at times need to delegate access between them. IAM lets you do this without sharing your credentials, and more recently, AWS released Control Tower to further simplify multi-account configurations. We also published a quick, hands-on tutorial on Securing Multi-Account Access on AWS.

• AWS Organizations: For fine-grained control for multiple AWS accounts, you can use AWS Organizations to segment accounts into groups and assign permission boundaries.

• Identity Federation: Many times, your organization will need to federate access from other identity providers such as Okta, G Suite, or Active Directory. IAM enables you to do this with a feature called Identity Federation

*By following these best practices, you can help ensure that your AWS resources are secure and compliant with industry standards and regulations. *

• Avoid the use of root account unless strictly necessary: Do not use the root account for day-to-day administration activities. By default, the root account user has access to all resources for all AWS services and its best practice to create IAM users with least privilege access. Additionally, do not create access keys for the root account unless strictly necessary. Finally, set up monitoring to detect and alert on root account activity, and
ensure hardware-based MFA is set up for root account access.

• Use temporary credentials: Never share your credentials with anyone. It’s advisable to create individual users for anyone who has access requirements and even better use temporary credentials. Dynamically generated credentials that expire after a configurable interval, are a great way to tackle this. You can visit our practical tutorial on Securing Multi-Account Access on AWS for detailed instructions on this.

• Embrace the least privilege principle and review all IAM permissions periodically: It is important to follow the security principle of least privilege, which means that if a user doesn’t need to interact with a resource, then it is best not to provide them access to that resource. IAM permissions allow for very granular access controls, so avoid the use of policy statements that grant access to all actions, all principals, or all resources.
Additionally, use the IAM Access Advisor regularly to ensure that all permissions assigned to a particular user are indeed being used.

• Enforce the least privilege principle to be implemented bi-directionally: Many AWS resources (such as S3 buckets) can have their access policy attached directly. Don’t fall into the trap of thinking that because access is tightly controlled in one direction (i.e., an IAM Role that grants very specific permissions), that you should be less rigid in the other direction (for example, when an S3 bucket access policy grants read access to all entities in your account). Optimally use both sides of the least privilege principle to achieve favorable outcomes.

• Monitor account activity regularly using IAM Access Analyzer and AWS CloudTrail: In addition to what we discussed about the newly released IAM Access Analyzer, the good old AWS CloudTrail is an excellent tool to monitor all activities in your account. You can easily use CloudTrail logs to identify suspicious activity and take necessary actions depending on your findings. You will find our deep-dive, practical tutorial on AWS Security Logging with CloudTrail interesting with step-by-step instructions to help you do this.

•Use Multi-Factor Authentication (MFA): Enable MFA to build an additional layer of security for interaction with the AWS API.

• Enforce strong passwords: Enforce strong passwords by configuring account password policy that involves password rotation, discourages the use of old passwords, only allows alphanumeric characters, and more.

• Monitor and Review Permissions Regularly: Regularly review permissions assigned to users and groups to ensure that they are still needed and appropriate. IAM provides reports and tools to help identify and remediate permissions issues.

• Use IAM Roles for EC2 Instances: Use IAM roles to grant temporary permissions to EC2 instances instead of storing credentials on the instance itself. This reduces the risk of credential exposure in case the instance is compromised.

• Enable AWS CloudTrail: AWS CloudTrail logs all API activity in your AWS account, including IAM actions. This provides a trail of activity that can be used for security analysis, resource change tracking, and compliance auditing.

• Use IAM Policy Conditions: Use IAM policy conditions to further restrict access based on context, such as IP address or time of day. This adds an additional layer of security beyond traditional permissions.

• Enable AWS Security Hub: AWS Security Hub provides a comprehensive view of security alerts and compliance status across your AWS accounts. It can help you identify security issues and provide recommendations for remediation.

Linux Troubleshooting Scenarios - Part 3

Sagar R Ravkhande — Fri, 06 Jan 2023 09:53:35 +0000

Finally!! After Part-2 Here is the final chapter of Linux Troubleshooting Scenarios - Part 3. Below are the scenarios:

Issue 1: Unable to Run Certain Commands

├── Troubleshooting and Resolution
│ ├── command
│ │ ├── Could be the system-related command to which non-root user 
        does not have the access
│ │ ├── Could be the user-defined script/command
│ ├── Troubleshooting
│ │ ├── permission/ownership of the command/script
│ │ ├── sudo permission
│ │ ├── absolute/relative path of command/script
│ │ ├── not defined in user $PATH variable
│ │ ├── command is not installed
│ │ ├── command library is missing or deleted

Issue 2: System Unexpectedly reboots and process restart?

├── Troubleshooting and Resolution
│ ├── System reboot/crash reasons
│ │ ├── CPU stress
│ │ ├── RAM stress
│ │ ├── Kernel fault
│ │ ├── Hardware fault
│ ├── Process restart
│ │ ├── System reboot
│ │ ├── Restart itself
│ │ ├── Watchdog application
│ │ │ ├── To prevent high stress on system resources
│ │ │ ├── If the application causing stress, so it will restart or 
          terminate
│ ├── Troubleshooting
│ │ ├── After logging in, check the status by using commands like 
        uptime, top, dmesg, journalctl, iostat -xz 1
│ │ ├── syslog.log, boot.log, dmesg, messages.log, etc
│ │ ├── custom log path of an application
│ │ ├── if not completely accessible, so take the virtual console 
        like ILO, IDRAC, etc.
│ │ ├── open a case and reach out to a vendor

Issue 3: Unable to get IP Address

├── IP Assignment Methods
│ ├── DHCP
│ │ ├── Fixed Allocation
│ │ ├── Dynamic Allocation
│ ├── Static
├── Troubleshooting
│ ├── check network settings from virtualization environments like 
      VMware, VirtualBox or etc.
│ ├── check whether the IP address is assigned or not
│ ├── check the NIC status from the host side using #lspci, #nmcli 
      etc.
│ ├── restart network service

Issue 4: Backup and Restore File Permissions in Linux

├── Troubleshooting
│ ├── The best option is to create the ACL file of Dir/Files 
      before changing the permissions in bulk
│ │ ├── Create the ACL file before changing the permission (or 
        backup the file permission): ~$ getfacl -R <dir> > 
        permissions.acl
│ │ ├── Restore File Permissions: ~$ setfacl -- 
        restore=permissions.acl
│ ├── Restore from the VM Snapshot (But not always a good option 
      for production)
│ ├── Rebuild the VM (this option is safe for the future)

Useful Tip-Related Disk Partition:

├── Tips
│ ├── After adding/attaching a new disk to a VM, we can get its 
      status from lsblk command by doing ~$echo 1 > 
      /sys/block/sda/device/rescan
│ ├── If we increase the disk size of the existing disk then the 
      additional space gets appended to the existing disk without 
      affecting the already existing Filesystem and Partition.
│ ├── We can also recreate the filesystem on the block device as 
      it will automatically format the old one
│ ├── If we have a disk (with created partition/FS) we can share 
      the .vmdk to another VM. So, after mounting we would have 
      the same data as it was on the previous one.

Linux Troubleshooting Scenarios - Part 2

Sagar R Ravkhande — Fri, 06 Jan 2023 09:53:22 +0000

Continuation from Part 1 of our Linux Troubleshooting Scenarios here is the Part-2 of the next scenarios to be discussed.

Issue 1: fstab file missing or bad entry

├── One of the errors that cause the system unable to BOOT UP
├── Check /var/log/messages, dmesg, and other log files
├── If we have a bad sector log, we have to run fsck
│ ├── True:
│ │ ├── reboot the system into resuce mode as booting it from 
        CDROM by applying ISO
│ │ ├── proceed with option 1, which mounts the original root 
        filesystem under/mnt/sysimage
│ │ ├── edit fstab entries or create a new file with the help of 
        blkid and reboot

Issue 2: Can’t cd to the directory even if the user has sudo privileges

├── Reasons and Resolution
│ ├── Directory does not exist
│ ├── Pathname conflict: relative vs absolute path
│ ├── Parent directory permission/ownership
│ ├── Doesn't have executable permission on the target directory
│ ├── Hidden directory

Issue 3: Can’t Create Links

├── Reasons and Resolution
│ ├── Target directory/File does not exist
│ ├── Pathname conflict: relative vs absolute path - (should be 
      complete path)
│ ├── Parent directory permission/ownership
│ ├── Target file permission/ownership - (as there should be read 
      permission)
│ ├── Hidden directory/file

Issue 4: Running Out of Memory

├── Types
│ ├── Cache (L1, L2, L3)
│ ├── RAM
│ │ ├── Usage
│ │ │ ├── #free -h
│ │ │ │ ├── Total (Total assigned memory)
│ │ │ │ ├── Used (Total actual used memory)
│ │ │ │ ├── Free (Actual free memory)
│ │ │ │ ├── Shared (Shared Memory)
│ │ │ │ ├── Buff/Cache (Pages cache memory)
│ │ │ │ ├── Available (Memory can be freed)
│ │ │ ├── /proc/meminfo
│ │ │ │ ├── file active
│ │ │ │ ├── file inactive
│ │ │ │ ├── anon active
│ │ │ │ ├── anon inactive
│ ├── Swap (Virtual Memory)
├── Resolution
│ ├── Identify the processes that are using high memory using top, 
      htop, ps etc.
│ ├── Check the OOM in logs and also check if there is a memory 
      commitment in sysctl.conf
│ ├── Kill or restart the process/service
│ ├── prioritize the process using nice
│ ├── Add/Extend the swap space
│ ├── Add more physical RAM

Issue 5: Add/ Extend the Swap Space

├── Due to running out of memory, we would need to add more swap 
    space
│ ├── Create a file with #dd, as it will reserve the blocks of 
      disk for swap file
│ ├── Set permission 600 and give root ownership
│ ├── #mkswap
│ ├── Now Turned swap on #swapon
│ ├── fstab entry for persistent

Linux Troubleshooting Scenarios - Part 1

Sagar R Ravkhande — Wed, 04 Jan 2023 08:48:08 +0000

It is always crucial to understand the issue. There should be the right approach or a step-by-step process to be followed to troubleshoot the issues. Doesn’t matter if you are a Software Developer or DevOps Engineer or Architect. Unix. /Linux is used widely, and you should be aware of the issues and the correct approach to resolve them.

Let’s discuss a few of them:

Issue 1: Server is not reachable or unable to connect

├── Ping the server by Hostname and IP Address
│ ├── Hostname/IP Address is pingable
│ │ ├── The Issue might be on the client side as the server is 
        reachable
│ ├── Hostname is not pingable but IP Address is pingable
│ │ ├── Could be the DNS issue
│ │ │ ├── check /etc/hosts
│ │ │ ├── check /etc/resolv.conf
│ │ │ ├── check /etc/nsswitch.conf
│ │ │ ├── (Optional) DNS can also be defined in the 
          /etc/sysconfig/network-scripts/ifcfg-<interface>
│ ├── Hostname/IP Address both are not pingable
│ │ ├── Check the other server on the same network to see if there 
        is it a Network side access issue or other overall 
        something bad
│ │ │ ├── False: The issue is not overall network side but with 
          that host/server
│ │ │ ├── True: Might be an overall network-side issue
│ │ ├── Logged into the server by Virtual Console, if the server 
        is Powered ON. Check the uptime
│ │ ├── Check if the server has the IP, and has UP status of the 
        Network interface
│ │ │ ├── (Optional) Also check IP-related information from
          /etc/sysconfig/network-scripts/ifcfg-<interface>
│ │ ├── Ping the gateway, also check routes
│ │ ├── Check Selinux, Firewall rules
│ │ ├── Check physical cable conn

Issue 2: Unable to connect to a website or an application

├── Ping the server by Hostname and IP Address
│ ├── False: Above Troubleshooting Diagram "Server is not 
      reachable or cannot connect"
│ ├── True: Check the service availability by using the telnet 
      command with port
│ │ ├── True: Service is running
│ │ ├── False: Service is not reachable or running
│ │ │ ├── Check the service status using systemctl or other 
          commands
│ │ │ ├── Check the firewall/selinux
│ │ │ ├── Check the service logs
│ │ │ ├── Check the service configuration

Issue 3: Unable to ssh as root or any other user.

├── Ping the server by Hostname and IP Address
│ ├── False: Above Troubleshooting Diagram "Server is not 
      reachable or cannot connect"
│ ├── True: Check the service availability by using the telnet 
      command with port
│ │ ├── True: Service is running
│ │ │ ├── Issue might be on the client side
│ │ │ ├── User might be disabled, no-login shell, disabled root 
          login and other configuration
│ │ ├── False: Service is not reachable or running
│ │ │ ├── Check the service status using systemctl or other 
          commands
│ │ │ ├── Check the firewall/selinux
│ │ │ ├── Check the service logs
│ │ │ ├── Check the service configuration

Issue 4: Disk Space is full issue or add/extend disk space

├── System Performance degradation detection
│ ├── Application getting slow/unresponsive
│ ├── Commands are not running (For Example: as / disk space is 
      full)
│ ├── Cannot do logging and other etc.
├── Analyse the issue
│ ├── df command to find the problematic filesystem space issue
├── Action
│ ├── After finding the specific filesystem, use du command in 
      that filesystem to get which files/directories are large
│ ├── Compress/remove big files
│ ├── Move the items to another partition/server
│ ├── Check the health status of the disks using badblocks command 
      (For Example, #badblocks -v /dev/sda)
│ ├── Check which process is IO Bound (using iostat)
│ ├── Create a link to file/dir
├── New disk addition
│ ├── Simple partition
│ │ ├── Add disk to VM
│ │ ├── Check the new disk with df/lsblk command
│ │ ├── fdisk to create the partition. Better to have LVM 
        partition
│ │ ├── Create filesystem and mount it
│ │ ├── fstab entry for persistent
│ ├── LVM Partition
│ │ ├── Add disk to VM
│ │ ├── Check the new disk with df/lsblk command
│ │ ├── fdisk to create LVM partition
│ │ ├── PV, VG, LV
│ │ ├── Create filesystem and mount it
│ │ ├── fstab entry for persistent
│ ├── Extend LVM partition
│ │ ├── Add disk, and create LVM partition
│ │ ├── Add LVM partition (PV) in existing VG
│ │ ├── Extend LV and resize the filesystem

Issue 5: Filesystem corrupted

├── One of the errors that cause the system unable to BOOT UP
├── Check /var/log/messages, dmesg, and other log files
├── If we have bad sector logs, we have to run fsck
│ ├── True:
│ │ ├── reboot the system into rescue mode by booting it from 
        CDROM by applying ISO
│ │ ├── proceed with option 1, which mounts the original root 
        filesystem under /mnt/sysimage.
│ │ ├── edit fstab entries or create a new file with the help of 
        blkid and reboot.

𝐒𝐎𝐂 1 𝐯𝐬. 𝐒𝐎𝐂 2 𝐯𝐬. 𝐒𝐎𝐂 3

Sagar R Ravkhande — Wed, 04 Jan 2023 08:27:07 +0000

SOC stands for Service Organization Control, and the nut of what it’s all about is summarized right there: You’re a service organization (in accountant-speak), and you need to prove that you have certain controls in place for said accountants to deem you SOC-compliant.

SOC compliance is important because most enterprises can't or won't adopt your product without it. Without SOC compliance, you can’t land the enterprise deals that make your startup sustainable.

In this article, we’re going to break down the meaning of SOC 1, SOC 2, and SOC 3, as well as the differences between all three. By the end, you’ll know which is most relevant and which is necessary, and you’ll understand how to embark on the path to compliance.

SOC 1 vs. SOC 2. vs. SOC 3: An Overview‍

TL;DR: SOC compliance demonstrates that your customers can rely on the services you provide. An accountant audits your company and certifies you with a SOC report that you supply to your customers. This report proves your trustworthiness.

However, understanding SOC compliance in greater detail is important for knowing when to get SOC compliance and which type of SOC report to get. So, let’s break it down further.

The Major differences between Soc 1 vs. SOC 2. vs. SOC 3

There are three primary types of SOC reports—the first two are the most used, and the second is of most concern to technology companies.

SOC 1 and SOC 2 are the most common SOC reports, so understanding the difference between them is essential. The difference between SOC 1 and SOC 2 is that SOC 1 focuses on financial reporting, whereas SOC 2 focuses on compliance and operations.

SOC 3 reports are less common. SOC 3 is a variation of SOC 2 and contains the same information as SOC 2, but it’s presented for a general audience rather than an informed one. If a SOC 2 report is for auditors and stakeholders inside the company you’re selling to, SOC 3 is for that company’s customers.

There are a couple of other SOC reports that are rarer and outside the scope of this article:

SOC for Cybersecurity reports on a service organization’s cybersecurity risk management effectiveness.
SOC for Supply Chain reports on the effectiveness of a service organization’s supply chain risk management.

Take a look at SOC 1, SOC 2, and SOC 3 from a higher level. Save these infographic notes to refer to when your memory of this article gets a little hazy.

Object Oriented Programming (OOP) Concepts

Sagar R Ravkhande — Fri, 11 Nov 2022 15:41:59 +0000

The high-level programming languages are broadly categorized into two categories:

Procedure-oriented programming (POP) language.
Object-oriented programming (OOP) language.

Procedure-Oriented Programming Language

In the procedure-oriented approach, the problem is viewed as a sequence of things to be done such as reading, calculation, and printing.
Procedure-oriented programming consists of writing a list of instructions or actions for the computer to follow and organizing this instruction into groups known as functions.

Characteristics of procedure-oriented programming:

Emphasis is on doing things(algorithm)
Large programs are divided into smaller programs known as functions.
Most of the functions share global data
Data move openly around the system from function to function
Function transforms data from one form to another.
Employs top-down approach in program design

The disadvantage of procedure-oriented programming languages is:

Global data access
It does not model real word problems very well
No data hiding

Object Oriented Programing Language

“Object-oriented programming is an approach that provides a way of modularizing programs by creating partitioned memory area for both data and functions that can be used as templates for creating copies of such modules on demand”.

Characteristics of Object-Oriented programming:

Emphasis is on doing rather than procedure.
programs are divided into what are known as objects.
Data structures are designed such that they characterize the objects.
Functions that operate on the data of an object are tied together in the data structure.
Data is hidden and can’t be accessed by external functions.
Objects may communicate with each other through functions.
New data and functions can be easily added.
Follows bottom-up approach in program design.

Procedure Oriented Programming (POP) Vs Object Oriented Programming (OOP)

Procedure Oriented Programming	Object Oriented Programming
Program is divided into small parts called functions.	Program is divided into parts called objects.
Importance is not given to data but to functions as well as the sequence of actions to be done.	Importance is given to the data rather than procedures or functions because it works in the real world.
Top-Down approach.	Bottom-Up approach.
It does not have any access specifier.	OOP has access specifiers named Public, Private, Protected, etc.
Data can move freely from function to function in the system.	Objects can move and communicate with each other through member functions.
Adding new data and functions in POP is not so easy.	OOP provides an easy way to add new data and functions.
Most function uses Global data for sharing that can be accessed freely from function to function in the system.	In OOP, data cannot move easily from function to function, it can be kept public or private so we can control the access of data.
It does not have any proper way of hiding data, so it is less secure.	OOP provides Data Hiding so provides more security.
Overloading is not possible.	In OOP, overloading is possible in the form of Function Overloading and Operator Overloading.
Examples of Procedure Oriented Programming are C, VB, FORTRAN, and Pascal.	Example of Object-Oriented Programming are C++, JAVA, VB.NET, C#.NET.

Object-Oriented Programming Principles

Encapsulation
Data abstraction
Polymorphism
Inheritance
Dynamic binding
Message Passing

Encapsulation

Wrapping data and functions together as a single unit is known as encapsulation. By default, data is not accessible to the outside world, and they are only accessible through the functions which are wrapped in a class. prevention of data direct access by the program is called data hiding or information hiding.

Data abstraction

Abstraction refers to the act of representing essential features without including the background details or explanation. Classes use the concept of abstraction and are defined as a list of attributes such as size, weight, cost, and functions to operate on these attributes. They encapsulate all essential properties of the object that are to be created. The attributes are called data members as they hold data and the functions which operate on these data are called member functions.
Classes use the concept of data abstraction, so they are called abstract data types (ADT).

Polymorphism

Polymorphism comes from the Greek word “poly” and “morphism”. “poly” means many and “morphism” means form i.e., many forms. Polymorphism means the ability to take more than one form. For example, an operation has different behavior in different instances. The behavior depends upon the type of data used in the operation.
Different ways to achieve polymorphism are:
1) Function overloading
2) Operator overloading

Inheritance

Inheritance is the process by which one object can acquire the properties of another.
Inheritance is the most promising concept of OOP, which helps realize the goal of constructing software from reusable parts, rather than hand-coding every system from scratch. Inheritance not only supports reuse across systems but also directly facilitates extensibility within a system. Inheritance coupled with polymorphism and dynamic binding minimizes the amount of existing code to be modified while enhancing a system.

When the class child, inherits the class parent, the class child is referred to as a derived class (sub-class) and the class parent as a base class (superclass). In this case, the class child has two parts: a derived part and an incremental part.
The derived part is inherited from the class parent. The incremental part is the new code written specifically for the class child.

Dynamic binding:

Binding refers to the linking of a procedure call to the code to be executed in response to the call. Dynamic binding (or late binding) means the code associated with a given procedure call is not known until the time of the call at run time.

Message passing:

An object-oriented program consists of a set of objects that communicate with each other. Objects communicate with each other by sending and receiving information.

A message for an object is a request for the execution of a procedure and therefore invokes the function that is called for an object and generates a result.

Key Takeaways:

OOP: Object-Oriented Programming. A programming paradigm or approach is used to analyze and solve problems that are based on the representation of real-world objects in the system.
Class: one of the building blocks of Object-Oriented Programming that acts as a “blueprint” where the data and the actions of the objects are defined.
Instance: a concrete object that is created from the class “blueprint”.
Method: an “action” defined in the class that the instances of the class can perform. It is very similar to a function but closely related to instances such that instances can call methods and methods can act on the individual data of the instances.

What is a NoSQL database?

Sagar R Ravkhande — Fri, 23 Sep 2022 20:35:03 +0000

The term NoSQL is used to describe a set of technologies for data
storage. In this section, I explain what NoSQL is, outline the
major types of NoSQL databases, and compare NoSQL to relational
databases.

Defining NoSQL

NoSQL describes technologies for data storage, but what exactly
does that mean? Is NoSQL an abbreviation for something?
Depending on whom you ask, NoSQL may stand for “not only SQL” or it may not stand for anything at all. Regardless of any 4 Redis disagreement over what NoSQL stands for, everyone agrees that
NoSQL is a robust set of technologies that enable data persistence
with the high performance necessary for today’s Internet-scale
applications.

SQL is an abbreviation for Standard Query Language, a standard
language for manipulating data within a relational database.

Identifying types of NoSQL databases

There are four major types of NoSQL databases — key/value,
column, document, and graph — and each has a particular use case for which it’s most suited.
The following sections go into greater detail on the four types of
NoSQL.

Key/value

With a key/value storage format, data uses keys, which are identifiers that are similar to a primary key in a relational database. The data element itself is then the value that corresponds to the key.

An example of a key/value pair looks like this:

"id": 123456

In this example, "id" is the key while 123456 is the value that
corresponds to that key.

Key features of the key-value store:

Simplicity.
Scalability.
Speed.

Column

With a column-oriented data store, data is arranged by column
rather than by row. The effect of this architectural design is that it makes aggregate queries over large amounts of data much faster to process.

Key features of columnar-oriented database:

Scalability.
Compression.
Very responsive.

Document

Document data storage in NoSQL uses a key as the basis for item retrieval. The key then corresponds to a more complex data structure called a document, which contains the data elements for
a given collection of data.

Key features of documents database:

Flexible schema: Documents in the database has a flexible schema. It means the documents in the database need not be the same schema.
Faster creation and maintenance: the creation of documents is easy and minimal maintenance is required once we create the document.
No foreign keys: There is no dynamic relationship between two documents so documents can be independent of one another. So, there is no requirement for a foreign key in a document database.
Open formats: To build a document we use XML, JSON, and others.

Graph

Graph databases use graph theory to store data relations in a series of vertices with edges, making queries that work with data
in such a manner much faster.

Key features of graph database:

In a graph-based database, it is easy to identify the relationship between the data by using the links.
The Query’s output is real-time results.
The speed depends upon the number of relationships among the database elements.

Comparison between NoSQL and relational databases

Regardless of the type of NoSQL database, the patterns, and tools that you use to work with data are different from the patterns and tools that you typically find with a relational database. As you just saw, the paradigm for storage and arrangement of the data typically requires a rethink of how applications are created.
Relational databases connect data elements through relations between tables. These relations become quite complex for many applications, and the resulting queries against the data become equally complex. The inherent complexity leads to performance issues for queries.

Many traditional databases include query tools and software to directly manipulate data. With NoSQL, most access will be programmatic only, through applications that you write using the tools and application programming interfaces (APIs) for the NoSQL database.

Relational databases have somewhat less flexibility than a multi-model databases such as Redis. Whereas a relational database thrives when data is consistent and well structured, Redis and NoSQL thrives on the unstructured data that is found in today’s modern applications while also providing the flexibility to structure data as needed.

Why use NoSQL Databases?

Different NoSQL Database models

6(R’) Strategies for Cloud Migration - All IN ONE

Sagar R Ravkhande — Sat, 23 Jul 2022 15:56:00 +0000

What is a Cloud Computing?

"NIST" take on cloud computing,

"The Gartner" take on cloud computing,

"Forrester" take on cloud computing,

and In general,

What are the Key Business Drivers for Cloud Adoption?

Cloud migration describes the process of moving the organization’s business applications from the on-premises data center to virtual cloud infrastructure or cloud services.

Many business factors are driving organizations to adopt a cloud migration path to realize the maximum benefits of the cloud.

Embracing Cloud migration services provides you with the following benefits:

Cloud computing allows businesses to shift their focus from infrastructure and platforms and focus on innovation at the application level.
Cloud deployments have unmatched availability, scalability, and agility of cloud resources as compared to on-premises deployments. One can achieve improved disaster recovery and business continuity by relying on an always-on infrastructure provided by the cloud service provider.
On-demand usage patterns and pay-as-you-go cost management offered by the cloud help organizations convert huge CAPEX spends to smaller chunks of OPEX.
Cloud computing is an attractive alternative to replace on-premises hardware/software components that have reached end-of-life.
Cloud computing also allows businesses to leverage services that may not be available on-premises, leading to an optimal best-of-breed hybrid architecture.
A side benefit of the move to cloud computing is that businesses can reduce environmental waste by reducing the hardware and physical products they use.

What are the Advantages of cloud computing?

Advantage #1: Trade capital expense for variable expense
Capital expenses (CAPEX) are funds that a company uses to acquire, upgrade, and maintain physical assets such as property, industrial buildings, or equipment. Do you remember the data center example in the traditional computing model where you needed to rack and stack the hardware, and then manage it all? You must pay for everything in the data center whether you use it or not. By contrast, a variable expense is an expense that the person who bears the cost can easily alter or avoid. Instead of investing heavily in data centers and servers before you know how you will use them, you can pay only when you consume resources and pay only for the amount you consume. Thus, you save money on technology. It also enables you to adapt to new applications with as much space as you need in minutes, instead of weeks or days. Maintenance is reduced, so you can spend focus more on the core goals of your business.

Advantage #2: Benefit from massive economies of scale
By using cloud computing, you can achieve a lower variable cost than you can get on your own. Because usage from hundreds of thousands of customers is aggregated in the cloud, providers such as AWS can achieve higher economies of scale, which translates into lower pay-as-you-go prices.

Advantage #3: Stop guessing capacity
Eliminate guessing about your infrastructure capacity needs. When you make a capacity decision before you deploy an application, you often either have expensive idle resources or deal with limited capacity. With cloud computing, these problems go away. You can access as much or as little as you need, and scale up and down as required with only a few minutes’ notice.

Advantage #4: Increase speed and agility
In a cloud computing environment, new IT resources are only a click away, which means that you reduce the time it takes to make those resources available to your developers from weeks to just minutes. The result is a dramatic increase in agility for the organization because the cost and time that it takes to experiment and develop are significantly lower.

Advantage #5: Stop spending money on running and maintaining data centers
Focus on projects that differentiate your business instead of focusing on the infrastructure. Cloud computing enables you to focus on your customers instead of the heavy lifting of racking, stacking, and powering servers.

Advantage #6: Go global in minutes
You can deploy your application in multiple AWS Regions around the world with just a few clicks. As a result, you can provide lower latency and a better experience for your customers simply and at a minimal cost.

A migration might consist of moving a single data center, a collection of data centers, or some other portfolio of systems that is larger than a single application.

What is Migration? Why Migrate to Cloud?

Cloud migration isn’t just about moving to the cloud, It is the process of transferring data, applications, and workloads, from On-Premises to the Cloud(AWS, Azure, GCP, etc.) It is an iterative process of optimization to reduce costs and reach the full potential of the cloud. It impacts all the organizational aspects including people, processes, and technology. But with flexible consumption and pricing models, the cloud can support high scalability, performance, agility, remote work, and cost-efficiency.
Cloud Migration is the process of transferring data, applications, and workloads, from On-Premises to the Cloud(AWS, Azure, GCP, etc.).

The decision to migrate to the cloud can be driven by several factors, including data center lease expiration, required hardware upgrades, software license renewals, location requirements to meet regulatory compliance, global market expansion, increased developer productivity, or the need for a standard architecture.
While there are several common components found in each successful migration, there is no one-size-fits-all solution to deciding on the best approach.

Why?

Operational Cost: Operational costs are the cost of running your infrastructure.
Workforce Productivity: Workforce productivity is how efficiently you can get your services to market.
Cost Avoidance: Cost Avoidance is setting up an environment that does not create unnecessary costs.
Operational Resilience: Operational resilience is reducing your organization's risk profile and the cost of risk migration.
Business agility: Ability to react quickly to changing market conditions. you can expand into new markets, and take products to market quickly.

The 6R’s of Strategies for Migration to the Cloud

There are six common cloud migration paths (also known as the six Rs) for cloud migration based on the level of cloud integration that an organization desires.

1) Refactor/Re-architect("Architected using Cloud-Native Feature.")

Definition of Refactor

Writing a new version of the existing application, with a new architecture and design in mind. In a refactor, you gain by removing any unnecessary components, leveraging newer application technologies in the cloud, and generally providing an improved user experience and performance.

This strategy involves converting a legacy monolithic application into a new, highly decoupled, and cloud-native architecture.
This is usually driven by a strong desire to improve a service or application. For highly critical applications that require cloud-native characteristics or applications that need thorough modernization due to outdatedness or performance issues a higher migration effort is typically profitable and hence should be part of cloud considerations.
It is the strategy that usually leads to the highest transformation cost and is much more complicated than other cloud migration approaches because it requires application code changes and must be tested carefully to avoid regressions in functionality. However, it allows optimized use of the cloud, leading to cloud-native benefits and making the application future-proof.
Typically this involves breaking down the application’s components into smaller building blocks, and microservices and wrapping them into (Docker) containers for deployment on a container platform.

Use Cases:

Use Refactor if,

The application will gain the most from the cloud.
There is a strong business drive to add scalability, speed, and performance.
An on-premise app is not compatible with the cloud.

2) Replatform("lift-tinker-and-shift.")

Definition of Replatform

Changing the underlying infrastructure technology that an application runs in today, as we move it to the cloud. Some application changes may be required, but not a complete refactor.

Replatforming leads to cloud optimizations to the application during the migration stage due to some cloud platform adoption, while keeping the application core architecture the same, so that does require some programming input and expertise.
Replatformed applications show some cloud-native characteristics like horizontal scaling and portability. Often, Replatforming is used when replacing database backends of applications with a corresponding PaaS database solution of a cloud provider.
For example, you might end up moving from your relational database system to a turnkey managed RDS on a cloud provider — same underlying tech, a different business model with cloud resiliency auto-added.

Use Cases

Use Replatform if you want to:

migrate with a time-crunch.
leverage the benefits of the cloud without refactoring the app migrate a complex on-premises app with minor tweaks for cloud benefits.

3) Repurchase("“drop and shop”")

Definition of Repurchase

Replacing the current system by purchasing a SaaS solution that meets the needs and requirements of the current application. Note: This can result in a data-migration and transformation project of its own.

This is sometimes referred to as “drop and shop,” as it refers to a decision to move to another product.
Repurchasing (also called Replacing) is the strategy where the legacy application is fully replaced by a SaaS solution that provides the same or similar capabilities.
The migration effort heavily depends on the requirements and options of migrating (live) data. Some SaaS replacements for on-premise products from the same vendor offer an option to quickly migrate data with little effort or even automatically. Some providers offer analysis tools to assess the to-be-expected migration effort. However, this might not be the case when switching to a product of a different vendor or if the migration path has been interrupted due to neglected maintenance of the on-premise application.

The “repurpose” strategy is often applied when using a proprietary database platform or a proprietary product and moving to something else.
An example of this is moving from an on-premises email server to AWS Simple Email Service (SES). Another example is moving the organization’s CRM to Salesforce.

Use Cases:

Use Repurchase if:

You’re replacing software for standard functions like finance, accounting, CRM, HRM, ERP, email, CMS, etc.
A legacy app is not compatible with the cloud.

4) Rehost(“lift-and-shift.”)

Definition of Rehost

Sometimes called ‘lift-and-shift’, this involves the replication of virtual machines from their current location into a cloud environment.

Rehosting is commonly referred to as lift and shift.
This strategy is a widely chosen strategy due to the relatively low migration effort and it carries the least amount of risk.
It lifts servers and applications from the on-premises infrastructure and shifts them to copied as-is to a cloud infrastructure.
The most important benefit of this strategy is migration speed because no architectural refactoring needs to be done.
Moreover, the migration can often be done automatically using a variety of lift-and-shift or so-called workload mobility tools.
There are significant benefits to running servers on the scalable, pay-as-you-go infrastructure of a cloud platform.
It’s a relatively low-resistance migration strategy, and it’s a great strategy for working backward from a fixed constraint or a hard deadline.
However, the Rehosting strategy has a major drawback. Using this approach it is not possible to exploit the cloud's entire potential since the applications are not built in a cloud-native fashion. Simply rehosted applications are, compared to cloud-native applications, not decoupled from the operating system[2] and are usually much more difficult to scale. Experience shows that from a cost perspective Rehosting usually does not lead to any major advantage.

Use Cases:

Use Rehost if you’re:

only if speed is the most important factor.
migrating a large-scale enterprise.
new to the cloud.
migrating off-the-shelf applications.
migrating with a deadline.

5) Retain(“revisit” or do nothing.)

Definition of Retain

Keeping the application as-is, in its current environment.

Retain (or Revisit) means that you do not migrate the application. Despite all the benefits of cloud technology, there are still reasons to retain some applications on-premises that are not ready to be migrated and will produce more benefits when kept on-premises, or you are not ready to prioritize an application that was recently upgraded and then make changes to it again.
For example, an application may be reaching its end-of-life soon, and you may not want to invest time and effort in migrating such an application. Retaining such applications as-is may be the right approach.

Use Cases:

Use Retain if:

You adopt a hybrid cloud model during migration.
You’re heavily invested in on-premise applications.
A legacy app is not compatible with the cloud and works well on-premise.
You decide to revisit an app later.

6) Retire("Get rid of old one.")

Definition of Retire

Getting rid of an application.

The Retire strategy means that an application is explicitly phased out.
This strategy involves identifying assets and services that can be turned off so the business can focus on services that are widely used and of immediate value.
If an application is considered not worth migrating to the cloud, it can either be eliminated or downsized. It allows you to explore all your applications in terms of their uses, dependencies, and cost to the company. It is a rather passive strategy as there is no migration.

Use Cases:

Use Retire if:

An app is redundant or obsolete.
A legacy app is not compatible with the cloud and provides no productive value anymore.
You decide to refactor or repurchase an app.

The following strategies are arranged in increasing order of complexity — this means that the time and cost required to enact the migration will be proportional to the increase, but will provide greater opportunity for optimization
Retire(simples) < Retain < Rehost < Repurchase < Replatform < Re-architect/Refactor (most complex)

Best Practices for a Successful Cloud Migration

Know your IT portfolio inside out –the data, applications, and infrastructure
Design your migration strategy
Select the right partner for your cloud migration journey
Prepare your team and the existing IT environment for the transition
Leverage automated tools and managed services from cloud services providers wherever possible
Track and monitor the migration process continuously
Test and validate for optimization

References

Connect with me at Sagar-R-Ravkhande

DEV Community: Sagar R Ravkhande

Flask Application Deployment using AWS ECS and AWS DynamoDB with Terraform

Architecture

Requirements

1. Contents

2. How to run and test locally

3. How-to Deploy into AWS ECS using github action CI/CD and Terraform for IAC

4. File Descriptions

5. Deployment Steps

6. Response Examples

Conclusion

Create Azure backup of VM using Automation script in AzCli

Summary

SonarQube Infrastructure Setup using AWS EC2 and PostgreSQL

Project: SonarQube Infrastructure Setup

Overview

Table of Contents

Infrastructure Setup

EC2 Instance Setup

PostgreSQL Database Setup

SonarQube Installation

Jenkins Integration

Jenkins Installation

Jenkins Configuration

Jenkins SonarQube Plugin

Pipeline Setup

Pipeline Definition (Jenkinsfile)

Triggering Code Analysis

Conclusion

Create MongoDB Atlas Cluster With Terraform and AWS

Prerequisites Needed:

Table of Contents

MongoDB Atlas API key for terraform

Configuring the MongoDB atlas provider for Terraform

Module for cluster resources

Cluster resources

Users and Roles

Network peering with atlas

Plan and Apply your terraform code

Resources

AWS IAM best practices to secure your resources...

Key Components of AWS IAM

Key features of AWS IAM

Linux Troubleshooting Scenarios - Part 3

Issue 1: Unable to Run Certain Commands

Issue 2: System Unexpectedly reboots and process restart?

Issue 3: Unable to get IP Address

Issue 4: Backup and Restore File Permissions in Linux

Useful Tip-Related Disk Partition:

Linux Troubleshooting Scenarios - Part 2

Issue 1: fstab file missing or bad entry

Issue 2: Can’t cd to the directory even if the user has sudo privileges

Issue 3: Can’t Create Links

Issue 4: Running Out of Memory

Issue 5: Add/ Extend the Swap Space

Linux Troubleshooting Scenarios - Part 1

Issue 1: Server is not reachable or unable to connect

Issue 2: Unable to connect to a website or an application

Issue 3: Unable to ssh as root or any other user.

Issue 4: Disk Space is full issue or add/extend disk space

Issue 5: Filesystem corrupted

𝐒𝐎𝐂 1 𝐯𝐬. 𝐒𝐎𝐂 2 𝐯𝐬. 𝐒𝐎𝐂 3

SOC 1 vs. SOC 2. vs. SOC 3: An Overview‍

The Major differences between Soc 1 vs. SOC 2. vs. SOC 3

Object Oriented Programming (OOP) Concepts

Procedure-Oriented Programming Language

Characteristics of procedure-oriented programming:

The disadvantage of procedure-oriented programming languages is:

Object Oriented Programing Language

Characteristics of Object-Oriented programming:

Procedure Oriented Programming (POP) Vs Object Oriented Programming (OOP)

Object-Oriented Programming Principles

Key Takeaways:

What is a NoSQL database?

Defining NoSQL

Identifying types of NoSQL databases

Key/value

Key features of the key-value store:

Column

Document