DEV Community: Sagar R Ravkhande

Deploy a Flask API on AWS ECS Fargate with DynamoDB and Terraform (End-to-End)

Sagar R Ravkhande — Mon, 31 Mar 2025 21:07:54 +0000

In this post we'll build and deploy a small Flask API end to end on AWS:

Flask + gunicorn in a hardened container
Amazon ECS on Fargate behind an Application Load Balancer
Amazon DynamoDB for storage
Terraform for all infrastructure, with tests
GitHub Actions for CI/CD, authenticating to AWS with OIDC (no stored keys)

sagary2j / python-flask-hello-world

Python Flask hello-world application deployment into AWS ECS

Hello World Flask Application Deployment using AWS ECS and AWS DynamoDB

This is a simple "Hello World" application that exposes two HTTP-based APIs:

PUT /hello/<username>: Saves or updates the given user's name and date of birth in the database.
GET /hello/<username>: Returns a hello message for the given user, including their birthday message if it's today or in the next N days.

Architecture

Requirements

Docker(local)
Terraform
Python 3.8+
Flask 2.0+
SQLite3(local)
AWS account for deployment
AWS DynamoDB

Deployment Architecture

Cloud Platform: AWS (Amazon Web Services)
Github Action: CI/CD Deployment.
AWS ECS: To run the application as a containers in EC2.
AWS DynamoDB: Amazon DynamoDB is a fully managed proprietary NoSQL database.
AWS S3: For storing terraform state file.
AWS CloudWatch: For logging and monitoring.

1. Contents

./applocal.py
- contains a Python Flask application, which is integrated with SQLite3
./app.py
- contains a Python Flask application, which is integrated with DynamoDB…

View on GitHub

What we're building

A "Hello Birthday" API with three endpoints:

Method	Path	Body	Success	Errors
`PUT`	`/hello/<username>`	`{"dateOfBirth": "YYYY-MM-DD"}`	`204 No Content`	`400` invalid input, `503` storage unavailable
`GET`	`/hello/<username>`	none	`200 {"message": "..."}`	`400` invalid username, `404` unknown user
`GET`	`/health`	none	`200 {"status": "ok"}`	none

The rules:

username must contain letters only.
dateOfBirth must be a valid date before today.
People born on 29 February celebrate on 28 February in non-leap years.

$ curl -X PUT localhost:8000/hello/alice \
    -H 'Content-Type: application/json' -d '{"dateOfBirth": "1990-07-05"}'
$ curl localhost:8000/hello/alice
{"message": "Hello, alice! Your birthday is in 5 day(s)"}
# ...and on 5 July:
{"message": "Hello, alice! Happy birthday!"}

Architecture

The ALB sits in two public subnets. It is the only thing that can reach the tasks; the service security group accepts traffic from the ALB security group only.
Fargate tasks run in private subnets with no public IP. They reach DynamoDB through a free gateway VPC endpoint, and reach ECR and CloudWatch Logs through a NAT gateway.
Application Auto Scaling keeps average CPU at 60%, between 2 and 5 tasks.
GitHub Actions assumes an IAM role via OIDC, pushes an image tagged with the commit SHA, then runs terraform apply.

The diagram itself is code: docs/architecture.py uses the diagrams library, so it is versioned with the infrastructure it describes.

Repository layout

app/                 Flask app: factory, routes, birthday logic, storage backends
tests/               pytest suite (no AWS needed; DynamoDB is mocked with moto)
wsgi.py              WSGI entry point (gunicorn wsgi:app)
gunicorn.conf.py     Production server settings
Dockerfile           Multi-stage, non-root image with a health check
terraform/           Main stack: VPC, ALB, ECS Fargate, DynamoDB, IAM, autoscaling
terraform/tests/     terraform test suite with a mocked AWS provider
terraform/bootstrap/ One-time stack: GitHub OIDC provider, deploy role, ECR repository
.github/workflows/   CI/CD pipeline

1. The application

One app, pluggable storage

v1 had two near-identical files: app.py for DynamoDB and applocal.py for SQLite. Now there's one app built by a factory, and storage sits behind a tiny interface:

class UserRepository(Protocol):
    def save(self, username: str, date_of_birth: date) -> None: ...

    def get(self, username: str) -> date | None: ...

DynamoDBUserRepository and SQLiteUserRepository both implement it, and an environment variable picks one:

STORAGE_BACKEND=sqlite python wsgi.py     # local development, no AWS
STORAGE_BACKEND=dynamodb TABLE_NAME=users # what ECS runs

The factory also accepts the repository and a today function as parameters. That's what makes the tests fast and deterministic (more on that below):

def create_app(
    settings: Settings | None = None,
    repository: UserRepository | None = None,
    today: Callable[[], date] = date.today,
) -> Flask:

Getting the birthday maths right

This is the whole calculation. It works on plain date objects, so there are no time-of-day surprises:

from datetime import date


def _birthday_in(year: int, date_of_birth: date) -> date:
    try:
        return date_of_birth.replace(year=year)
    except ValueError:
        # 29 February in a non-leap year: celebrate on 28 February.
        return date(year, 2, 28)


def days_until_birthday(date_of_birth: date, today: date) -> int:
    next_birthday = _birthday_in(today.year, date_of_birth)
    if next_birthday < today:
        next_birthday = _birthday_in(today.year + 1, date_of_birth)
    return (next_birthday - today).days


def greeting(username: str, date_of_birth: date, today: date) -> str:
    days = days_until_birthday(date_of_birth, today)
    if days == 0:
        return f"Hello, {username}! Happy birthday!"
    return f"Hello, {username}! Your birthday is in {days} day(s)"

Validating input properly

A missing or non-JSON body used to raise a KeyError and return a 500. Now every input is checked before anything touches the database:

@bp.put("/hello/<username>")
def save_user(username: str):
    if not username.isalpha():
        return _error("Username must contain only letters.", 400)

    payload = request.get_json(silent=True)
    raw_date = payload.get("dateOfBirth") if isinstance(payload, dict) else None
    if not isinstance(raw_date, str):
        return _error('Request body must be JSON like {"dateOfBirth": "YYYY-MM-DD"}.', 400)

    try:
        date_of_birth = datetime.strptime(raw_date, "%Y-%m-%d").date()
    except ValueError:
        return _error("Invalid date format. Use YYYY-MM-DD.", 400)

    if date_of_birth >= _today():
        return _error("Date of birth must be in the past.", 400)

    try:
        _repository().save(username, date_of_birth)
    except RepositoryError:
        current_app.logger.exception("Could not save user %s", username)
        return _error("Storage is temporarily unavailable.", 503)
    return "", 204

All errors are JSON, and AWS error details are logged, never returned to the client.

2. Tests that don't need AWS or a lucky date

The old tests called a real DynamoDB table and asserted things like "the birthday is in 1 day" for a hardcoded date, so they only passed on certain days of the year. The new suite injects an in-memory repository and a fixed clock:

TODAY = date(2026, 6, 30)


@pytest.fixture
def client(repository):
    app = create_app(settings=Settings(), repository=repository, today=lambda: TODAY)
    return app.test_client()

The birthday logic gets table-driven tests, including the leap-year edge cases:

@pytest.mark.parametrize(
    ("date_of_birth", "today", "expected"),
    [
        (date(1990, 6, 30), date(2026, 6, 30), 0),
        (date(1990, 7, 1), date(2026, 6, 30), 1),
        # The next birthday falls after 29 Feb 2028, so the gap is 366 days.
        (date(1990, 3, 4), date(2027, 3, 5), 365),
        # Leap-day birthdays are celebrated on 28 Feb in non-leap years.
        (date(2000, 2, 29), date(2027, 2, 28), 0),
        (date(2000, 2, 29), date(2028, 2, 29), 0),
    ],
)
def test_days_until_birthday(date_of_birth, today, expected):
    assert days_until_birthday(date_of_birth, today) == expected

The DynamoDB repository is tested against moto, so the real boto3 calls run with no AWS account. The suite has 44 tests at 100% coverage and runs in under two seconds:

make install
make test

3. A production-grade container

The old image ran python app.py, which meant the Flask development server with debug=True on 0.0.0.0, as root. The Werkzeug debugger allows arbitrary code execution, so that is never safe to expose. The new Dockerfile:

# syntax=docker/dockerfile:1

FROM python:3.13-slim AS build
ENV PIP_NO_CACHE_DIR=1 PIP_DISABLE_PIP_VERSION_CHECK=1
COPY requirements.txt .
RUN pip install --prefix=/install -r requirements.txt

FROM python:3.13-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1 \
    PORT=8000
RUN useradd --system --uid 10001 --no-create-home app
WORKDIR /srv
COPY --from=build /install /usr/local
COPY app ./app
COPY wsgi.py gunicorn.conf.py ./
USER 10001
EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
    CMD ["python", "-c", "import os, urllib.request; urllib.request.urlopen(f'http://127.0.0.1:{os.environ[\"PORT\"]}/health', timeout=2)"]
CMD ["gunicorn", "wsgi:app"]

It's a multi-stage build, and dependencies come from pinned requirements.txt.
It runs as a non-root user (uid 10001).
gunicorn replaces the dev server.
A .dockerignore allowlist keeps Terraform, tests and .git out of the image.

4. Infrastructure with Terraform

Pin everything

In v1, terraform init pulled whatever AWS provider was latest. When v6 removed the vpc = true argument on aws_eip, the configuration stopped validating, and every deploy would have failed. Now versions are pinned, lock files are committed, and the S3 backend uses native state locking (Terraform 1.10+):

terraform {
  required_version = ">= 1.10"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }

  backend "s3" {
    bucket       = "<your-state-bucket>"
    key          = "global/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true
    use_lockfile = true
  }
}

Fargate service with automatic rollback

The ECS-on-EC2 setup needed a launch configuration (deprecated, and blocked for new AWS accounts), an Auto Scaling group, an ECS-optimised AMI and instance roles. Fargate removes all of that. The service is a single resource:

resource "aws_ecs_service" "app" {
  name                              = var.project_name
  cluster                           = aws_ecs_cluster.main.id
  task_definition                   = aws_ecs_task_definition.app.arn
  desired_count                     = var.min_tasks
  launch_type                       = "FARGATE"
  health_check_grace_period_seconds = 30
  wait_for_steady_state             = true

  deployment_circuit_breaker {
    enable   = true
    rollback = true
  }

  network_configuration {
    subnets          = aws_subnet.private[*].id
    security_groups  = [aws_security_group.service.id]
    assign_public_ip = false
  }

  load_balancer {
    target_group_arn = aws_lb_target_group.app.arn
    container_name   = local.container_name
    container_port   = var.container_port
  }

  lifecycle {
    # Application Auto Scaling owns the running task count.
    ignore_changes = [desired_count]
  }
}

wait_for_steady_state makes terraform apply fail if the new tasks never become healthy.
The circuit breaker automatically rolls back to the last working task definition.

Least-privilege IAM

The task role can do exactly two things, on exactly one table:

data "aws_iam_policy_document" "task_dynamodb" {
  statement {
    actions   = ["dynamodb:GetItem", "dynamodb:PutItem"]
    resources = [aws_dynamodb_table.user_table.arn]
  }
}

There's also a separate execution role, which ECS uses to pull the image and write logs. The table has point-in-time recovery and deletion protection.

Test the Terraform too

terraform validate only checks syntax. The repo also has a terraform test suite that plans against a mocked AWS provider, so CI can check the logic without credentials:

run "defaults" {
  command = plan

  assert {
    condition     = length(aws_nat_gateway.this) == 1 && length(aws_route.private_nat) == 2
    error_message = "Default should share one NAT gateway across both private subnets."
  }

  assert {
    condition     = jsondecode(aws_ecs_task_definition.app.container_definitions)[0].image == "123456789012.dkr.ecr.us-east-1.amazonaws.com/flask-docker-app:0123abc"
    error_message = "Task definition must deploy the image_tag that CI passes in."
  }
}

Other runs check that setting certificate_arn switches the ALB to HTTPS with an HTTP-to-HTTPS redirect, and that single_nat_gateway = false creates one NAT gateway per AZ.

5. CI/CD with GitHub Actions and OIDC

No more access keys

v1 stored AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as GitHub secrets. Now a small bootstrap stack, applied once by an admin, creates:

the GitHub OIDC identity provider;
a deploy role that only a job in the repo's production environment can assume;
an ECR repository with immutable tags, scan on push and a lifecycle policy.

The important part is the trust policy:

condition {
  test     = "StringEquals"
  variable = "token.actions.githubusercontent.com:sub"
  values   = ["repo:${var.github_repository}:environment:${var.github_environment}"]
}

The deploy role gets PowerUserAccess, plus IAM permissions limited to roles prefixed with the project name. It can't create arbitrary admin roles.

The pipeline

Every pull request runs three jobs in parallel:

Lint and test: ruff and pytest, with a 90% coverage gate.
Build and smoke-test image: builds the Dockerfile, runs it, does a real PUT/GET round trip, and fails if the container runs as root.
Terraform checks: fmt, validate for both stacks, and terraform test.

Merges to the default branch then run the deploy job:

deploy:
  needs: [test, container, terraform]
  if: >-
    github.ref_name == github.event.repository.default_branch &&
    (github.event_name == 'push' || github.event_name == 'workflow_dispatch')
  environment: production
  permissions:
    contents: read
    id-token: write
  env:
    IMAGE_TAG: ${{ github.sha }}
  steps:
    - uses: actions/checkout@v7

    - uses: aws-actions/configure-aws-credentials@v6
      with:
        role-to-assume: ${{ vars.AWS_DEPLOY_ROLE_ARN }}
        aws-region: ${{ env.AWS_REGION }}

    # ...log in to ECR, build and push $IMAGE_TAG...

    - name: Terraform plan
      working-directory: terraform
      run: |
        terraform init -input=false
        terraform plan -input=false -var="image_tag=$IMAGE_TAG" -out=tfplan

    - name: Terraform apply
      working-directory: terraform
      run: terraform apply -input=false tfplan

    - name: Smoke test
      working-directory: terraform
      run: |
        url="$(terraform output -raw alb_url)"
        curl -fsS --retry 12 --retry-delay 10 --retry-all-errors "$url/health"

Commit-SHA image tags. v1 always pushed :latest. The task definition never changed, so Terraform saw nothing to do and ECS kept running the old image. A unique tag per commit forces a new task definition and a rolling deployment.
plan -out then apply tfplan. What gets applied is exactly what was planned.
The production environment is the only thing the deploy role trusts. Restrict it to the default branch and optionally add required reviewers for a manual approval gate.

6. Deploy it yourself

Bootstrap (once, with admin credentials):

   cd terraform/bootstrap
   terraform init
   terraform apply   # -var create_oidc_provider=false if your account already has the GitHub OIDC provider

Configure GitHub:
- Create an environment named production, restricted to your default branch.
- Add a repository variable AWS_DEPLOY_ROLE_ARN set to the deploy_role_arn output.
Push to the default branch. The pipeline tests, builds, pushes, applies and smoke-tests. The ALB URL is printed in the job summary.

To serve HTTPS, pass an ACM certificate: -var certificate_arn=arn:aws:acm:.... The ALB then serves TLS 1.2/1.3 and redirects port 80.

7. What was wrong with v1 (and what I learned)

v1 problem	Impact	Fix
Unpinned Terraform providers	AWS provider v6 removed `aws_eip.vpc`, so the config stopped validating	`required_providers` pins and committed lock files
Always pushing `:latest`	Task definition never changed, so deploys didn't roll out new code	Immutable commit-SHA tags
`days + 1` and `== 365` birthday check	Said "in 366 day(s)" on the actual birthday whenever the next birthday came after 29 Feb, and crashed for 29 Feb birthdays	Plain `date` arithmetic with a leap-day rule
Flask debug server as root	Remote code execution risk through the Werkzeug debugger	gunicorn, non-root user
Tests needed live DynamoDB and specific dates	Failed most days of the year, and in CI	Injected clock, in-memory repo, moto
Long-lived AWS keys in CI	Keys could leak and never expire	GitHub OIDC with a scoped role
Two CloudWatch alarms with the same name	One silently overwrote the other	Target tracking only, which manages its own alarms
Launch configuration and EC2 ASG	Deprecated, more to patch and pay for	Fargate

The bigger lesson: a pipeline that only runs terraform apply isn't CI/CD. None of these bugs were hard to fix; they survived because nothing tested the code before it shipped. Adding the test, image and Terraform checks on every PR made all of them visible.

Cost and clean-up

For a demo, the biggest line items are usually the NAT gateway and the ALB, not Fargate. The stack defaults to a single NAT gateway (single_nat_gateway = false gives one per AZ for resilience), and DynamoDB traffic goes through the free gateway endpoint instead of NAT.

To tear everything down:

cd terraform
terraform apply   -var image_tag=<tag> -var dynamodb_deletion_protection=false
terraform destroy -var image_tag=<tag>
cd bootstrap && terraform destroy   # empty the ECR repository first

Conclusion

The API is the same small birthday service, but it's now built the way production services should be:

the application is testable;
the container is locked down;
the infrastructure is pinned and tested;
the pipeline deploys immutable images without storing a single AWS key.

The full code, including the bootstrap stack, Terraform tests and diagram-as-code, is in the repository. Questions and suggestions are welcome in the comments.

Automate Azure VM Backups with Az CLI: Vault Provisioning, Protection, and On-Demand Backup

Sagar R Ravkhande — Mon, 31 Mar 2025 20:25:30 +0000

If your backup process depends on manual portal actions, recovery reliability is at risk. This script-first approach automates Recovery Services vault checks, VM protection, and backup execution from CI or scheduled jobs.

The script authenticates with a service principal, selects the target subscription, verifies or creates a Recovery Services vault, configures storage redundancy, checks VM backup protection status, and triggers an on-demand backup if no backup job is already running.

Architecture / flow

Authenticate non-interactively via service principal.
Set active subscription.
Validate Recovery Services vault existence; create if missing.
Set vault backup storage redundancy.
Verify VM protection; enable protection when absent.
Check for in-progress jobs.
Trigger backup-now and print backup job status.

Prerequisites

Azure CLI installed and authenticated for automation use
Service principal with permissions for backup and vault operations
Existing VM and resource group
Target subscription ID/name

echo "Logging In...."
az login --service-principal -u $Clientid -p $Clientsecret --tenant $Tenantid

az account set -s "ba-ib-at23055-neu-dev"

RecoveryServicesVault="at23055vault-test"
resourceGroup="AT23055_DRMDASHBOARD_DEV"
vmName="xd934b23055dev2"

az backup vault show --name $RecoveryServicesVault --resource-group $resourceGroup

retVal=$?
if [ $retVal -eq 0 ]; then
        echo "Vault Exists Already"
else
        echo "Vault doesn't Exist!"
        echo "Creating a new Vault.."
        az backup vault create --resource-group $resourceGroup \
        --name $RecoveryServicesVault \
        --location northeurope
fi

az backup vault backup-properties set \
    --name $RecoveryServicesVault  \
    --resource-group $resourceGroup \
    --backup-storage-redundancy "LocallyRedundant"
#GeoRedundant


az backup protection check-vm --resource-group $resourceGroup --vm $vmName

if [ $retVal -eq 0 ]; then
        echo "Virtual machine is protected Already"
else
        echo "Virtual machine is not protected"
        echo "Creating Virtual machine Protection..."
        az backup protection enable-for-vm \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --vm $vmName \
        --policy-name DefaultPolicy
fi

retention=`date +'%d-%m-%Y' -d "+1 year"`

count=$(az backup job list --resource-group $resourceGroup --vault-name $RecoveryServicesVault --output table | grep -i 'InProgress' | wc -l)

if [ $count -gt 0 ]; then
        echo "Backup is InProgress, Unable to initiate backup as another backup operation is currently in progress."
        echo "Checking the status of backup jobs..."
        az backup job list \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --output table
        exit 0
else
        echo "Initiating backup job.."
        az backup protection backup-now \
        --resource-group $resourceGroup \
        --vault-name $RecoveryServicesVault \
        --container-name $vmName \
        --item-name $vmName \
        --backup-management-type AzureIaaSVM \
        --retain-until $retention
fi


echo "Checking the status of backup jobs..."
az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table

Please find below the concise explanation of the provided code snippet that is intended to manage Azure backup vaults and virtual machines using the Azure CLI:

Explanation of the Code

echo "Logging In...."  
az login --service-principal -u $Clientid -p $Clientsecret --tenant $Tenantid

This logs into Azure using a service principal (a special Azure account typically used for automated scripts). It uses the client ID, client secret, and tenant ID provided in the variables.

Set the Azure Account:

az account set -s "<Subscription_ID>"

Sets the specific Azure subscription to use for the subsequent commands.

Define Variables:

RecoveryServicesVault="vault-test"  
resourceGroup="RG_DEV"  
vmName="xdvmdev2"

These lines define variables for the recovery services vault name, resource group, and virtual machine name.

Check if the Backup Vault Exists:

az backup vault show --name $RecoveryServicesVault --resource-group $resourceGroup  
retVal=$?  
if [ $retVal -eq 0 ]; then  
    echo "Vault Exists Already"  
else  
    echo "Vault doesn't Exist!"  
    echo "Creating a new Vault.."  
    az backup vault create --resource-group $resourceGroup \
    --name $RecoveryServicesVault \
    --location northeurope  
fi

It checks if the specified backup vault exists.
If it does, it confirms its existence; if not, it creates a new backup vault in the specified location.

Set Backup Properties:

az backup vault backup-properties set \
    --name $RecoveryServicesVault  \
    --resource-group $resourceGroup \
    --backup-storage-redundancy "LocallyRedundant"

This sets the backup storage redundancy to "LocallyRedundant," meaning that the backups will be stored in a way that protects them from local failures.

Check VM Backup Protection Status:

az backup protection check-vm --resource-group $resourceGroup --vm $vmName

This checks if the specified virtual machine is already protected by the backup vault.

Enable VM Protection if Not Already Protected:

if [ $retVal -eq 0 ]; then  
    echo "Virtual machine is protected Already"  
else  
    echo "Virtual machine is not protected"  
    echo "Creating Virtual machine Protection..."  
    az backup protection enable-for-vm \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --vm $vmName \
    --policy-name DefaultPolicy  
fi

If the VM is not protected, it enables backup protection for the VM with the default policy.

Initiate Backup Job:

retention=`date +'%d-%m-%Y' -d "+1 year"`  
count=$(az backup job list --resource-group $resourceGroup --vault-name $RecoveryServicesVault --output table | grep -i 'InProgress' | wc -l)  

if [ $count -gt 0 ]; then  
    echo "Backup is InProgress, Unable to initiate backup as another backup operation is currently in progress."  
    echo "Checking the status of backup jobs..."  
    az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table  
    exit 0  
else  
    echo "Initiating backup job.."  
    az backup protection backup-now \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --container-name $vmName \
    --item-name $vmName \
    --backup-management-type AzureIaaSVM \
    --retain-until $retention  
fi

It defines a retention policy by calculating a date one year from now.
It checks if another backup job is in progress. If so, it displays the active jobs and exits.
If no jobs are in progress, it initiates a new backup job for the VM.

Check Backup Job Status:

echo "Checking the status of backup jobs..."  
az backup job list \
    --resource-group $resourceGroup \
    --vault-name $RecoveryServicesVault \
    --output table

Finally, it retrieves and displays the status of all backup jobs associated with the vault.

Summary

This script automates the process of logging into Azure, checking or creating a backup vault, managing the backup protection of a specified VM, and initiating a backup job, while also checking for existing operations to avoid conflicts.

SonarQube Infrastructure Setup using AWS EC2 and PostgreSQL

Sagar R Ravkhande — Sun, 31 Mar 2024 22:41:30 +0000

Providing a project explanation or documentation for setting up SonarQube infrastructure on EC2, using PostgreSQL as a database, and integrating it with Jenkins to run sonar-scanner on multiple jobs as part of a Groovy pipeline is essential for clarity and future reference. Here's a sample project explanation:

Project: SonarQube Infrastructure Setup

Overview

This project aims to set up a SonarQube environment on Amazon EC2, utilizing PostgreSQL as the database to store code analysis results. Additionally, it integrates SonarQube with Jenkins to automate code quality analysis using the SonarScanner tool in multiple Jenkins jobs.

Project Overview
Infrastructure Setup
- EC2 Instance Setup
- PostgreSQL Database Setup
- SonarQube Installation
Jenkins Integration
- Jenkins Installation
- Jenkins Configuration
- Jenkins SonarQube Plugin
Pipeline Setup
- Pipeline Definition (Jenkinsfile)
- Triggering Code Analysis
Conclusion

Infrastructure Setup

EC2 Instance Setup

Launch an Amazon EC2 instance using the desired Amazon Machine Image (AMI) with necessary security group settings. Ensure that the instance has adequate resources (CPU, RAM, etc.) for SonarQube.
SSH into the EC2 instance and configure it according to the SonarQube installation requirements.

PostgreSQL Database Setup

Set up a PostgreSQL database on a separate EC2 instance or on a managed PostgreSQL service like Amazon RDS.
Create a dedicated database and user for SonarQube with appropriate permissions.
Update the SonarQube configuration to point to the PostgreSQL database.

SonarQube Installation

Download and install SonarQube on the EC2 instance.
Configure SonarQube settings, including database connection details, authentication, and security settings.
Start the SonarQube service and ensure it's running.

Jenkins Integration

Jenkins Installation

Install Jenkins on a separate EC2 instance or use an existing Jenkins installation.
Configure Jenkins to run as a service and access it via its web interface.

Jenkins Configuration

Install and configure necessary plugins for Jenkins, including Git, Pipeline, and others as needed.
Set up authentication and authorization settings for Jenkins.

Jenkins SonarQube Plugin

Install the SonarQube Scanner for Jenkins plugin in Jenkins.
Configure the SonarQube server URL and authentication token within Jenkins.

Pipeline Setup

Pipeline Definition (Jenkinsfile)

Define a Jenkins pipeline using a Jenkinsfile. This pipeline defines how code analysis using SonarScanner should be executed.
The Jenkinsfile should specify the Git repository, branch, and the build steps including sonar-scanner execution.

Triggering Code Analysis

Create multiple Jenkins jobs or pipeline stages for different projects or branches.
Configure these jobs to execute the Jenkins pipeline, which in turn triggers the sonar-scanner for code analysis.
Set up triggers or schedules to periodically run code analysis jobs or integrate them into your CI/CD workflow.

Conclusion

This project provides a comprehensive setup for SonarQube infrastructure on EC2, integrates it with PostgreSQL as a database, and uses Jenkins to automate code quality analysis. By following the steps outlined in this documentation, you can maintain code quality and ensure continuous improvement in your software development process.

This project explanation should serve as a high-level guide for setting up and integrating SonarQube with Jenkins for code analysis in your organization. You can expand on each section with specific details, commands, and configuration options based on your environment and requirements. Additionally, consider providing links to relevant documentation and resources for further reference.

Create MongoDB Atlas Cluster With Terraform and AWS

Sagar R Ravkhande — Sun, 31 Mar 2024 22:39:27 +0000

This project aims to set up an Atlas MongoDB cluster with an AWS network peering to access the resources from AWS EC2, utilizing Terraform as an Infrastructure as a Code(IAC).

Prerequisites Needed:

Basic understanding of Terraform concepts and Terraform CLI installed.
AWS VPC with subnets and route tables.
MongoDB Atlas account.
MongoDB Atlas Organization and a Project under your account with public and private keys with Organization Project Creator API key.

MongoDB Atlas API key for terraform
MongoDB atlas terraform provider
Module for cluster resource
- Cluster Resources
- Users and Roles
- Network peering with atlas
Plan and apply
Resources

MongoDB Atlas API key for terraform

Once you create an organization in Atlas, access to an organization or project can only be managed using the API key, so we need to create an API key.

API keys have two parts: a Public Key and a Private Key. These two parts serve the same function as a username and a personal API key when you make API requests to Atlas.

You must grant roles to API keys as you would for users to ensure the API keys can call API endpoints without errors. Here we will need an API key that will have Organization Project Creator permissions.
e.g.
All API keys belong to the organization. You can give an API key access to a project. Each API key belongs to only one organization, but you can grant an API key access to any number of projects in that organization.

Configuring the MongoDB atlas provider for Terraform

The Terraform MongoDB Atlas Provider is a plugin that allows you to manage MongoDB Atlas resources using Terraform.
Syntax:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.4"
    }
    mongodbatlas = {
      source  = "mongodb/mongodbatlas"
      version = "~> 1.9"
    }
  }
}

provider "mongodbatlas" {
  public_key = "<YOUR PUBLIC KEY HERE>"
  private_key  = "<YOUR PRIVATE KEY HERE>"
}

In our case, all the required variables like Organization ID, Public key, and Private key are created in the AWS Systems Manager Parameter Store and are being referred to respectively.

data "aws_ssm_parameter" "private_key" {
  name            = "/atlas/private-key"
  with_decryption = true
}

data "aws_ssm_parameter" "public_key" {
  name            = "/atlas/public-key"
  with_decryption = true
}

data "aws_ssm_parameter" "atlas_organization_id" {
  name            = "/atlas/org-id"
  with_decryption = true
}

Module for cluster resources

Cluster resources

mongodbatlas_project provides a Project resource. This allows the project to be created.
mongodbatlas_network_container provides a Network Peering Container resource. The resource lets you create, edit, and delete network peering containers. You must delete network peering containers before creating clusters in your project. You can't delete a network peering container if your project contains clusters. The resource requires your Project ID.
mongodbatlas_project_ip_access_list provides an IP Access List entry resource that grants access from IPs, CIDRs, or AWS Security Groups (if VPC Peering is enabled) to clusters within the Project.

resource "mongodbatlas_project" "this" {
  name   = var.atlas_project_name
  org_id = var.atlas_organization_id
}

resource "mongodbatlas_network_container" "this" {
  atlas_cidr_block = var.atlas_cidr_block
  project_id       = mongodbatlas_project.this.id
  provider_name    = "AWS"
  region_name      = local.region_name
}

resource "mongodbatlas_project_ip_access_list" "this" {
  project_id = mongodbatlas_network_peering.this.project_id
  cidr_block = var.vpc_cidr_block
  comment    = "Grant AWS ${var.vpc_cidr_block} environment access to Atlas resources"
}

resource "aws_route" "this" {
  for_each                  = toset(var.private_route_table_ids)
  route_table_id            = each.value
  destination_cidr_block    = mongodbatlas_network_container.this.atlas_cidr_block
  vpc_peering_connection_id = aws_vpc_peering_connection_accepter.this.vpc_peering_connection_id
}

Users and Roles

mongodbatlas_custom_db_role allows you to create custom roles in Atlas when the built-in roles don't include your desired set of privileges. Atlas applies each database user's custom roles together with:

Any built-in roles you assign when you add a database user or modify a database user.
Any specific privileges you assign when you add a database user or modify a database user. You can assign multiple custom roles to each database user. E.g.

resource "mongodbatlas_custom_db_role" "roles" {
  for_each   = var.custom_roles
  project_id = module.atlas_project.project_id
  role_name  = each.value.role_name

  dynamic "actions" {
    for_each = each.value.actions
    content {
      action = actions.value.action
      resources {
        collection_name = actions.value.resources.collection_name
        database_name   = actions.value.resources.database_name
      }
    }
  }
}

mongodbatlas_database_user Creates database users to provide clients access to the database deployments in your project. A database user's access is determined by the roles assigned to the user.

resource "mongodbatlas_database_user" "database_users" {
  for_each           = var.database_users
  username           = each.value.username
  password           = jsondecode(data.aws_secretsmanager_secret_version.creds.secret_string)[each.value.username]
  auth_database_name = "admin"
  project_id         = module.atlas_project.project_id

  dynamic "roles" {
    for_each = each.value.roles
    content {
      database_name = roles.value.database_name
      role_name     = roles.value.role_name
    }
  }
  depends_on = [mongodbatlas_custom_db_role.roles]
}

You can create and upload the username and password details as a key pair values in AWS secret manager using,

aws secretsmanager create-secret --name atlas-users --description "Mytest with multiples values" --secret-string file://secretmanagervalues.json

Where,

Which will have below values e.g.
{"Juan":"mykey1","Pedro":"mykey2","Pipe":"mykey3"}

Network peering with atlas

mongodbatlas_network_peering Network peering establishes a private connection between your Atlas VPC and your cloud provider's VPC. The connection isolates traffic from public networks for added security.

resource "mongodbatlas_network_peering" "this" {
  container_id           = mongodbatlas_network_container.this.id
  project_id             = mongodbatlas_project.this.id
  provider_name          = "AWS"
  accepter_region_name   = data.aws_region.current.id
  vpc_id                 = var.vpc_id
  aws_account_id         = data.aws_caller_identity.current.account_id
  route_table_cidr_block = var.vpc_cidr_block
}

resource "aws_vpc_peering_connection_accepter" "this" {
  vpc_peering_connection_id = mongodbatlas_network_peering.this.connection_id
  auto_accept               = true
  tags = {
    Name = var.peering_connection_name
  }
}

E.g.

Plan and Apply your terraform code

Your tfvars file should look like this,

environment.tfvars

vpc_id = "<VPC-ID>"
vpc_cidr_block = "10.0.0.0/16"
peering_connection_name = "tf-mongo-atlas"
atlas_project_name = "<Cluster-Name>"

atlas_cidr_block = "<Atlas-CIDR>"

database_users = {
  user1 = {
    username           = "Juan",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
    ]
  },
  user2 = {
    username           = "Pedro",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
      { database_name = "admin", role_name = "oplogRead" },
    ]
  },
  user3 = {
    username           = "Pipe",
    password           = "",
    auth_database_name = "admin"
    roles = [
      { database_name = "admin", role_name = "readWriteAnyDatabase" },
    ]
  },
  // Add more users as needed
}

custom_roles = {
  oplogRead = {
    role_name = "oplogRead"
    actions = [
      {
        action = "FIND"
        resources = {
          collection_name = ""
          database_name   = "anyDatabase"
        }
      },
      {
        action = "CHANGE_STREAM"
        resources = {
          collection_name = ""
          database_name   = "anyDatabase"
        }
      },
    ]
  }
  # Add more roles if needed
}

Now all you need to do is to plan your terraform code using the terraform plan and verify the changes in the printed plan. You should be seeing one module to be added which will be the atlas cluster with the provided configuration.

After verifying the plan in the above step, run terraform apply to apply your changes. You will see the cluster getting created message on the terminal and your changes will start appearing in your mongoDB console as well. It usually takes about 10 to 15 mins for the cluster to be created.

You will see cluster created like this,

Resources

You can find the project demo code here,
https://github.com/sagary2j/atlas-mongodb-terraform/tree/initial-code
MongoDB atlas: https://registry.terraform.io/providers/mongodb/mongodbatlas/latest
Atlas organizations apiKeys create: https://www.mongodb.com/docs/atlas/cli/stable/command/atlas-organizations-apiKeys-create/#inherited-options

AWS IAM best practices to secure your resources...

Sagar R Ravkhande — Mon, 13 Mar 2023 10:47:38 +0000

AWS IAM (Identity and Access Management) is a web service that enables you to securely control access to AWS resources. IAM enables you to create and manage AWS users and groups and assign policies to control the level of access that each user or group has to specific AWS resources.

Key Components of AWS IAM

User Management: IAM enables you to create and manage individual users within your AWS account. This enables you to control who has access to your AWS resources.

Group Management: IAM enables you to create and manage groups of users. This enables you to assign policies to groups, rather than having to assign policies to individual users.

Access Management: IAM enables you to control access to AWS resources using permissions. Permissions are defined using policies, which specify which actions a user or group can perform on specific AWS resources.

Multi-factor Authentication (MFA): IAM enables you to require MFA for users when they sign into your AWS account. MFA provides an extra layer of security beyond a user's password.

Integration with AWS Services: IAM is integrated with many AWS services, such as Amazon S3, Amazon EC2, and Amazon RDS. This enables you to control access to these services using IAM policies.

Auditing and Logging: IAM provides detailed logging and auditing of all user activity within your AWS account. This enables you to monitor user activity and detect any suspicious behavior.

Overall, AWS IAM provides a robust set of tools for managing access to your AWS resources, helping you maintain security and compliance.

Key features of AWS IAM

• Authentication: AWS IAM lets you create and manage identities such as users, groups, and roles, meaning you can issue and enable authentication for resources, people, services, and apps within your AWS account.

• Authorization: Access management or authorization in IAM is made of two primary components: Policies and Permissions. In the next section, we’ll also look at each of these.

• Fine-grained permissions: Consider this — you want to provide the sales team in your organization access to billing information, but also need to allow the developer team full access to the EC2 service, and the marketing team access to selected S3 buckets. Using IAM, you can configure and tune these permissions as per the needs of your users.

• Shared access to AWS accounts: Most organizations have more than one AWS account, and at times need to delegate access between them. IAM lets you do this without sharing your credentials, and more recently, AWS released Control Tower to further simplify multi-account configurations. We also published a quick, hands-on tutorial on Securing Multi-Account Access on AWS.

• AWS Organizations: For fine-grained control for multiple AWS accounts, you can use AWS Organizations to segment accounts into groups and assign permission boundaries.

• Identity Federation: Many times, your organization will need to federate access from other identity providers such as Okta, G Suite, or Active Directory. IAM enables you to do this with a feature called Identity Federation

*By following these best practices, you can help ensure that your AWS resources are secure and compliant with industry standards and regulations. *

• Avoid the use of root account unless strictly necessary: Do not use the root account for day-to-day administration activities. By default, the root account user has access to all resources for all AWS services and its best practice to create IAM users with least privilege access. Additionally, do not create access keys for the root account unless strictly necessary. Finally, set up monitoring to detect and alert on root account activity, and
ensure hardware-based MFA is set up for root account access.

• Use temporary credentials: Never share your credentials with anyone. It’s advisable to create individual users for anyone who has access requirements and even better use temporary credentials. Dynamically generated credentials that expire after a configurable interval, are a great way to tackle this. You can visit our practical tutorial on Securing Multi-Account Access on AWS for detailed instructions on this.

• Embrace the least privilege principle and review all IAM permissions periodically: It is important to follow the security principle of least privilege, which means that if a user doesn’t need to interact with a resource, then it is best not to provide them access to that resource. IAM permissions allow for very granular access controls, so avoid the use of policy statements that grant access to all actions, all principals, or all resources.
Additionally, use the IAM Access Advisor regularly to ensure that all permissions assigned to a particular user are indeed being used.

• Enforce the least privilege principle to be implemented bi-directionally: Many AWS resources (such as S3 buckets) can have their access policy attached directly. Don’t fall into the trap of thinking that because access is tightly controlled in one direction (i.e., an IAM Role that grants very specific permissions), that you should be less rigid in the other direction (for example, when an S3 bucket access policy grants read access to all entities in your account). Optimally use both sides of the least privilege principle to achieve favorable outcomes.

• Monitor account activity regularly using IAM Access Analyzer and AWS CloudTrail: In addition to what we discussed about the newly released IAM Access Analyzer, the good old AWS CloudTrail is an excellent tool to monitor all activities in your account. You can easily use CloudTrail logs to identify suspicious activity and take necessary actions depending on your findings. You will find our deep-dive, practical tutorial on AWS Security Logging with CloudTrail interesting with step-by-step instructions to help you do this.

•Use Multi-Factor Authentication (MFA): Enable MFA to build an additional layer of security for interaction with the AWS API.

• Enforce strong passwords: Enforce strong passwords by configuring account password policy that involves password rotation, discourages the use of old passwords, only allows alphanumeric characters, and more.

• Monitor and Review Permissions Regularly: Regularly review permissions assigned to users and groups to ensure that they are still needed and appropriate. IAM provides reports and tools to help identify and remediate permissions issues.

• Use IAM Roles for EC2 Instances: Use IAM roles to grant temporary permissions to EC2 instances instead of storing credentials on the instance itself. This reduces the risk of credential exposure in case the instance is compromised.

• Enable AWS CloudTrail: AWS CloudTrail logs all API activity in your AWS account, including IAM actions. This provides a trail of activity that can be used for security analysis, resource change tracking, and compliance auditing.

• Use IAM Policy Conditions: Use IAM policy conditions to further restrict access based on context, such as IP address or time of day. This adds an additional layer of security beyond traditional permissions.

• Enable AWS Security Hub: AWS Security Hub provides a comprehensive view of security alerts and compliance status across your AWS accounts. It can help you identify security issues and provide recommendations for remediation.

Linux Troubleshooting Scenarios - Part 3

Sagar R Ravkhande — Fri, 06 Jan 2023 09:53:35 +0000

Finally!! After Part-2 Here is the final chapter of Linux Troubleshooting Scenarios - Part 3. Below are the scenarios:

Issue 1: Unable to Run Certain Commands

├── Troubleshooting and Resolution
│ ├── command
│ │ ├── Could be the system-related command to which non-root user 
        does not have the access
│ │ ├── Could be the user-defined script/command
│ ├── Troubleshooting
│ │ ├── permission/ownership of the command/script
│ │ ├── sudo permission
│ │ ├── absolute/relative path of command/script
│ │ ├── not defined in user $PATH variable
│ │ ├── command is not installed
│ │ ├── command library is missing or deleted

Issue 2: System Unexpectedly reboots and process restart?

├── Troubleshooting and Resolution
│ ├── System reboot/crash reasons
│ │ ├── CPU stress
│ │ ├── RAM stress
│ │ ├── Kernel fault
│ │ ├── Hardware fault
│ ├── Process restart
│ │ ├── System reboot
│ │ ├── Restart itself
│ │ ├── Watchdog application
│ │ │ ├── To prevent high stress on system resources
│ │ │ ├── If the application causing stress, so it will restart or 
          terminate
│ ├── Troubleshooting
│ │ ├── After logging in, check the status by using commands like 
        uptime, top, dmesg, journalctl, iostat -xz 1
│ │ ├── syslog.log, boot.log, dmesg, messages.log, etc
│ │ ├── custom log path of an application
│ │ ├── if not completely accessible, so take the virtual console 
        like ILO, IDRAC, etc.
│ │ ├── open a case and reach out to a vendor

Issue 3: Unable to get IP Address

├── IP Assignment Methods
│ ├── DHCP
│ │ ├── Fixed Allocation
│ │ ├── Dynamic Allocation
│ ├── Static
├── Troubleshooting
│ ├── check network settings from virtualization environments like 
      VMware, VirtualBox or etc.
│ ├── check whether the IP address is assigned or not
│ ├── check the NIC status from the host side using #lspci, #nmcli 
      etc.
│ ├── restart network service

Issue 4: Backup and Restore File Permissions in Linux

├── Troubleshooting
│ ├── The best option is to create the ACL file of Dir/Files 
      before changing the permissions in bulk
│ │ ├── Create the ACL file before changing the permission (or 
        backup the file permission): ~$ getfacl -R <dir> > 
        permissions.acl
│ │ ├── Restore File Permissions: ~$ setfacl -- 
        restore=permissions.acl
│ ├── Restore from the VM Snapshot (But not always a good option 
      for production)
│ ├── Rebuild the VM (this option is safe for the future)

Useful Tip-Related Disk Partition:

├── Tips
│ ├── After adding/attaching a new disk to a VM, we can get its 
      status from lsblk command by doing ~$echo 1 > 
      /sys/block/sda/device/rescan
│ ├── If we increase the disk size of the existing disk then the 
      additional space gets appended to the existing disk without 
      affecting the already existing Filesystem and Partition.
│ ├── We can also recreate the filesystem on the block device as 
      it will automatically format the old one
│ ├── If we have a disk (with created partition/FS) we can share 
      the .vmdk to another VM. So, after mounting we would have 
      the same data as it was on the previous one.

Linux Troubleshooting Scenarios - Part 2

Sagar R Ravkhande — Fri, 06 Jan 2023 09:53:22 +0000

Continuation from Part 1 of our Linux Troubleshooting Scenarios here is the Part-2 of the next scenarios to be discussed.

Issue 1: fstab file missing or bad entry

├── One of the errors that cause the system unable to BOOT UP
├── Check /var/log/messages, dmesg, and other log files
├── If we have a bad sector log, we have to run fsck
│ ├── True:
│ │ ├── reboot the system into resuce mode as booting it from 
        CDROM by applying ISO
│ │ ├── proceed with option 1, which mounts the original root 
        filesystem under/mnt/sysimage
│ │ ├── edit fstab entries or create a new file with the help of 
        blkid and reboot

Issue 2: Can’t cd to the directory even if the user has sudo privileges

├── Reasons and Resolution
│ ├── Directory does not exist
│ ├── Pathname conflict: relative vs absolute path
│ ├── Parent directory permission/ownership
│ ├── Doesn't have executable permission on the target directory
│ ├── Hidden directory

Issue 3: Can’t Create Links

├── Reasons and Resolution
│ ├── Target directory/File does not exist
│ ├── Pathname conflict: relative vs absolute path - (should be 
      complete path)
│ ├── Parent directory permission/ownership
│ ├── Target file permission/ownership - (as there should be read 
      permission)
│ ├── Hidden directory/file

Issue 4: Running Out of Memory

├── Types
│ ├── Cache (L1, L2, L3)
│ ├── RAM
│ │ ├── Usage
│ │ │ ├── #free -h
│ │ │ │ ├── Total (Total assigned memory)
│ │ │ │ ├── Used (Total actual used memory)
│ │ │ │ ├── Free (Actual free memory)
│ │ │ │ ├── Shared (Shared Memory)
│ │ │ │ ├── Buff/Cache (Pages cache memory)
│ │ │ │ ├── Available (Memory can be freed)
│ │ │ ├── /proc/meminfo
│ │ │ │ ├── file active
│ │ │ │ ├── file inactive
│ │ │ │ ├── anon active
│ │ │ │ ├── anon inactive
│ ├── Swap (Virtual Memory)
├── Resolution
│ ├── Identify the processes that are using high memory using top, 
      htop, ps etc.
│ ├── Check the OOM in logs and also check if there is a memory 
      commitment in sysctl.conf
│ ├── Kill or restart the process/service
│ ├── prioritize the process using nice
│ ├── Add/Extend the swap space
│ ├── Add more physical RAM

Issue 5: Add/ Extend the Swap Space

├── Due to running out of memory, we would need to add more swap 
    space
│ ├── Create a file with #dd, as it will reserve the blocks of 
      disk for swap file
│ ├── Set permission 600 and give root ownership
│ ├── #mkswap
│ ├── Now Turned swap on #swapon
│ ├── fstab entry for persistent

Linux Troubleshooting Scenarios - Part 1

Sagar R Ravkhande — Wed, 04 Jan 2023 08:48:08 +0000

It is always crucial to understand the issue. There should be the right approach or a step-by-step process to be followed to troubleshoot the issues. Doesn’t matter if you are a Software Developer or DevOps Engineer or Architect. Unix. /Linux is used widely, and you should be aware of the issues and the correct approach to resolve them.

Let’s discuss a few of them:

Issue 1: Server is not reachable or unable to connect

├── Ping the server by Hostname and IP Address
│ ├── Hostname/IP Address is pingable
│ │ ├── The Issue might be on the client side as the server is 
        reachable
│ ├── Hostname is not pingable but IP Address is pingable
│ │ ├── Could be the DNS issue
│ │ │ ├── check /etc/hosts
│ │ │ ├── check /etc/resolv.conf
│ │ │ ├── check /etc/nsswitch.conf
│ │ │ ├── (Optional) DNS can also be defined in the 
          /etc/sysconfig/network-scripts/ifcfg-<interface>
│ ├── Hostname/IP Address both are not pingable
│ │ ├── Check the other server on the same network to see if there 
        is it a Network side access issue or other overall 
        something bad
│ │ │ ├── False: The issue is not overall network side but with 
          that host/server
│ │ │ ├── True: Might be an overall network-side issue
│ │ ├── Logged into the server by Virtual Console, if the server 
        is Powered ON. Check the uptime
│ │ ├── Check if the server has the IP, and has UP status of the 
        Network interface
│ │ │ ├── (Optional) Also check IP-related information from
          /etc/sysconfig/network-scripts/ifcfg-<interface>
│ │ ├── Ping the gateway, also check routes
│ │ ├── Check Selinux, Firewall rules
│ │ ├── Check physical cable conn

Issue 2: Unable to connect to a website or an application

├── Ping the server by Hostname and IP Address
│ ├── False: Above Troubleshooting Diagram "Server is not 
      reachable or cannot connect"
│ ├── True: Check the service availability by using the telnet 
      command with port
│ │ ├── True: Service is running
│ │ ├── False: Service is not reachable or running
│ │ │ ├── Check the service status using systemctl or other 
          commands
│ │ │ ├── Check the firewall/selinux
│ │ │ ├── Check the service logs
│ │ │ ├── Check the service configuration

Issue 3: Unable to ssh as root or any other user.

├── Ping the server by Hostname and IP Address
│ ├── False: Above Troubleshooting Diagram "Server is not 
      reachable or cannot connect"
│ ├── True: Check the service availability by using the telnet 
      command with port
│ │ ├── True: Service is running
│ │ │ ├── Issue might be on the client side
│ │ │ ├── User might be disabled, no-login shell, disabled root 
          login and other configuration
│ │ ├── False: Service is not reachable or running
│ │ │ ├── Check the service status using systemctl or other 
          commands
│ │ │ ├── Check the firewall/selinux
│ │ │ ├── Check the service logs
│ │ │ ├── Check the service configuration

Issue 4: Disk Space is full issue or add/extend disk space

├── System Performance degradation detection
│ ├── Application getting slow/unresponsive
│ ├── Commands are not running (For Example: as / disk space is 
      full)
│ ├── Cannot do logging and other etc.
├── Analyse the issue
│ ├── df command to find the problematic filesystem space issue
├── Action
│ ├── After finding the specific filesystem, use du command in 
      that filesystem to get which files/directories are large
│ ├── Compress/remove big files
│ ├── Move the items to another partition/server
│ ├── Check the health status of the disks using badblocks command 
      (For Example, #badblocks -v /dev/sda)
│ ├── Check which process is IO Bound (using iostat)
│ ├── Create a link to file/dir
├── New disk addition
│ ├── Simple partition
│ │ ├── Add disk to VM
│ │ ├── Check the new disk with df/lsblk command
│ │ ├── fdisk to create the partition. Better to have LVM 
        partition
│ │ ├── Create filesystem and mount it
│ │ ├── fstab entry for persistent
│ ├── LVM Partition
│ │ ├── Add disk to VM
│ │ ├── Check the new disk with df/lsblk command
│ │ ├── fdisk to create LVM partition
│ │ ├── PV, VG, LV
│ │ ├── Create filesystem and mount it
│ │ ├── fstab entry for persistent
│ ├── Extend LVM partition
│ │ ├── Add disk, and create LVM partition
│ │ ├── Add LVM partition (PV) in existing VG
│ │ ├── Extend LV and resize the filesystem

Issue 5: Filesystem corrupted

├── One of the errors that cause the system unable to BOOT UP
├── Check /var/log/messages, dmesg, and other log files
├── If we have bad sector logs, we have to run fsck
│ ├── True:
│ │ ├── reboot the system into rescue mode by booting it from 
        CDROM by applying ISO
│ │ ├── proceed with option 1, which mounts the original root 
        filesystem under /mnt/sysimage.
│ │ ├── edit fstab entries or create a new file with the help of 
        blkid and reboot.

𝐒𝐎𝐂 1 𝐯𝐬. 𝐒𝐎𝐂 2 𝐯𝐬. 𝐒𝐎𝐂 3

Sagar R Ravkhande — Wed, 04 Jan 2023 08:27:07 +0000

SOC stands for Service Organization Control, and the nut of what it’s all about is summarized right there: You’re a service organization (in accountant-speak), and you need to prove that you have certain controls in place for said accountants to deem you SOC-compliant.

SOC compliance is important because most enterprises can't or won't adopt your product without it. Without SOC compliance, you can’t land the enterprise deals that make your startup sustainable.

In this article, we’re going to break down the meaning of SOC 1, SOC 2, and SOC 3, as well as the differences between all three. By the end, you’ll know which is most relevant and which is necessary, and you’ll understand how to embark on the path to compliance.

SOC 1 vs. SOC 2. vs. SOC 3: An Overview‍

TL;DR: SOC compliance demonstrates that your customers can rely on the services you provide. An accountant audits your company and certifies you with a SOC report that you supply to your customers. This report proves your trustworthiness.

However, understanding SOC compliance in greater detail is important for knowing when to get SOC compliance and which type of SOC report to get. So, let’s break it down further.

The Major differences between Soc 1 vs. SOC 2. vs. SOC 3

There are three primary types of SOC reports—the first two are the most used, and the second is of most concern to technology companies.

SOC 1 and SOC 2 are the most common SOC reports, so understanding the difference between them is essential. The difference between SOC 1 and SOC 2 is that SOC 1 focuses on financial reporting, whereas SOC 2 focuses on compliance and operations.

SOC 3 reports are less common. SOC 3 is a variation of SOC 2 and contains the same information as SOC 2, but it’s presented for a general audience rather than an informed one. If a SOC 2 report is for auditors and stakeholders inside the company you’re selling to, SOC 3 is for that company’s customers.

There are a couple of other SOC reports that are rarer and outside the scope of this article:

SOC for Cybersecurity reports on a service organization’s cybersecurity risk management effectiveness.
SOC for Supply Chain reports on the effectiveness of a service organization’s supply chain risk management.

Take a look at SOC 1, SOC 2, and SOC 3 from a higher level. Save these infographic notes to refer to when your memory of this article gets a little hazy.

Object Oriented Programming (OOP) Concepts

Sagar R Ravkhande — Fri, 11 Nov 2022 15:41:59 +0000

The high-level programming languages are broadly categorized into two categories:

Procedure-oriented programming (POP) language.
Object-oriented programming (OOP) language.

Procedure-Oriented Programming Language

In the procedure-oriented approach, the problem is viewed as a sequence of things to be done such as reading, calculation, and printing.
Procedure-oriented programming consists of writing a list of instructions or actions for the computer to follow and organizing this instruction into groups known as functions.

Characteristics of procedure-oriented programming:

Emphasis is on doing things(algorithm)
Large programs are divided into smaller programs known as functions.
Most of the functions share global data
Data move openly around the system from function to function
Function transforms data from one form to another.
Employs top-down approach in program design

The disadvantage of procedure-oriented programming languages is:

Global data access
It does not model real word problems very well
No data hiding

Object Oriented Programing Language

“Object-oriented programming is an approach that provides a way of modularizing programs by creating partitioned memory area for both data and functions that can be used as templates for creating copies of such modules on demand”.

Characteristics of Object-Oriented programming:

Emphasis is on doing rather than procedure.
programs are divided into what are known as objects.
Data structures are designed such that they characterize the objects.
Functions that operate on the data of an object are tied together in the data structure.
Data is hidden and can’t be accessed by external functions.
Objects may communicate with each other through functions.
New data and functions can be easily added.
Follows bottom-up approach in program design.

Procedure Oriented Programming (POP) Vs Object Oriented Programming (OOP)

Procedure Oriented Programming	Object Oriented Programming
Program is divided into small parts called functions.	Program is divided into parts called objects.
Importance is not given to data but to functions as well as the sequence of actions to be done.	Importance is given to the data rather than procedures or functions because it works in the real world.
Top-Down approach.	Bottom-Up approach.
It does not have any access specifier.	OOP has access specifiers named Public, Private, Protected, etc.
Data can move freely from function to function in the system.	Objects can move and communicate with each other through member functions.
Adding new data and functions in POP is not so easy.	OOP provides an easy way to add new data and functions.
Most function uses Global data for sharing that can be accessed freely from function to function in the system.	In OOP, data cannot move easily from function to function, it can be kept public or private so we can control the access of data.
It does not have any proper way of hiding data, so it is less secure.	OOP provides Data Hiding so provides more security.
Overloading is not possible.	In OOP, overloading is possible in the form of Function Overloading and Operator Overloading.
Examples of Procedure Oriented Programming are C, VB, FORTRAN, and Pascal.	Example of Object-Oriented Programming are C++, JAVA, VB.NET, C#.NET.

Object-Oriented Programming Principles

Encapsulation
Data abstraction
Polymorphism
Inheritance
Dynamic binding
Message Passing

Encapsulation

Wrapping data and functions together as a single unit is known as encapsulation. By default, data is not accessible to the outside world, and they are only accessible through the functions which are wrapped in a class. prevention of data direct access by the program is called data hiding or information hiding.

Data abstraction

Abstraction refers to the act of representing essential features without including the background details or explanation. Classes use the concept of abstraction and are defined as a list of attributes such as size, weight, cost, and functions to operate on these attributes. They encapsulate all essential properties of the object that are to be created. The attributes are called data members as they hold data and the functions which operate on these data are called member functions.
Classes use the concept of data abstraction, so they are called abstract data types (ADT).

Polymorphism

Polymorphism comes from the Greek word “poly” and “morphism”. “poly” means many and “morphism” means form i.e., many forms. Polymorphism means the ability to take more than one form. For example, an operation has different behavior in different instances. The behavior depends upon the type of data used in the operation.
Different ways to achieve polymorphism are:
1) Function overloading
2) Operator overloading

Inheritance

Inheritance is the process by which one object can acquire the properties of another.
Inheritance is the most promising concept of OOP, which helps realize the goal of constructing software from reusable parts, rather than hand-coding every system from scratch. Inheritance not only supports reuse across systems but also directly facilitates extensibility within a system. Inheritance coupled with polymorphism and dynamic binding minimizes the amount of existing code to be modified while enhancing a system.

When the class child, inherits the class parent, the class child is referred to as a derived class (sub-class) and the class parent as a base class (superclass). In this case, the class child has two parts: a derived part and an incremental part.
The derived part is inherited from the class parent. The incremental part is the new code written specifically for the class child.

Dynamic binding:

Binding refers to the linking of a procedure call to the code to be executed in response to the call. Dynamic binding (or late binding) means the code associated with a given procedure call is not known until the time of the call at run time.

Message passing:

An object-oriented program consists of a set of objects that communicate with each other. Objects communicate with each other by sending and receiving information.

A message for an object is a request for the execution of a procedure and therefore invokes the function that is called for an object and generates a result.

Key Takeaways:

OOP: Object-Oriented Programming. A programming paradigm or approach is used to analyze and solve problems that are based on the representation of real-world objects in the system.
Class: one of the building blocks of Object-Oriented Programming that acts as a “blueprint” where the data and the actions of the objects are defined.
Instance: a concrete object that is created from the class “blueprint”.
Method: an “action” defined in the class that the instances of the class can perform. It is very similar to a function but closely related to instances such that instances can call methods and methods can act on the individual data of the instances.

What is a NoSQL database?

Sagar R Ravkhande — Fri, 23 Sep 2022 20:35:03 +0000

The term NoSQL is used to describe a set of technologies for data
storage. In this section, I explain what NoSQL is, outline the
major types of NoSQL databases, and compare NoSQL to relational
databases.

Defining NoSQL

NoSQL describes technologies for data storage, but what exactly
does that mean? Is NoSQL an abbreviation for something?
Depending on whom you ask, NoSQL may stand for “not only SQL” or it may not stand for anything at all. Regardless of any 4 Redis disagreement over what NoSQL stands for, everyone agrees that
NoSQL is a robust set of technologies that enable data persistence
with the high performance necessary for today’s Internet-scale
applications.

SQL is an abbreviation for Standard Query Language, a standard
language for manipulating data within a relational database.

Identifying types of NoSQL databases

There are four major types of NoSQL databases — key/value,
column, document, and graph — and each has a particular use case for which it’s most suited.
The following sections go into greater detail on the four types of
NoSQL.

Key/value

With a key/value storage format, data uses keys, which are identifiers that are similar to a primary key in a relational database. The data element itself is then the value that corresponds to the key.

An example of a key/value pair looks like this:

"id": 123456

In this example, "id" is the key while 123456 is the value that
corresponds to that key.

Key features of the key-value store:

Simplicity.
Scalability.
Speed.

Column

With a column-oriented data store, data is arranged by column
rather than by row. The effect of this architectural design is that it makes aggregate queries over large amounts of data much faster to process.

Key features of columnar-oriented database:

Scalability.
Compression.
Very responsive.

Document

Document data storage in NoSQL uses a key as the basis for item retrieval. The key then corresponds to a more complex data structure called a document, which contains the data elements for
a given collection of data.

Key features of documents database:

Flexible schema: Documents in the database has a flexible schema. It means the documents in the database need not be the same schema.
Faster creation and maintenance: the creation of documents is easy and minimal maintenance is required once we create the document.
No foreign keys: There is no dynamic relationship between two documents so documents can be independent of one another. So, there is no requirement for a foreign key in a document database.
Open formats: To build a document we use XML, JSON, and others.

Graph

Graph databases use graph theory to store data relations in a series of vertices with edges, making queries that work with data
in such a manner much faster.

Key features of graph database:

In a graph-based database, it is easy to identify the relationship between the data by using the links.
The Query’s output is real-time results.
The speed depends upon the number of relationships among the database elements.

Comparison between NoSQL and relational databases

Regardless of the type of NoSQL database, the patterns, and tools that you use to work with data are different from the patterns and tools that you typically find with a relational database. As you just saw, the paradigm for storage and arrangement of the data typically requires a rethink of how applications are created.
Relational databases connect data elements through relations between tables. These relations become quite complex for many applications, and the resulting queries against the data become equally complex. The inherent complexity leads to performance issues for queries.

Many traditional databases include query tools and software to directly manipulate data. With NoSQL, most access will be programmatic only, through applications that you write using the tools and application programming interfaces (APIs) for the NoSQL database.

Relational databases have somewhat less flexibility than a multi-model databases such as Redis. Whereas a relational database thrives when data is consistent and well structured, Redis and NoSQL thrives on the unstructured data that is found in today’s modern applications while also providing the flexibility to structure data as needed.

Why use NoSQL Databases?

Different NoSQL Database models

6(R’) Strategies for Cloud Migration - All IN ONE

Sagar R Ravkhande — Sat, 23 Jul 2022 15:56:00 +0000

What is a Cloud Computing?

"NIST" take on cloud computing,

"The Gartner" take on cloud computing,

"Forrester" take on cloud computing,

and In general,

What are the Key Business Drivers for Cloud Adoption?

Cloud migration describes the process of moving the organization’s business applications from the on-premises data center to virtual cloud infrastructure or cloud services.

Many business factors are driving organizations to adopt a cloud migration path to realize the maximum benefits of the cloud.

Embracing Cloud migration services provides you with the following benefits:

Cloud computing allows businesses to shift their focus from infrastructure and platforms and focus on innovation at the application level.
Cloud deployments have unmatched availability, scalability, and agility of cloud resources as compared to on-premises deployments. One can achieve improved disaster recovery and business continuity by relying on an always-on infrastructure provided by the cloud service provider.
On-demand usage patterns and pay-as-you-go cost management offered by the cloud help organizations convert huge CAPEX spends to smaller chunks of OPEX.
Cloud computing is an attractive alternative to replace on-premises hardware/software components that have reached end-of-life.
Cloud computing also allows businesses to leverage services that may not be available on-premises, leading to an optimal best-of-breed hybrid architecture.
A side benefit of the move to cloud computing is that businesses can reduce environmental waste by reducing the hardware and physical products they use.

What are the Advantages of cloud computing?

Advantage #1: Trade capital expense for variable expense
Capital expenses (CAPEX) are funds that a company uses to acquire, upgrade, and maintain physical assets such as property, industrial buildings, or equipment. Do you remember the data center example in the traditional computing model where you needed to rack and stack the hardware, and then manage it all? You must pay for everything in the data center whether you use it or not. By contrast, a variable expense is an expense that the person who bears the cost can easily alter or avoid. Instead of investing heavily in data centers and servers before you know how you will use them, you can pay only when you consume resources and pay only for the amount you consume. Thus, you save money on technology. It also enables you to adapt to new applications with as much space as you need in minutes, instead of weeks or days. Maintenance is reduced, so you can spend focus more on the core goals of your business.

Advantage #2: Benefit from massive economies of scale
By using cloud computing, you can achieve a lower variable cost than you can get on your own. Because usage from hundreds of thousands of customers is aggregated in the cloud, providers such as AWS can achieve higher economies of scale, which translates into lower pay-as-you-go prices.

Advantage #3: Stop guessing capacity
Eliminate guessing about your infrastructure capacity needs. When you make a capacity decision before you deploy an application, you often either have expensive idle resources or deal with limited capacity. With cloud computing, these problems go away. You can access as much or as little as you need, and scale up and down as required with only a few minutes’ notice.

Advantage #4: Increase speed and agility
In a cloud computing environment, new IT resources are only a click away, which means that you reduce the time it takes to make those resources available to your developers from weeks to just minutes. The result is a dramatic increase in agility for the organization because the cost and time that it takes to experiment and develop are significantly lower.

Advantage #5: Stop spending money on running and maintaining data centers
Focus on projects that differentiate your business instead of focusing on the infrastructure. Cloud computing enables you to focus on your customers instead of the heavy lifting of racking, stacking, and powering servers.

Advantage #6: Go global in minutes
You can deploy your application in multiple AWS Regions around the world with just a few clicks. As a result, you can provide lower latency and a better experience for your customers simply and at a minimal cost.

A migration might consist of moving a single data center, a collection of data centers, or some other portfolio of systems that is larger than a single application.

What is Migration? Why Migrate to Cloud?

Cloud migration isn’t just about moving to the cloud, It is the process of transferring data, applications, and workloads, from On-Premises to the Cloud(AWS, Azure, GCP, etc.) It is an iterative process of optimization to reduce costs and reach the full potential of the cloud. It impacts all the organizational aspects including people, processes, and technology. But with flexible consumption and pricing models, the cloud can support high scalability, performance, agility, remote work, and cost-efficiency.
Cloud Migration is the process of transferring data, applications, and workloads, from On-Premises to the Cloud(AWS, Azure, GCP, etc.).

The decision to migrate to the cloud can be driven by several factors, including data center lease expiration, required hardware upgrades, software license renewals, location requirements to meet regulatory compliance, global market expansion, increased developer productivity, or the need for a standard architecture.
While there are several common components found in each successful migration, there is no one-size-fits-all solution to deciding on the best approach.

Why?

Operational Cost: Operational costs are the cost of running your infrastructure.
Workforce Productivity: Workforce productivity is how efficiently you can get your services to market.
Cost Avoidance: Cost Avoidance is setting up an environment that does not create unnecessary costs.
Operational Resilience: Operational resilience is reducing your organization's risk profile and the cost of risk migration.
Business agility: Ability to react quickly to changing market conditions. you can expand into new markets, and take products to market quickly.

The 6R’s of Strategies for Migration to the Cloud

There are six common cloud migration paths (also known as the six Rs) for cloud migration based on the level of cloud integration that an organization desires.

1) Refactor/Re-architect("Architected using Cloud-Native Feature.")

Definition of Refactor

Writing a new version of the existing application, with a new architecture and design in mind. In a refactor, you gain by removing any unnecessary components, leveraging newer application technologies in the cloud, and generally providing an improved user experience and performance.

This strategy involves converting a legacy monolithic application into a new, highly decoupled, and cloud-native architecture.
This is usually driven by a strong desire to improve a service or application. For highly critical applications that require cloud-native characteristics or applications that need thorough modernization due to outdatedness or performance issues a higher migration effort is typically profitable and hence should be part of cloud considerations.
It is the strategy that usually leads to the highest transformation cost and is much more complicated than other cloud migration approaches because it requires application code changes and must be tested carefully to avoid regressions in functionality. However, it allows optimized use of the cloud, leading to cloud-native benefits and making the application future-proof.
Typically this involves breaking down the application’s components into smaller building blocks, and microservices and wrapping them into (Docker) containers for deployment on a container platform.

Use Cases:

Use Refactor if,

The application will gain the most from the cloud.
There is a strong business drive to add scalability, speed, and performance.
An on-premise app is not compatible with the cloud.

2) Replatform("lift-tinker-and-shift.")

Definition of Replatform

Changing the underlying infrastructure technology that an application runs in today, as we move it to the cloud. Some application changes may be required, but not a complete refactor.

Replatforming leads to cloud optimizations to the application during the migration stage due to some cloud platform adoption, while keeping the application core architecture the same, so that does require some programming input and expertise.
Replatformed applications show some cloud-native characteristics like horizontal scaling and portability. Often, Replatforming is used when replacing database backends of applications with a corresponding PaaS database solution of a cloud provider.
For example, you might end up moving from your relational database system to a turnkey managed RDS on a cloud provider — same underlying tech, a different business model with cloud resiliency auto-added.

Use Cases

Use Replatform if you want to:

migrate with a time-crunch.
leverage the benefits of the cloud without refactoring the app migrate a complex on-premises app with minor tweaks for cloud benefits.

3) Repurchase("“drop and shop”")

Definition of Repurchase

Replacing the current system by purchasing a SaaS solution that meets the needs and requirements of the current application. Note: This can result in a data-migration and transformation project of its own.

This is sometimes referred to as “drop and shop,” as it refers to a decision to move to another product.
Repurchasing (also called Replacing) is the strategy where the legacy application is fully replaced by a SaaS solution that provides the same or similar capabilities.
The migration effort heavily depends on the requirements and options of migrating (live) data. Some SaaS replacements for on-premise products from the same vendor offer an option to quickly migrate data with little effort or even automatically. Some providers offer analysis tools to assess the to-be-expected migration effort. However, this might not be the case when switching to a product of a different vendor or if the migration path has been interrupted due to neglected maintenance of the on-premise application.

The “repurpose” strategy is often applied when using a proprietary database platform or a proprietary product and moving to something else.
An example of this is moving from an on-premises email server to AWS Simple Email Service (SES). Another example is moving the organization’s CRM to Salesforce.

Use Cases:

Use Repurchase if:

You’re replacing software for standard functions like finance, accounting, CRM, HRM, ERP, email, CMS, etc.
A legacy app is not compatible with the cloud.

4) Rehost(“lift-and-shift.”)

Definition of Rehost

Sometimes called ‘lift-and-shift’, this involves the replication of virtual machines from their current location into a cloud environment.

Rehosting is commonly referred to as lift and shift.
This strategy is a widely chosen strategy due to the relatively low migration effort and it carries the least amount of risk.
It lifts servers and applications from the on-premises infrastructure and shifts them to copied as-is to a cloud infrastructure.
The most important benefit of this strategy is migration speed because no architectural refactoring needs to be done.
Moreover, the migration can often be done automatically using a variety of lift-and-shift or so-called workload mobility tools.
There are significant benefits to running servers on the scalable, pay-as-you-go infrastructure of a cloud platform.
It’s a relatively low-resistance migration strategy, and it’s a great strategy for working backward from a fixed constraint or a hard deadline.
However, the Rehosting strategy has a major drawback. Using this approach it is not possible to exploit the cloud's entire potential since the applications are not built in a cloud-native fashion. Simply rehosted applications are, compared to cloud-native applications, not decoupled from the operating system[2] and are usually much more difficult to scale. Experience shows that from a cost perspective Rehosting usually does not lead to any major advantage.

Use Cases:

Use Rehost if you’re:

only if speed is the most important factor.
migrating a large-scale enterprise.
new to the cloud.
migrating off-the-shelf applications.
migrating with a deadline.

5) Retain(“revisit” or do nothing.)

Definition of Retain

Keeping the application as-is, in its current environment.

Retain (or Revisit) means that you do not migrate the application. Despite all the benefits of cloud technology, there are still reasons to retain some applications on-premises that are not ready to be migrated and will produce more benefits when kept on-premises, or you are not ready to prioritize an application that was recently upgraded and then make changes to it again.
For example, an application may be reaching its end-of-life soon, and you may not want to invest time and effort in migrating such an application. Retaining such applications as-is may be the right approach.

Use Cases:

Use Retain if:

You adopt a hybrid cloud model during migration.
You’re heavily invested in on-premise applications.
A legacy app is not compatible with the cloud and works well on-premise.
You decide to revisit an app later.

6) Retire("Get rid of old one.")

Definition of Retire

Getting rid of an application.

The Retire strategy means that an application is explicitly phased out.
This strategy involves identifying assets and services that can be turned off so the business can focus on services that are widely used and of immediate value.
If an application is considered not worth migrating to the cloud, it can either be eliminated or downsized. It allows you to explore all your applications in terms of their uses, dependencies, and cost to the company. It is a rather passive strategy as there is no migration.

Use Cases:

Use Retire if:

An app is redundant or obsolete.
A legacy app is not compatible with the cloud and provides no productive value anymore.
You decide to refactor or repurchase an app.

The following strategies are arranged in increasing order of complexity — this means that the time and cost required to enact the migration will be proportional to the increase, but will provide greater opportunity for optimization
Retire(simples) < Retain < Rehost < Repurchase < Replatform < Re-architect/Refactor (most complex)

Best Practices for a Successful Cloud Migration

Know your IT portfolio inside out –the data, applications, and infrastructure
Design your migration strategy
Select the right partner for your cloud migration journey
Prepare your team and the existing IT environment for the transition
Leverage automated tools and managed services from cloud services providers wherever possible
Track and monitor the migration process continuously
Test and validate for optimization

References

Connect with me at Sagar-R-Ravkhande

DEV Community: Sagar R Ravkhande

Deploy a Flask API on AWS ECS Fargate with DynamoDB and Terraform (End-to-End)

sagary2j / python-flask-hello-world

Python Flask hello-world application deployment into AWS ECS

Hello World Flask Application Deployment using AWS ECS and AWS DynamoDB

Architecture

Requirements

Deployment Architecture

1. Contents

What we're building

Architecture

Repository layout

1. The application

One app, pluggable storage

Getting the birthday maths right

Validating input properly

2. Tests that don't need AWS or a lucky date

3. A production-grade container

4. Infrastructure with Terraform

Pin everything

Fargate service with automatic rollback

Least-privilege IAM

Test the Terraform too

5. CI/CD with GitHub Actions and OIDC

No more access keys

The pipeline

6. Deploy it yourself

7. What was wrong with v1 (and what I learned)

Cost and clean-up

Conclusion

Automate Azure VM Backups with Az CLI: Vault Provisioning, Protection, and On-Demand Backup

Architecture / flow

Prerequisites

Summary

SonarQube Infrastructure Setup using AWS EC2 and PostgreSQL

Project: SonarQube Infrastructure Setup

Overview

Table of Contents

Infrastructure Setup

EC2 Instance Setup

PostgreSQL Database Setup

SonarQube Installation

Jenkins Integration

Jenkins Installation

Jenkins Configuration

Jenkins SonarQube Plugin

Pipeline Setup

Pipeline Definition (Jenkinsfile)

Triggering Code Analysis

Conclusion

Create MongoDB Atlas Cluster With Terraform and AWS

Prerequisites Needed:

Table of Contents

MongoDB Atlas API key for terraform

Configuring the MongoDB atlas provider for Terraform

Module for cluster resources

Cluster resources

Users and Roles

Network peering with atlas

Plan and Apply your terraform code

Resources

AWS IAM best practices to secure your resources...

Key Components of AWS IAM

Key features of AWS IAM

Linux Troubleshooting Scenarios - Part 3

Issue 1: Unable to Run Certain Commands

Issue 2: System Unexpectedly reboots and process restart?

Issue 3: Unable to get IP Address

Issue 4: Backup and Restore File Permissions in Linux

Useful Tip-Related Disk Partition:

Linux Troubleshooting Scenarios - Part 2

Issue 1: fstab file missing or bad entry

Issue 2: Can’t cd to the directory even if the user has sudo privileges

Issue 3: Can’t Create Links

Issue 4: Running Out of Memory

Issue 5: Add/ Extend the Swap Space

Linux Troubleshooting Scenarios - Part 1

Issue 1: Server is not reachable or unable to connect

Issue 2: Unable to connect to a website or an application

Issue 3: Unable to ssh as root or any other user.