DEV Community: sagark

📢 SOX Compliance Isn’t Just for Auditors — It Starts with DevOps

sagark — Tue, 15 Jul 2025 06:51:58 +0000

👋 Hey, I'm a DevOps Engineer at a SaaS Company

We build a B2B financial analytics product. It processes customer billing data and integrates with ERPs like NetSuite and SAP.

One day, our CTO dropped a bombshell:

"We’re going public in 18 months. Time to get SOX compliant."

My first thought?
"Wait… isn’t that for finance and auditors?"

Nope. Turns out, SOX isn’t just a legal checkbox ,it’s a mandate that touches how we write, deploy, and manage code.

Let me walk you through how I learned that and what we did.

🔍 What Even Is SOX?

SOX (Sarbanes-Oxley Act) is a U.S. law that protects shareholders from fraud.
It forces companies to ensure:

Data accuracy
Strong internal controls
Proper access management

For DevOps? It means:

🔐“Only the right people can touch the right systems at the right time — and everything must be logged.”

🎯 Where DevOps Fits Into SOX

Turns out, a huge chunk of SOX requirements land squarely on DevOps:

Requirement	DevOps Role
Code changes must be reviewed	PR workflows & branch protection
No cowboy deploys	CI/CD approvals, GitOps
No one-man root access	IAM, SSO, RBAC
Logs must be available for audits	Centralized logging & retention
DR plans must exist	Infra backup & testing

🛠️ Here’s What We Did — Step by Step

1. 🔐 Locked Down Access

We started with access.
No more "I just need prod access for a minute."

Okta SSO across GitHub, Jenkins, AWS
Role-based access (devs can’t touch prod)
Access expires automatically unless renewed
Logs pushed to Datadog for traceability

✅ Result: No shadow access. Everything is tracked.

2. 🧾 Introduced Git Discipline

We couldn’t allow code to sneak into production anymore.

All changes go through Pull Requests
2 reviewers required on financial modules
PRs must be tied to Jira tasks
No force pushes, no direct main commits

We even enforced GPG commit signing.
We treat Git like the Bible during audits.

3. 🚦 Gate the Pipeline

Deploying to production now looks like this:

PR Approved → Merged to Main → GitHub Actions → Manual Approver → ArgoCD Sync → Production

GitHub Environments require human approval before deploy
ArgoCD ensures only what’s in Git gets deployed
Every deployment is logged with SHA, author, and timestamp

We literally can't bypass this process — and that's the point.

4. 📦 Used Terraform for Everything

Infrastructure was our next target.

Every subnet, RDS, role, or bucket is in Terraform
PR-based changes with peer review
We run tfsec to catch misconfigurations
terraform plan logs get archived in S3

SOX loves this. Auditors love this. We love this.

5. 🔐 Secured Our Secrets

We ditched .env files for good.

Switched to Vault or secrets manager(aws) + OIDC from GitHub Actions
Secrets rotate automatically
Access scoped per environment
Nothing ever hits disk or logs

Secrets are no longer "tribal knowledge." They’re managed.

6. 📜 Built an Audit Trail

We feed everything into Datadog/logging tools and S3:

CloudTrail logs
GitHub audit logs
Vault/SM access logs
ArgoCD sync events

Our auditors can search "who touched X system on Y day" and find an exact answer — instantly.

7. 💾 Ran Recovery Drills

We scheduled quarterly DR tests:

Snapshots of prod databases
Full restoration to staging
Compare actual vs. expected values

We even simulate outages.
Auditors want to see you don’t just have a plan — you’ve used it.

🤯 What Surprised Me

SOX doesn’t stop innovation — it sharpens it.
Most of what SOX requires are just good DevOps practices:
- Immutable infra
- Git-based workflows
- Principle of least privilege
- Automated logging

We just had to formalize and enforce them.

✅ My Personal SOX DevOps Checklist

🔒 SSO + RBAC across all tools
📜 All infra in Terraform, all changes in PRs
🧾 Deployment via GitOps only, no manual pushes
🔐 Vault for secrets, rotated and scoped
📦 All logs centralized + retained for few years
💾 Disaster recovery tested and documented

💬 Final Thought

If your company is headed toward IPO, or you work on systems that touch finance — start implementing SOX-aligned DevOps today.

Not just for compliance — but for clarity, security, and control.

The more you automate for compliance, the more time you earn for engineering.

🙌 Let’s Talk

Have you built a compliant pipeline?
Need a Terraform/GitHub starter for SOX?
Want a checklist for your infra team?

Let’s chat — I’d love to compare notes.

🔨🤖 Built it. Secured it. Audited it.

✨ Deploying MongoDB Atlas on AWS EKS Using Terraform Like a Pro💡with automated secrets and dynamic environments

sagark — Wed, 16 Apr 2025 09:47:35 +0000

👋 Introduction

Managing database infrastructure for multiple environments (dev, staging, prod) can be a headache — especially when it comes to security, secrets, and scaling.

In this I’ll walk you through how we’ve automated the provisioning of MongoDB Atlas on top of Amazon EKS using Terraform, complete with:

Role-based access
Secure secret handling (via AWS Secrets Manager)
Dynamic environment setup (dev, stg, prod)
IP whitelisting based on your VPC setup

Terraform Project Structure

Here’s a high-level view of the files in our infrastructure setup:

.
├── data.tf
├── main.tf
├── provider.tf
├── secrets.tf
└── variables.tf

🧱 main.tf – MongoDB Cluster + Project Setup

data "mongodbatlas_roles_org_id" "mongodbatlas_roles_org_id" {}

resource "mongodbatlas_project" "mongodbatlas_project" {
  name                                             = "Mongo-${var.environment}"
  org_id                                           = data.mongodbatlas_roles_org_id.mongodbatlas_roles_org_id.org_id
  is_collect_database_specifics_statistics_enabled = true
  is_data_explorer_enabled                         = true
  is_extended_storage_sizes_enabled                = true
  is_performance_advisor_enabled                   = true
  is_realtime_performance_panel_enabled            = true
  is_schema_advisor_enabled                        = true
}

resource "mongodbatlas_cluster" "mongodbatlas_cluster" {
  project_id                     = mongodbatlas_project.mongodbatlas_project[0].id
  name                           = "${var.environment}-cluster"
  cluster_type                   = "REPLICASET"
  provider_name                  = "TENANT"
  backing_provider_name          = "AWS"
  provider_region_name           = upper(replace(var.aws_region, "-", "_"))
  provider_instance_size_name    = "M10"
  auto_scaling_disk_gb_enabled   = false
  termination_protection_enabled = false
  mongo_db_major_version         = "7.0"
}

resource "mongodbatlas_project_ip_access_list" "mongodbatlas_project_ip_access_list" {
  project_id = mongodbatlas_project.mongodbatlas_project[0].id
  cidr_block = "${data.terraform_remote_state.vpc.outputs.nat_elastic_ip}/32"
  comment    = "cidr block for AWS ${var.environment}"
}

resource "mongodbatlas_project_ip_access_list" "mongodbatlas_project_ip_access_list_prod" {
  project_id = data.mongodbatlas_project.mongodbatlas_project[0].id
  cidr_block = "${data.terraform_remote_state.vpc.outputs.nat_elastic_ip}/32"
  comment    = "cidr block for AWS ${var.environment}"
}

resource "mongodbatlas_database_user" "mongodbatlas_database_user" {
  username           = var.environment
  password           = random_password.password[0].result
  project_id         = mongodbatlas_project.mongodbatlas_project[0].id
  auth_database_name = "admin"

  roles {
    role_name     = "readWriteAnyDatabase"
    database_name = "admin"
  }

  scopes {
    name = mongodbatlas_cluster.mongodbatlas_cluster[0].name
    type = "CLUSTER"
  }
}

🧾 In this setup, we automate the creation of a fully managed MongoDB Atlas environment using Terraform for the stage of our application. The Terraform script provisions the following resources:

A MongoDB Atlas Project inside the organization's Atlas account.

A MongoDB Cluster with a REPLICASET architecture, running MongoDB version 7.0, hosted on AWS with an M10 instance size—ideal for development workloads.

A Project IP Access List that whitelists the NAT Gateway IP of the AWS VPC to ensure secure and controlled access to the cluster from within our infrastructure.

A Database User with the same name as the environment , with readWriteAnyDatabase permissions scoped to the cluster, enabling it to interact with any database within that cluster securely.

This setup is part of our Infrastructure-as-Code (IaC) strategy to maintain consistency, improve security, and reduce manual operations during the lifecycle of our environment.

🔐 secrets.tf – MongoDB Secrets in AWS Secrets Manager

data "aws_secretsmanager_secret" "aws_secretsmanager_secret" {
  name = "mongo"
}

data "aws_secretsmanager_secret_version" "aws_secretsmanager_secret_version" {
  secret_id = data.aws_secretsmanager_secret.aws_secretsmanager_secret.id
}

resource "aws_secretsmanager_secret" "mongo" {
  name                    = "/${var.environment}/mongo"
  recovery_window_in_days = 0
}

resource "aws_secretsmanager_secret_version" "mongo" {
  secret_id     = aws_secretsmanager_secret.mongo[0].id
  secret_string = <<EOF
   {
    "MONGO_URL"             : "mongodb+srv://${mongodbatlas_database_user.mongodbatlas_database_user[0].username}:${mongodbatlas_database_user.mongodbatlas_database_user[0].password}@${trimprefix(mongodbatlas_cluster.mongodbatlas_cluster[0].connection_strings[0].standard_srv, "mongodb+srv://")}/${var.environment}?authSource=${mongodbatlas_database_user.mongodbatlas_database_user[0].auth_database_name}"
   }
EOF
}


data "aws_secretsmanager_secret" "aws_secretsmanager_secret_beta" {
  name  = "/beta/mongo"
}

data "aws_secretsmanager_secret_version" "aws_secretsmanager_secret_version_beta" {
  secret_id = data.aws_secretsmanager_secret.aws_secretsmanager_secret_beta[0].id
}

This secret is then pulled securely by our app pods using Kubernetes External Secrets or the AWS Secrets CSI driver.

⚙️ provider.tf – AWS & MongoDB Atlas Providers

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
    mongodbatlas = {
      source = "mongodb/mongodbatlas"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

provider "mongodbatlas" {
  "MONGO_PUBLIC_KEY": "your_mongodb_atlas_public_key",
  "MONGO_PRIVATE_KEY": "your_mongodb_atlas_private_key"
}

With this provider.tf setup, you can automate both cloud infrastructure and MongoDB Atlas resources in your development workflow. Using Terraform helps you stay consistent, reduce manual errors, and scale your deployments with ease.

🔑 data.tf – Account Info + Random Password

data "aws_caller_identity" "caller" {}

resource "random_password" "password" {
  length      = 25
  upper       = true
  special     = false
  min_numeric = 5
}

}

Terraform generates a secure password for the MongoDB user that we store in Secrets Manager along with the URI.

✅ Final output.tf File

output "mongodb_endpoint" {
  value = "mongodb+srv://${mongodbatlas_database_user.mongodbatlas_database_user[0].username}:${mongodbatlas_database_user.mongodbatlas_database_user[0].password}@${trimprefix(mongodbatlas_cluster.mongodbatlas_cluster[0].connection_strings[0].standard_srv, "mongodb+srv://")}/${var.environment}?authSource=${mongodbatlas_database_user.mongodbatlas_database_user[0].auth_database_name}"
}

📝 Description
This output block dynamically generates the MongoDB connection string depending on the environment

✅ Conclusion:

A Scalable, Secure, and Dev-Friendly MongoDB Setup on EKS
With this Terraform-driven approach, we’ve not only provisioned MongoDB Atlas clusters directly from our infrastructure code, but we’ve also implemented environment-specific behavior, secure secret management via AWS Secrets Manager, and seamless integration with our EKS-based platform.

This setup ensures:

🔐 Security-first practices for handling sensitive data

🔁 Reusability and scalability across environments (dev, stg, prod)

⚙️ Automation of cloud-native MongoDB provisioning with zero manual steps

☁️ Tight integration between MongoDB Atlas, AWS, and Kubernetes (EKS)

Whether you're spinning up a new feature branch or deploying production-grade infrastructure, this solution gives your team a solid and secure foundation to build, test, and scale apps faster.

[Boost]

sagark — Tue, 18 Mar 2025 07:17:56 +0000

Deploying AWS EKS with ALB Using Terraform

sagark ・ Mar 11

#aws #kubernetes #terraform #devops

How to Set Up AWS EFS with Persistent Volumes & PVCs Using Terraform

sagark — Tue, 18 Mar 2025 07:16:16 +0000

Introduction
When deploying applications on AWS, managing persistent storage is crucial for stateful workloads. Amazon Elastic File System (EFS) provides scalable, shared file storage that integrates seamlessly with AWS services like Lambda, Kubernetes (EKS), and EC2.

In this guide, we’ll use Terraform to:

Create an AWS EFS file system
Configure security groups for secure access
Set up EFS mount targets for private subnets
Define access points for applications
Integrate with Kubernetes Persistent Volumes (PV) and Persistent Volume Claims (PVC)
Output essential EFS parameters for further integrations
By the end of this tutorial, you'll have a fully functional EFS setup, ready to be used in a Kubernetes cluster or other AWS environments.

Step 1: Create an AWS Security Group for EFS
The security group defines access rules for EFS connections from private subnets.

resource "aws_security_group" "efs_sg" {
  name   = "efs-app-${var.environment}-sg"
  vpc_id = data.terraform_remote_state.vpc.outputs.vpc_id

  ingress {
    description = "Allow access from Lambda & VPC subnets"
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = concat(
      data.terraform_remote_state.vpc.outputs.private_subnets_cidr,
      data.terraform_remote_state.vpc.outputs.db_subnets_cidr
    )
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

✅ Why?

Ensures secure access to EFS from private subnets.
Outbound connections are unrestricted for seamless communication.

Step 2: Create AWS EFS File System
This block provisions an encrypted EFS instance with lifecycle policies to optimize costs.

resource "aws_efs_file_system" "aws_efs_file_system" {
  creation_token = "${var.environment}-efs-app"
  encrypted      = true

  lifecycle_policy {
    transition_to_ia = "AFTER_14_DAYS"
  }

  lifecycle_policy {
    transition_to_primary_storage_class = "AFTER_1_ACCESS"
  }

  tags = {
    Name = "${var.environment}-efs-app"
  }
}

✅ Why?

Uses lifecycle policies to reduce storage costs.
Encryption enabled for data security.

Step 3: Create EFS Mount Targets
Mount targets enable EC2 instances, Kubernetes pods, and Lambda functions to access EFS.

resource "aws_efs_mount_target" "efs_mt" {
  count           = length(data.terraform_remote_state.vpc.outputs.db_subnets)
  file_system_id  = aws_efs_file_system.aws_efs_file_system.id
  subnet_id       = data.terraform_remote_state.vpc.outputs.db_subnets[count.index]
  security_groups = [aws_security_group.efs_sg.id]
}

✅ Why?

Creates mount targets in each private subnet.
Ensures secure communication via the security group.

Step 4: Create an EFS Access Point
Access points define specific access rules for applications using EFS.

resource "aws_efs_access_point" "aws_efs_access_point" {
  file_system_id = aws_efs_file_system.aws_efs_file_system.id
  posix_user {
    gid = 1000
    uid = 1000
  }
  root_directory {
    path = "/app"
    creation_info {
      owner_gid   = 1000
      owner_uid   = 1000
      permissions = "777"
    }
  }
}

✅ Why?

Defines POSIX permissions for secure file access.
Ensures controlled multi-user access to the file system.

Step 5: Kubernetes Integration with PV & PVC
To use EFS as a Persistent Volume (PV) in Kubernetes, define a PersistentVolume (PV) and a PersistentVolumeClaim (PVC) using the kubectl_manifest resource.

Persistent Volume (PV)
Create a file (for example, pv-pvc.tf) with the following code:

resource "kubectl_manifest" "persistent_volume" {
  yaml_body = <<YAML
apiVersion: v1
kind: PersistentVolume
metadata:
  name: var.name
spec:
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 10Gi
  csi:
    driver: efs.csi.aws.com
    volumeHandle: ${data.terraform_remote_state.efs.outputs.efs_id}::${data.terraform_remote_state.efs.outputs.efs_access_point_id}
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  volumeMode: Filesystem
YAML
}

Persistent Volume Claim (PVC)

resource "kubectl_manifest" "persistent_volume_claims" {
  depends_on = [kubectl_manifest.persistent_volume]
  yaml_body  = <<YAML
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: var.name-claim
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 10Gi
YAML
}

output.tf

output "efs_dns_name" {
  value = aws_efs_file_system.aws_efs_file_system.dns_name
}

output "efs_id" {
  value = aws_efs_file_system.aws_efs_file_system.id
}

output "efs_access_point_id" {
  value = aws_efs_access_point.aws_efs_access_point.id
}

output "efs_id_app2" {
  value = aws_efs_file_system.aws_efs_file_system_app2.id
}

output "efs_access_point_app2_id" {
  value = aws_efs_access_point.aws_efs_access_point_app2.id
}

✅ Why?

The output blocks expose essential EFS parameters, which can be referenced by other Terraform modules or Kubernetes manifests.
The Persistent Volume (PV) uses the AWS EFS CSI driver to connect to your EFS file system.
The Persistent Volume Claim (PVC) lets Kubernetes applications request storage dynamically.

Conclusion
With this complete setup, you now have a scalable, secure, and high-availability persistent storage solution using AWS EFS and Terraform. This guide not only provisions the necessary AWS infrastructure but also integrates with Kubernetes to manage persistent storage using PVs and PVCs.

Next Steps
Experiment with dynamic scaling by integrating AWS Auto Scaling or EFS Intelligent-Tiering.
Continue exploring Terraform modules to further streamline your infrastructure as code.
📌 Have questions or suggestions? Drop a comment below! 🚀

Deploying AWS EKS with ALB Using Terraform

sagark — Tue, 11 Mar 2025 07:14:50 +0000

🔥 Introduction
In cloud-native architectures, Amazon EKS (Elastic Kubernetes Service) is a powerful way to manage containerized applications at scale. When combined with AWS ALB (Application Load Balancer), it ensures seamless traffic management, automatic scaling, and security.

In this guide, I'll walk you through setting up an EKS cluster and deploying an ALB Ingress Controller using Terraform.

📌 Why Use ALB with EKS?
Amazon's Application Load Balancer (ALB) integrates well with Kubernetes to:
✅ Distribute traffic efficiently across multiple pods
✅ Enable SSL/TLS termination for security
✅ Support path-based & host-based routing
✅ Improve scalability with auto-healing features

🛠️ Tech Stack
Terraform – Infrastructure as Code
AWS EKS – Kubernetes Cluster
AWS ALB – Load Balancer
IAM Roles & Policies – Secure Access
Helm – Package Manager for Kubernetes

🚀 Step 1: Setting Up AWS EKS with Terraform
First, define the EKS cluster in Terraform:

resource "aws_eks_cluster" "eks" {
  name     = "my-eks-cluster"
  role_arn = aws_iam_role.eks_role.arn

  vpc_config {
    subnet_ids = [aws_subnet.public_1.id, aws_subnet.public_2.id]
  }
}

Create IAM Role for the ALB Controller:

resource "aws_iam_role" "alb_controller" {
  name = "alb-controller-role"
  assume_role_policy = jsonencode({
    Statement = [{
      Effect = "Allow"
      Principal = {
        Service = "eks.amazonaws.com"
      }
      Action = "sts:AssumeRole"
    }]
  })
}

🚀 Step 2: Deploy ALB Ingress Controller
Once EKS is up and running, install the ALB Ingress Controller using Helm:

helm repo add eks https://aws.github.io/eks-charts

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  --set clusterName=my-eks-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=alb-controller \
  -n kube-system

🚀 Step 3: Define Ingress Rules
Now, create an Ingress resource to route traffic via ALB:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
  - host: my-app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80

Apply the changes:

kubectl apply -f ingress.yaml

✅ Verify Deployment
Check if the ALB is created:

kubectl get ingress -A

Head over to AWS Console → EC2 → Load Balancers and verify the ALB instance.

🎯 Conclusion
With Terraform and Helm, deploying EKS with ALB is now streamlined and automated. This setup ensures a highly scalable, secure, and manageable cloud-native architecture.

If you found this guide helpful, drop a comment or share your experience! 🚀