DEV Community: waqas_ahmed01

Securely Encrypting Secrets Using Open Source Tools: SOPS and AGE

waqas_ahmed01 — Sun, 21 May 2023 14:32:52 +0000

Introduction

In today's world, data security is of utmost importance, especially when dealing with sensitive information like passwords, API keys, and other confidential data. Encrypting secrets ensures that even if unauthorized individuals gain access to the data, they won't be able to decipher its contents. In this blog post, we will explore two powerful open-source tools, SOPS and AGE, that enable secure encryption of secrets.

Installing SOPS and AGE:

Before we dive into encrypting secrets, let's ensure that we have SOPS and AGE installed on our system. You can download and install SOPS from the official GitHub repository.

SOPS → https://github.com/mozilla/sops/releases

Similarly, AGE can be installed from its GitHub repository

AGE → https://github.com/FiloSottile/age/releases

Creating the Encryption Key:

To get started, we need to generate an encryption key using AGE. Open your terminal and execute the following command:

age-keygen -o key.txt

the output of this command will be

age-keygen -o key.txt
Public key: age1rua2rfy0uhzywprgwclavsp39uhfwmrxpanutt4y3zfcjurjs3msa0hnu9

This will create an encryption key file named "key.txt". Next, copy this file to the location ~/.sops/key.txt. You can do this by running the following command:

cp key.txt ~/.sops/key.txt

Configuring SOPS Environment:

To configure SOPS to use the AGE encryption key, we need to make an entry in our shell configuration file. If you're using the Zsh shell, you can open the configuration file using the following command:

nano ~/.zshrc

add following line to the .zshrc file

export SOPS_AGE_KEY_FILE=~/$HOME/.sops/key.txt

Save the file and exit the editor. This configures SOPS to use the AGE encryption key file we generated earlier.

Encrypting the Secret.yaml File:

Now, let's encrypt the "secret.yaml" file that contains the secrets we want to protect. Here is the content of the "secret.yaml" file:

apiVersion: v1
kind: Secret
metadata:
    creationTimestamp: null
    name: dev-db-secret
data:
    username: root
    password: supersecretpassword

To encrypt this file, run the following command in your terminal:

sops --encrypt --age $(cat $SOPS_AGE_KEY_FILE | grep -oP "public key: \K(.*)") --encrypted-regex '^(data|stringData)$' --in-place ./secret.yaml

This command uses SOPS along with the AGE encryption key to encrypt the file in-place. The --encrypted-regex option specifies the fields that should be encrypted (in this case, all fields under data and stringData).
Now the secret file has been encrypted as

apiVersion: v1
kind: Secret
metadata:
    creationTimestamp: null
    name: dev-db-secret
data:
    username: ENC[AES256_GCM,data:hW4VXQ==,iv:nkM9UHvHwTx6oUvjcfq/olO/FcuijHvrVmJZfT2eB6k=,tag:pBV7nvNbGSauOCSy5Bar4Q==,type:str]
    password: ENC[AES256_GCM,data:QX7Bb5Idlyf+0sVsTbRaQd4afQ==,iv:VHu8vnW5vfSW7c4fuBNUAznhH+j2QTfij6iPFd9ww0U=,tag:RSd8e6JNbPhGuYFqOcHNAg==,type:str]
sops:
    kms: []
    gcp_kms: []
    azure_kv: []
    hc_vault: []
    age:
        - recipient: age1w6mnsqrank3f3e9rxv6xz4nnpnvrr9zyed2zsm8jkyya8gq5zazqzt58sm
          enc: |
            -----BEGIN AGE ENCRYPTED FILE-----
            YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBUdzhOQjV0VlBjT0cvSUtk
            VlVBcU11RG5DRkNWTzROdlNrVHo2bUhVK0NnCmNvR3hJSjY5VHFoMm5Va2ViMFho
            b1k1MFhaM0hOa2p1ODh0R25Vb3NsOWsKLS0tIHZSdy93MVhtZGlwL2M5UktpSDds
            UkdHa0VqcTl3TGM0MXpzMXlJeEJrdUEKdVQmdzWWndJQ1V3WZjgIEB5vQXPM5QfZ
            zv7WhnpN0gHMn2G8oZYbSmIPPT0UFI7+JaySZ5EkZeP/vqcK1Qhmow==
            -----END AGE ENCRYPTED FILE-----
    lastmodified: "2023-05-21T14:20:25Z"
    mac: ENC[AES256_GCM,data:kAr8Meo9jeMvfAgiMwhSWIVwaLVd7sU9XHck51hgA67qenE6ORlm2yZcZ75LWo1WkTGoZ+sUdByyYKMFR+zc2SHTT9fnYtLrREtBv9xHz6Kbn/rOEDGDmCNQcBLhQbPdRjzA67rrA8M0V337IJYiIywID2ur8OSXlOSF2M2vW8I=,iv:ZKLCPfpJBcLr8oG/sIVqLbZqd74UMKpS9+YSgQdDsy8=,tag:p6WgJJ3DsCdNbZB9coolhg==,type:str]
    pgp: []
    encrypted_regex: ^(data|stringData)$
    version: 3.7.3

Decrypting the Secret.yaml File:

If you need to access the decrypted contents of the "secret.yaml" file, you can use the following command:

sops --decrypt --age $(cat $SOPS_AGE_KEY_FILE | grep -oP "public key: \K(.*)") --encrypted-regex '^(data|stringData)$' --in-place ./secret.yaml

This command will decrypt the encrypted fields in the file, allowing you to view and modify the secrets as needed.

Conclusion:

Encrypting secrets is crucial for maintaining data security. In this blog post, we explored the usage of two open-source tools, SOPS and AGE, to encrypt and decrypt secrets. By following the steps outlined, you can effectively protect sensitive information and ensure its confidentiality. Remember to always store your encryption keys securely and follow best practices for secret management to maintain a robust security posture in your projects.

Youtube --> YoutubeVideo

Ingest VPC Flow Logs into NewRelic

waqas_ahmed01 — Tue, 11 Oct 2022 18:59:25 +0000

There are many use cases where we wanted to monitor the VPC Flow Logs to view the data going IN / OUT into our VPC. These network traces helps us to troubleshoot many network-related issues.

We do have a choice in AWS to save VPC Flow Log either into

AWS CloudWatch or
AWS S3 Buckets.

However, both of these solutions don't provide a good user-friendly view and can become cumbersome when trying to find a specific IP Address or Port.

Well thanks to Kinesis Data Firehose to provide us pretty much option to cope up these situation. We can ingest the data from many possible AWS services into Kinesis Data Firehose and send that to 3rd party monitoring solution to create some AWSome custom Dashboard and monitor the logs.

I will walk you through step by step to configure this solution in this blog. We can divide this into 3 parts

Create Kinesis Data Firehose
Create the VPC Flow Logs
Transform the Log using Lambda function (Optional)
Send the Logs to NewRelic Monitoring Solution

Create Kinesis Data Firehose

Create a Kinesis Data firehouse and select Source as Direct PUT and Destination as New Relic. Please note that Kinesis Data Firehose is near real time solution but not the real time solution as Kinesis.

Under Destination Setting - Select HTTP Endpoint URL as NEW Relic Log - US. Enter the API KEY (Copy the API Key form New Relic)

Click on following URL, this will land you to NewRelic API-Key screen, as shown below

https://one.newrelic.com/admin-portal/api-keys/home?

Create VPC Flow Log

Go to VPC --> Action and click on Create flow log

Under Filter, select weather you only want to monitor the ACCEPTED Traffic, REJECTED Traffic or ALL Traffic.

Under Destination, select Send to Kinesis Data Firehose in same account and select the Kinesis Data Firehose

This will take few seconds and then you will start seeing the data into NewRelic platform

If you like this article then don't forget to hit the like and share with others :)

Secure the S3 Bucket with MFA

waqas_ahmed01 — Fri, 07 Oct 2022 18:24:41 +0000

Do you know that you can secure your S3 Bucket by integrating the MFA to avoid any object deletion accidently?

The answer is Yes...!!
You can enable the MFA on S3 bucket but first you will need to enable the versioning on the bucket. Also the MFA can't be enable via AWS Management Console so either use AWS CLI or AWS SDK to enable MFA. In this article, I will walk you though the step by step instruction to enable MFA.

Step - 1: Configure MFA Device

AWS Support multiple types of MFA device both physical hardware on virtual. In this blog, we will configure virtual MFA

Login in your AWS Account, on right top click on Avatar and select Security Credentials , select the first option Authentication App

I will be using Twillo Authy app for authentication. Generate the secret key and enter into Authy app to configure new account, as shown in figure below

Once configure it'll show virtual device under the MFA on AWS Console

Step - 2: Enable the Versioning on S3 Bucket

If versioning is not enable on S3 bucket make sure to enable that before enabling the MFA. We will use AWS CLI to configure the Versioning.

aws s3api put-bucket-versioning --bucket <bucket_name> --versioning-configuration Status=Enabled

Step - 3: Enable the MFA

We will be using following AWS CLI command to enable versioning.

aws s3api put-bucket-versioning --bucket <bucket_name>--versioning-configuration Status=Enabled,MFADelete=Enabled --mfa "arn:aws:iam::<>:mfa/root-account-mfa-device Passcode"

MFA Serial can be found into AWS Console

tarrahhhh! Congratulation, you have configured the MFA for S3 Bucket.

If you like this article than don't forget to share it with others ;)

Create the Jenkins Server in AWS using Terraform

waqas_ahmed01 — Sun, 04 Sep 2022 11:40:24 +0000

In this article, we will learn how to setup the Jenkins Server using the power of Terraform.
There are number of ways to setup the Jenkins Server but that took some efforts and time to setup that. We can use Terraform to setup the infrastructure and configure the Jenkins Server without knowing any code complexity

Pre-requisite:

Basic knowledge of Terraform
Terraform should be installed on your machine
An Active AWS account to be used in this lab

Terraform Configuration

Provider

In the provider block, specify the name of provider (in our case its AWS) and region where you want to configure the infrastructure
Optional - you can provide the Access_KEY & SECRET_KEY inside the block code but it's not recommended to provide the keys in the code

provider "aws" {
  region = "us-east-1"
}

Networking

In this lab we will be setting up our own VPC, Subnets, NAT Gateway, Route Table etc

# Internet gateway to reach the internet
resource "aws_vpc" "web_vpc" {
  cidr_block = var.network_cidr
}

resource "aws_internet_gateway" "web_igw" {
  vpc_id = aws_vpc.web_vpc.id
}
# Route table with a route to the internet
resource "aws_route_table" "public_rt" {
  vpc_id = aws_vpc.web_vpc.id
  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.web_igw.id
  }
  #   tags {
  #     Name = "Public Subnet Route Table"
  #   }
}
# Subnets with routes to the internet
resource "aws_subnet" "public_subnet" {
  # Use the count meta-parameter to create multiple copies
  count             = 2
  vpc_id            = aws_vpc.web_vpc.id
  cidr_block        = cidrsubnet(var.network_cidr, 2, count.index + 2)
  availability_zone = element(var.availability_zones, count.index)
  #   tags =  {
  #     Name = "Public Subnet ${count.index + 1}"
  #   }
}

# Associate public route table with the public subnets
resource "aws_route_table_association" "public_subnet_rta" {
  count          = 2
  subnet_id      = aws_subnet.public_subnet.*.id[count.index]
  route_table_id = aws_route_table.public_rt.id
}

We will be using Amazon Linux-2 AMI, instead of hardcoding the code with the AMI id, we will take help of DataSource block to retrieve the value of AMI from AWS

data "aws_ami" "AmazonLinux2" {
  most_recent = true

  filter {
    name   = "name"
    values = ["amzn2-ami-kernel-5.10-hvm-2.0.20220805.0-x86_64-gp2"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["137112412989"] # Canonical
}

Now it's time to create the EC2 and configure the Jenkins Sever, however one last piece of the puzzle here. I want to expose my Jenkins Server on port 8080 and expose the InitialAdminPassword to configure Jenkins Server on port 80 so we can retrieve that information without login into the EC2 instance. For this we will have to create the Security Group and allow both of these ports

resource "aws_security_group" "jenkins-sg" {
  name        = "jenkins-sg-12345678"
  description = "Allow incoming HTTP traffic from the internet"
  vpc_id      = aws_vpc.web_vpc.id
  ingress {
    from_port = 8080
    to_port   = 8080
    protocol  = "tcp"
    #cidr_blocks = ["0.0.0.0/0"]
  }
  ingress {
    from_port = 80
    to_port   = 80
    protocol  = "tcp"
    #cidr_blocks = ["0.0.0.0/0"]
  }
  # Allow all outbound traffic
  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

So now we are ready to create the EC2 instance and configure the Jenkins Server. I will be using the bash script and pass that in user data using the file("path") function

#!/bin/bash
            yum install update -y
            sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo
            sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io.key
            sudo yum install fontconfig java-11-openjdk
            sudo  yum install jenkins
            sudo systemctl start jenkins
            sudo amazon-linux-extras install nginx1 -y
            sudo systemctl start nginx
            cp /var/lib/jenkins/secrets/initialAdminPassword /var/www/html/index.html
            sudo amazon-linux-extras install java-openjdk11
            chown -R root:root /var/lib/jenkins 
            chown -R root:root /var/cache/jenkins 
            chown -R root:root /var/log/jenkins
            sudo systemctl restart docker.service
            sed -i 's/JENKINS_USER="jenkins"/JENKINS_USER="root"/' /etc/sysconfig/jenkins
            sudo systemctl restart jenkins

Save this Bash script file as install_jenkins.sh

Configure Jenkins Server

resource "aws_instance" "jenkins" {
  ami                         = data.aws_ami.AmazonLinux2.id
  instance_type               = var.ec2_instance_type
  vpc_security_group_ids      = [aws_security_group.jenkins-sg.id]
  subnet_id                   = aws_subnet.public_subnet[0].id
  associate_public_ip_address = true
  user_data                   = file("install_jenkins.sh")
  root_block_device {
    delete_on_termination = true
    volume_size           = "20"
  }
  tags = {
    Name        = "Jenkins_Server"
    Environment = "Dev"
  }
}

Variables Block

To make our code more dynamic, we use couple of variables above so you will have to define the variable.tf file and declare those variable

variable "ec2_instance_type" {
  type        = string
  default     = "t2.micro"
  description = "Please enter the instance type, if you want to provision different than T2 Micro"
}

variable "service_ports" {
  type        = list(number)
  description = "list of ingress ports"
  default     = [8080, 80]
}

variable "network_cidr" {
  default     = "192.168.0.0/24"
  description = "Please enter the CIDR"
}

variable "availability_zones" {
  type        = list(any)
  default     = ["us-east-1a", "us-east-1d"]
  description = "Please enter the AZs"
}

Ola, we have successfully configured the Jenkins Server without going to the AWS Console.

In order to retrieve the IP address of newly created instance, we will use the output { } block.

Output

To make the output more menaningful I'm using a temp file and will display output in more customize way. Save the following code as outputtemp.tpl

%{for ports in port ~}
    backend = ${ip_addr}:${ports}
%{endfor ~}

Now save the following to output.tf. The output will get the value form Port list and using for loop, will print Jenkins Sever IP and nginx IP on separate line

output "IPAddress" {
  value = templatefile("outputtemp.tpl", { port = ["8080", "80"], ip_addr = aws_instance.jenkins.public_ip })
}

We have successfully setup the Jenkins Sever. To destroy the above created infrastructure

terraform destroy --auto-approve

Thank you for reading my article, if you like this article then don't forget to share it with others. In case you have any question or suggestion please left the comments below, I would love to answer your queries

You can follow me on
LinkedIn
GitHub
Terraform