DEV Community: CodeHiRise

Going Production on Cloud: Infrastructure Best Practices Every Engineer Should Know (Part 1)

CodeHiRise — Thu, 07 May 2026 18:40:38 +0000

Hi everyone,

So you have been tinkering on a new project for a while, the demos look great, and now it is time to ship to real customers. It is an exciting moment, but also one where small oversights can turn into expensive incidents at 2 AM.

This two-part series covers the practical checklist I wish every team went through before flipping the switch to production. Most recommendations are cloud-agnostic, although a few are AWS-specific. The underlying principles apply equally to GCP, Azure, or any other cloud provider.

In this first part we focus on the infrastructure foundations: environments, networking, compute, configuration, and deployment strategy. The second part will cover security, monitoring, logging, and operational work.

Let's get started.

1. Always Maintain a Production-Like Non-Production Environment

This sounds obvious, but it is one of the most commonly skipped fundamentals. Always maintain at least one non-production environment (usually staging or qa) where releases are tested before they reach production.

The catch is that this environment should genuinely mirror production. That means:

The same services and dependencies.
The same versions of every runtime, library, and managed service.
The same network topology (VPCs, subnets, security groups).
The same authentication and IAM patterns.

If your staging uses PostgreSQL 15 but production runs PostgreSQL 14, or staging talks to a different message broker version, your "tested" release can still fail in production. Smaller instance sizes are perfectly fine for cost reasons; mismatched versions are not.

A scaled-down staging environment is usually enough. You don't need the same number of replicas, just the same configurations.

2. Segregate Your Network with Public and Private Subnets

A common mistake on new cloud projects is putting everything in a single public subnet because it "just works." It works, until it doesn't.

The standard pattern is straightforward.

A public subnet holds resources that must be reachable from the internet, such as load balancers, NAT gateways, and bastion hosts. A private subnet holds everything else: application servers, databases, internal services.

Resources in the private subnet have no direct route from the internet, which dramatically reduces the attack surface. If an attacker cannot reach your database, it gets much harder them to exploit it.

You might argue that exposing a database publicly is still safe when access is tightly controlled through security groups. While that is technically true, security often fails at the edges of human error. A single misconfigured security group rule can unintentionally expose the database to the internet.

Keeping the database in a private subnet adds an additional layer of protection. Even if a security group is accidentally misconfigured, the database still remains unreachable from the outside world. This design reduces the likelihood of accidental exposure and makes costly mistakes much harder to make.

Internet
   │
   ▼
[Load Balancer]   ── public subnet
   │
   ▼
[Application]     ── private subnet
   │
   ▼
[Database]        ── private subnet (often a separate one)

For outbound access from private subnets (calling third-party APIs, pulling container images, downloading OS updates), use a NAT gateway placed in the public subnet.

For high availability, deploy a NAT gateway in each Availability Zone you use. A single NAT in one AZ becomes a cross-AZ single point of failure.

3. Control Your Ingress and Egress IP Addresses

Once your application is in production, you will inevitably need to integrate with third parties. Many of them, particularly payment providers, require you to provide a fixed list of IP addresses to whitelist on their firewall. If your egress IPs change unpredictably, those integrations break.

This is where a NAT gateway with a static Elastic IP comes in. All outbound traffic from your private subnet exits through that single, predictable address (or a small set of addresses for high availability).

The same logic applies to ingress. If a partner organization controls your DNS but does not control your infrastructure, they will need stable IPs for the records they manage.
Also, if your users are behind a corporate firewall and ask for a list of IPs to whitelist, the same situation can arise.

Auto-provisioned load balancer IPs that change on recreation are a frequent source of friction. Wherever possible, allocate static public IPs (Elastic IPs in AWS, reserved IPs in GCP) and assign them deliberately.

Always allocate static IPs early. Retrofitting them after a third-party integration is already pointing at a dynamic address is painful.

4. Choose the Right Compute Type for Each Workload

Spot instances and preemptible VMs are tempting because the discount can reach 70 to 90 percent. However, the cloud provider can reclaim them at any time, often with only a couple of minutes of warning.

A simple rule of thumb:

Stateful workloads (databases, primary application servers holding session state, anything you cannot kill mid-task) should never run on spot.
Stateless but critical workloads such as your main API serving customer traffic should stick to on-demand or reserved instances.
Batch jobs, build runners, ML training, queue workers, and ephemeral dev environments are excellent candidates for spot, because the workload can be interrupted and resumed safely. I have personally seen a team get burned by this. They moved their main production fleet to spot to save money, hit a regional capacity shortage, and lost a chunk of their fleet within minutes. The cost savings did not come close to covering the downtime.

Use spot for what spot is good at: interruptible, idempotent work. Treat your customer-facing serving capacity as reserved infrastructure.

5. Never Hardcode Environment-Specific Values

This one shows up most often with cloud resource identifiers. The classic example is the S3 bucket name (or GCS bucket, or Azure blob container).

A bad pattern looks like this: storing the full path including the bucket name in your database alongside every object reference. Six months later you need to migrate to a different region, or split workloads into a separate account, and now you have to backfill millions of database rows.

The better pattern:

Store only the object key (path) in the database.
Store the bucket name as an environment variable read by the application. When you migrate, you change one environment variable and redeploy. The database stays untouched.

// bad - bucket name baked into stored data
const objectUrl = "s3://my-prod-bucket-2026/uploads/user-123/avatar.png";

// good - bucket comes from config, only the key is persisted
const bucket = process.env.UPLOADS_BUCKET;       // e.g. "my-prod-bucket-2026"
const objectKey = "uploads/user-123/avatar.png"; // stored in DB
const objectUrl = `s3://${bucket}/${objectKey}`; // construct the full object URL dynamically

The same principle applies to API endpoints, region names, queue URLs, and any other value that is environment-dependent. Configuration belongs in environment variables or a configuration service, not in code or in your database.

6. Manage Infrastructure as Code

Clicking through a cloud console to provision resources is fine for experimenting. It is not fine for production. Every manual change is a snowflake a configuration that exists only in that environment, undocumented, unrepeatable, and invisible until it causes a problem.

Use an IaC tool like Terraform to define every resource declaratively, and a configuration management tool like Ansible for anything that runs on the instance itself.

In practice, this means your all your cloud resources are checked into version control alongside your application code. Every change goes through a pull request, gets reviewed, and is applied via a pipeline not by someone running commands in a terminal. Your staging environment is defined by the same Terraform modules as production, with different variable values.

The payoff is consistency and confidence: a new environment is a terraform apply away, rollbacks are a git revert away, and the answer to "what changed?" is always in the commit history rather than in someone's memory.

7. Tag Every Resource From Day One

Tags are easy to skip on day one and almost impossible to backfill cleanly on day three hundred. Adopt a tagging convention before you start provisioning real resources. A reasonable starter set:

Environment: production, staging, dev.
Service or Application: which component does this belong to.
Owner or Team: who is responsible.
CostCenter: for billing attribution.
ManagedBy: terraform, cloudformation, manual.

Tagging pays off when you start asking questions like "what is our staging environment costing us?", "who owns this orphaned EBS volume?", or "which resources need patching this quarter?".
Without tags, those questions become much harder to answer, leading to wasted time and effort.

8. Use Safer Deployment Strategies

A direct in-place deployment to production, where the new version replaces the old one for everyone at once, is the simplest pattern and also the riskiest. If the new version has a bug, every customer hits it instantly and your only path forward is rolling back.

Two patterns are worth investing in early.

Blue-Green deployment runs two identical environments (blue and green). You deploy the new version to the idle one, run smoke tests, and then switch traffic over. If something goes wrong, switch back. This works very well with load balancers and managed container services.

Canary deployment rolls the new version out to a small percentage of traffic first (1 to 10 percent). Monitor error rates, latency, and business metrics. If everything looks healthy, gradually increase the percentage. If not, roll back having only affected a small portion of users.

Both strategies dramatically reduce blast radius. Pick whichever fits your tooling, but pick something other than "replace everything at once and hope for the best."

Wrapping up Part 1

We have covered the infrastructure groundwork: environments that match, networks that protect you, predictable IPs, the right compute for each workload, externalized configuration, consistent tagging, and safer deployments.

In Part 2 we move into the operational layer: keyless authentication, secrets management and monitoring. These are the practices that keep you sleeping through the night once you are live on production.

What did I miss? Share it in the comments section.

Thank you for reading. Share if it helped you, and stay tuned for Part 2.

How to Grant Temporary Read-Only Access to a Kubernetes Cluster

CodeHiRise — Tue, 14 Apr 2026 10:54:30 +0000

Hi Everyone,

In production Kubernetes environments, giving full cluster admin access to all actor is a major security risk. Instead, you should follow the principle of least privilege and provide only the exact permissions they need.

This tutorial shows you exactly how an admin can create a limited-access ServiceAccount, attach a Role with the required permissions, generate a short-lived token, and hand over a ready-to-use kubeconfig file to developers, who would not need full admin access to the cluster but only limited access. In this tutorial we will demonstrate temporary read-only access to pods, deployments, and pod logs.

By the end of this guide, developers will be able to run commands like:

kubectl get pods
kubectl get deployments
kubectl logs <pod-name>

while being unable to delete resources, create new ones, or access other namespaces.

Why this approach? ServiceAccounts + short-lived tokens are the recommended way in modern Kubernetes for temporary access. They are easy to revoke, don’t require managing client certificates, and expire automatically.

Let’s get started

Prerequisites

Before you begin, make sure you have:

Admin access to your Kubernetes cluster (kubeconfig with cluster-admin rights).
kubectl installed and configured.
A target namespace where developers will work (we’ll use dev-team in this example change it to your actual namespace).
Basic familiarity with Kubernetes objects (Pods, Deployments, YAML).

If you are new to Kubernetes RBAC, don’t worry we’ll explain every step.

Step 1: Create a Dedicated Namespace (Optional but Recommended)

It’s best practice to isolate developer access to a specific namespace.

kubectl create namespace dev-team

This ensures developers can only see resources inside dev-team and cannot accidentally affect production namespaces.

Step 2: Create a ServiceAccount for the Developer

ServiceAccounts are Kubernetes built-in identities for workloads and users.

kubectl create serviceaccount dev-reader -n dev-team

Using a ServiceAccount instead of a real user certificate makes the setup simpler and easier to manage/revoke.

Step 3: Create a Role with Read-Only Permissions

Create a file named dev-reader-role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: dev-team
  name: dev-reader-role
rules:
  # Read-only access to Pods and Deployments
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]
  # Allow reading pod logs (very common developer need)
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]

Apply it:

kubectl apply -f dev-reader-role.yaml

Rationale for each verb:

get, list, watch -> Allows viewing resources without modification.
pods/log -> Specifically enables kubectl logs command.
No create, update, delete, or patch -> Completely read-only.

Step 4: Bind the Role to the ServiceAccount

Create a file named dev-reader-rolebinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: dev-team
  name: dev-reader-rolebinding
subjects:
  - kind: ServiceAccount
    name: dev-reader
    namespace: dev-team
roleRef:
  kind: Role
  name: dev-reader-role
  apiGroup: rbac.authorization.k8s.io

Apply it:

kubectl apply -f dev-reader-rolebinding.yaml

The RoleBinding connects the permissions (Role) to the identity (ServiceAccount). Without it, the ServiceAccount has zero permissions.

Step 5: Generate a Temporary Token

Kubernetes now supports short-lived tokens out of the box.

kubectl create token dev-reader --namespace dev-team --duration=24h > dev-reader.token

--duration=24h makes the token automatically expire after 24 hours (you can use 1h, 12h, 7d, etc.).
This is much safer than long-lived tokens or certificates.

Copy the token (it will be a long JWT string):

cat dev-reader.token

Step 6: Create the Custom kubeconfig File for the Developer

How to get the CA and API server URL

Run these commands on your admin machine to get required values for kube config.

# Get API server URL
kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'

# Get CA certificate (base64)
kubectl config view --minify --raw -o jsonpath='{.clusters[0].cluster.certificate-authority-data}'

Create a file named dev-reader.kubeconfig with the following content and replace the placeholders with correct values from above commands

apiVersion: v1
kind: Config
clusters:
- cluster:
    certificate-authority-data: <YOUR_CLUSTER_CA_BASE64>   # e.g. LS......................
    server: https://<YOUR_API_SERVER_URL>                  # e.g. https://api.yourcluster.com:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    namespace: dev-team
    user: dev-reader
  name: dev-reader-context
current-context: dev-reader-context
users:
- name: dev-reader
  user:
    token: <PREVIOUSLY_GENERATED_TOKEN>

Step 7: Hand Over the kubeconfig to the Developer

Send the dev-reader.kubeconfig file to the developer securely. The developer can now use it with any kubectl command:

# Test access
kubectl --kubeconfig=dev-reader.kubeconfig get pods
kubectl --kubeconfig=dev-reader.kubeconfig get deployments
kubectl --kubeconfig=dev-reader.kubeconfig logs <pod-name>

They will not be able to run kubectl delete, kubectl apply, or access other namespaces.

Cleanup

kubectl delete rolebinding dev-reader-rolebinding -n dev-team
kubectl delete serviceaccount dev-reader -n dev-team

Security Best Practices Summary

Always use namespace-scoped Roles instead of ClusterRoles when possible.
Use short-lived tokens (--duration flag).
Never share the cluster-admin kubeconfig.
Rotate or delete ServiceAccounts regularly.

Conclusion

You have now successfully given a developer temporary, read-only access to pods, deployments, and logs in your Kubernetes cluster with zero risk of destructive actions.
This pattern scales well, you can create different ServiceAccounts for different teams, different namespaces, or even different permission levels (e.g., one for “view only”, another for “view + logs”).

Thank you for reading. Share if it helped your team!

Integrate Home Assistant InfluxDB and Grafana for Stunning Dashboards

CodeHiRise — Sun, 24 Aug 2025 19:07:54 +0000

Hi everyone,

In this post we will dive in to integrating Home Assistant with InfluxDB V2, and Grafana to store sensor data and create beautiful, real-time dashboards.

This step-by-step guide is perfect for home automation enthusiasts and IoT developers looking to monitor and visualize sensor data efficiently.

Prerequisites

Before we begin, ensure you have the following set up:

A working Home Assistant instance.
InfluxDB v2 installed and accessible (local or cloud-based).
Grafana installed and running (local or cloud-hosted).
Basic knowledge of YAML configuration for Home Assistant.
Access to your InfluxDB and Grafana admin interfaces.

Architecture

To achieve real-time dashboards, we need a time-series database to store sensor data from Home Assistant, which Grafana can then use as a data source. InfluxDB is an open-source, high-performance time-series database, making it ideal for this use case. Home Assistant has a built-in integration for InfluxDB, allowing sensor data to be sent to a specified InfluxDB bucket. Grafana then connects to InfluxDB to create visually appealing dashboards for data visualization.

Below is a high-level architecture diagram:

Step 1 Setup home assistant with InfluxDB integration

The InfluxDB integration in Home Assistant is a legacy integration, meaning it must be configured manually via the configuration file (configuration.yaml) rather than the UI.

Below, we’ll walk through the steps to set it up.

Refer InfluxDB integration for more details.

Gather InfluxDB Details

You’ll need the following information from your InfluxDB v2 instance:

Hostname or IP address of your InfluxDB server.
Port (default is 8086 for InfluxDB v2).
API Token for authentication.
Organization ID.
Bucket Name for storing sensor data.

Finding Your Organization ID

Log in to your InfluxDB v2 web interface. Click your profile in the top-left corner. Select the About tab to view your Organization ID.

Creating Bucket

In the InfluxDB UI, navigate to the Load Data > Buckets section.

Click Create Bucket.

Provide a Bucket Name (e.g., home_assistant_data).

Optionally, set a Retention Period (e.g., 30 days) to manage data storage.

Creating API Token

Go to the Load Data > Tokens section in the InfluxDB UI.

Click Generate API Token > Custom API Token.

Provide a descriptive name (e.g., Home_Assistant_Write).

Assign Write permissions to the bucket you just created. This adheres to the least privilege principle for enhanced security.

Now, update your Home Assistant configuration.yaml file with the InfluxDB integration settings.

Below is an example configuration that includes specific entities using entity_globs to filter data.

For example, we’ll include Sonoff smart switches and Zigbee2MQTT temperature sensors.

You can customize this to include other entities or entire domains as needed.

influxdb:
  api_version: 2
  ssl: false
  host: <INFLUXDB_IP>
  port: <POST default - 8086>
  token: <API_TOKEN>
  organization: <ORGANIZATION_ID>
  bucket: <BUCKET_NAME>
  tags:
    source: HA
  tags_attributes:
    - friendly_name
  default_measurement: units
  include:
    entity_globs:
      - sensor.sonoff*
      - sensor.l1_tah_sensor*
      - sensor.l2_tah_sensor*

Replace <INFLUXDB_IP>, <API_TOKEN>, <ORGANIZATION_ID>, and <BUCKET_NAME> with your InfluxDB details.

Make sure api_version: 2 since we use InfluxDB V2.

The entity_globs section filters specific entities. Adjust this based on your sensors.

The ssl: false setting assumes a non-SSL connection. Set to true if your InfluxDB instance uses SSL.

After updating the configuration, restart Home Assistant to apply the changes.

Verify Data in InfluxDB

Navigate to the Data Explorer section.

Select your bucket and filter by the entities you configured (e.g., sensor.sonoff*).

You should see sensor data populating over time.

If data isn’t appearing, verify your configuration, ensure the sensors are sending data, and confirm the API token has the correct permissions.

If you change home assistant integration configurations, you must restart home assistant to take them into effect.

Step 2 Setup InfluxDB integration with Grafana

With data flowing into InfluxDB, it’s time to connect it to Grafana as a data source.

Navigate to Configuration > Data Sources > Add Data Source.

Select InfluxDB from the list.

Configure the data source with the following details:

Name: A descriptive name (e.g., Home_Assistant_InfluxDB).
Query Language: Select Flux (since we use InfluxDB v2).
URL: Enter your InfluxDB URL (e.g., http://INFLUXDB_IP:8086).
Organization: Your InfluxDB Organization name.
Token: Generate a new API token in InfluxDB with Read permissions for your bucket (follow same steps we did earlier. make sure to only provide read permissions).

Click Save & Test. You should see a success message confirming the connection.

Security Tip: Use separate API tokens for Home Assistant (write) and Grafana (read) adhering to least privileges to maintain secure access control.

Step 3 Create Stunning Grafana Dashboards

Now comes the exciting part, building your real-time dashboards in Grafana!

In Grafana, go to Dashboards > Create Dashboard

Add Visualization.

Select your newly created InfluxDB data source.

Write a Flux query to fetch your sensor data. If you’re unsure how to write Flux queries, use the InfluxDB Data Explorer to build and test queries visually.

In InfluxDB’s Data Explorer, select your bucket and desired entities.

Filter and aggregate data as needed (e.g., mean temperature over time).

Click script editor to switch to the Script Editor to view the generated Flux query.

Copy the Flux query from InfluxDB and paste it into Grafana’s query editor.

Click Refresh to visualize the data.

Customize your dashboard:

Adjust the visualization type (e.g., line graph, gauge, or heatmap).
Add multiple panels for different sensors (e.g., temperature, humidity, or power usage).
Use Grafana’s styling options to enhance the look and feel.

Pro Tip: Experiment with Grafana’s Thresholds and Alerts to highlight critical values or receive notifications when sensors report unusual data.

Final Thoughts

Congratulations! 🥳 You’ve successfully integrated Home Assistant with InfluxDB and Grafana to create stunning, real-time dashboards.
From here, your imagination is the only limit build dashboards to monitor energy usage, track indoor climate, or visualize smart home trends over time.

Additional Tips

Optimize Data Retention: Set appropriate retention policies in InfluxDB to manage storage efficiently.

Secure Your Setup: Use SSL for InfluxDB and Grafana in production environments.

Explore Grafana Plugins: Enhance your dashboards with plugins like the Worldmap Panel for geolocation data.

Backup Configurations: Regularly back up your Home Assistant and Grafana configurations to avoid data loss.

For further details, check the official documentation:

Thank you for reading, and happy dashboarding!

Stop Wasting Money 💰 on Idle Jenkins Instances: A Serverless 🚀 Solution to Slash Your AWS Bill

CodeHiRise — Sun, 09 Mar 2025 17:53:33 +0000

Hello DevOps Community,

In many environments, Jenkins instances sit idle between infrequent builds, accruing unnecessary EC2 costs. If your team uses Jenkins only periodically—for deployments, pull requests, or CI/CD pipelines you’re likely overpaying for compute resources.

In this tutorial, we’ll explore a serverless solution to stop the Jenkins instance during idle periods and start it automatically when a new build is triggered.

🚀 How It Works

This solution uses two AWS Lambda functions:

Stop Jenkins: Monitors activity and shuts down the EC2 instance when idle.
Start Jenkins: Starts the instance when a GitHub webhook triggers a build and forwards the request to Jenkins.

By running the EC2 instance only during active builds, you eliminate idle compute costs.

⚠️ Important Consideration:

This approach disrupts scheduled Jenkins jobs (e.g., nightly builds). Ensure your instance operates purely on-demand via webhook triggers and does not rely on schedules.

🔷 Note that This guide assumes you use the GitHub plugin to trigger builds on GitHub push events. Adjust the code if your setup differs.

🛑 Stop Jenkins workflow

stop Jenkins function will run periodically using event bridge scheduler to check whether Jenkins server is idle every 5 minutes and if it is idle this lambda function will stop the Jenkins server.

to check jenkins is idle we use Jenkins computer api/computer/api/json.

import boto3
import urllib.request
import json
import os
import base64

ec2 = boto3.client('ec2')

jenkins_ip = os.environ['JENKINS_IP']

INSTANCE_ID = os.environ['EC2_INSTANCE_ID']
JENKINS_URL = f'http://{jenkins_ip}:8080'
JENKINS_USER = os.environ['JENKINS_USER']   
JENKINS_TOKEN = os.environ['JENKINS_TOKEN']  

def lambda_handler(event, context):

    try:
        state = get_instance_state()
        print(f"Instance state: {state}")
        if state == 'stopped':
            return {'statusCode': 200, 'body': 'Instance is stopped'}
        else:
            if is_jenkins_idle():
                ec2.stop_instances(InstanceIds=[INSTANCE_ID])
                return {'statusCode': 200, 'body': 'Instance is idle initiated stop action'}

            return {'statusCode': 200, 'body': 'Instance is running and Jenkins is busy'}

    except Exception as e:
        print(e)
        return {'statusCode': 500, 'body': "error occurred"}

def get_instance_state():
    response = ec2.describe_instances(InstanceIds=[INSTANCE_ID])
    return response['Reservations'][0]['Instances'][0]['State']['Name']

def is_jenkins_idle():
    try:
        # Check running builds via executors
        computer_url = f"{JENKINS_URL}/computer/api/json"
        request = urllib.request.Request(computer_url)
        request.add_header('Authorization', 'Basic ' + 
                           base64.b64encode(f"{JENKINS_USER}:{JENKINS_TOKEN}".encode()).decode())

        response = urllib.request.urlopen(request)
        if response.status != 200:
            print(f"Failed to get computer info: {response.status}")
            return False
        computer_data = json.loads(response.read().decode())
        print(computer_data)
        for computer in computer_data['computer']:
            if not computer['idle']:
                return False
        print("Jenkins is idle")
        return True
    except Exception as e:
        print(f"Error checking Jenkins status: {e}")
        return False
    finally:
        response.close()

🚀 Start Jenkins workflow

When a GitHub webhook triggers:
- The Start Jenkins Lambda checks if the instance is running.
- If stopped, it starts the instance and waits until it’s healthy.
Once Jenkins is active, the Lambda forwards the webhook payload to /github-webhook/.

import boto3
import time
import urllib.request
import json
import os

ec2 = boto3.client('ec2')

jenkins_ip = os.environ['JENKINS_IP']

INSTANCE_ID = os.environ['EC2_INSTANCE_ID']
JENKINS_URL = f'http://{jenkins_ip}:8080'

def lambda_handler(event, context):

    try:
        payload = json.loads(event['body'])
        print(payload)

        state = get_instance_state()
        print(f"Instance state: {state}")
        if state == 'stopped':
            ec2.start_instances(InstanceIds=[INSTANCE_ID])
            wait_for_instance_running()

        wait_for_jenkins_ready()

        trigger_build(payload)
        return {'statusCode': 200, 'body': 'Build triggered'}

    except Exception as e:
        print(e)
        return {'statusCode': 500, 'body': "error occurred"}

def get_instance_state():
    response = ec2.describe_instances(InstanceIds=[INSTANCE_ID])
    return response['Reservations'][0]['Instances'][0]['State']['Name']

def wait_for_instance_running():
    while get_instance_state() != 'running':
        time.sleep(10)

def wait_for_jenkins_ready():
    while True:
        try:
            req = urllib.request.Request(f"{JENKINS_URL}/login")
            print(f"Attempting to connect to Jenkins at {JENKINS_URL}")
            response = urllib.request.urlopen(req)
            print(f"Jenkins is ready: {response.status}")
            break
        except Exception as e:  
            print(f"Jenkins is not ready error: {e}")
            time.sleep(10)

def trigger_build(payload):

    url = f"{JENKINS_URL}/github-webhook/"
    try:
        req = urllib.request.Request(url, method='POST', data=json.dumps(payload).encode(), headers={
            'Content-Type': 'application/json',
            'User-Agent': 'lambda-function',
            'X-GitHub-Event': 'push'
        })
        response = urllib.request.urlopen(req)
        print(f"Jenkins triggered build successfully with response: {response.status}")
    except Exception as e:
        print(f"Failed to trigger Jenkins build: {e}")
        print(f"Error response: {e.read().decode()}")

Key Benefits

💸 Cost Savings: Pay only for active build time. For example, if you use the Jenkins instance for 2 hours daily for active builds, your monthly uptime totals 60 hours (versus 720 hours for 24/7 operation), resulting in over 90% cost savings.
✨ Automation: No manual intervention required.

Implement this solution to align your Jenkins costs with actual usage! 🚀💰📉

✋ If you need to use the Jenkins server for extended periods (e.g., debugging), you can either:

Temporarily disable the EventBridge scheduler to prevent automatic shutdowns.
Enable stop-protection on the EC2 instance until debugging is complete.

This ensures uninterrupted access while retaining cost optimization as the default behavior.

What do you think will you use this on your environment.

share your thoughts in the comment section.

Thank you for reading

Cheers 🥂

Setup a GitHub Self-Hosted Runner on an Ubuntu VM: A DevOps Guide

CodeHiRise — Sun, 16 Feb 2025 17:03:39 +0000

Hi everyone,

In this article, we will walk you through the steps for setting up GitHub Self-hosted Runners in Ubuntu virtual machine as a service for persistence runner installation.

The primary reason organizations to adopt GitHub self-hosted runners is to securely access internal, private networks such as databases, services, or virtual machines without exposing them to the public internet. By keeping workflows within their controlled infrastructure, they eliminate risks like accidental data leaks or compliance violations.

Additionally, self-hosted runners safeguarding sensitive code, credentials, or artifacts since runners are hosted on organization network.

Prerequisites

A GitHub account.
An Ubuntu Virtual machine with root access with internet connectivity.
Basic familiarity with Linux commands and GitHub Actions.

Lets get started

First go to your desired GitHub repository and go to Settings > Actions > Runners > New self-hosted runner.

When you click New self hosted runner button you will get below page with all required steps.

It outlines downloading and setting up action runner but in our case we will take some extra steps to make sure our setup is more secure and configure runner as a systemd service.

Step 1: Create a Dedicated System User for GitHub action runner

we will create new system user user in our ubuntu vm.

Run below commad.

sudo useradd --system --shell /usr/sbin/nologin gh-action-runner-user

--system: Creates a non-interactive service account.
--shell /usr/sbin/nologin: Blocks SSH/login access.

Step 2: Download the GitHub Runner

Run below command to create runners installation directory in /opt directory.

sudo mkdir /opt/gh-runner
cd /opt

Use the /opt directory for installation of add-on application software package and avoid using user's home directory

Run below command to change ownership of the directory to our current owner for downloading and extracting files.

sudo chown codehirise:codehirise gh-runner/
cd gh-runner

then run below to extract action runner.

curl -o actions-runner-linux-x64-2.322.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.322.0/actions-runner-linux-x64-2.322.0.tar.gz

optionally run below command to verify downloaded tar file.

echo "b13b784808359f31bc79b08a191f5f83757852957dd8fe3dbfcc38202ccf5768  actions-runner-linux-x64-2.322.0.tar.gz" | shasum -a 256 -c

Next run below command to extact all actions runner tar.gz file.

tar xzf ./actions-runner-linux-x64-2.322.0.tar.gz

Step 3: Configure the Runner

After downloading run below command to configure GitHub action runner.

./config.sh --url https://github.com/codehirise/github-actions-private-runner-demo --token YOUR_TOKEN

Replace YOUR_TOKEN with the token displayed in GitHub add new self hosted action runner page.

Here you will be prompled for some input like runner name and lables. You can enter them as you see fit.

It will create some additional files like svc.sh which we will be using to configure action runner as a systemd service.

Before that make sure to change the ownership of the directory to our Linux service user which is gh-action-runner-user.
Run below command

sudo chown gh-action-runner-user:gh-action-runner-user /opt/gh-runner/ -R

Step 4: Configure the Runner Systemd Service

Next run below command to install systemd service for github action runner. This will install action runner as a systemd service and set user as gh-action-runner-user.

sudo ./svc.sh install gh-action-runner-user

Next you can run status command to check status

sudo ./svc.sh status

and run start command to start github action runner

sudo ./svc.sh start

to check github action run related logs simply run

journalctl -f -u actions.runner.codehirise-github-actions-private-runner-demo.gh-private-runner-demo.service

Note that actions.runner.codehirise-github-actions-private-runner-demo.gh-private-runner-demo.service is the service name which is displayed when running sudo ./svc.sh status command.

It will show connected to GitHub which indicate that we have successfully setup our GitHub action runner as a service.
if you now visit runners page you can see our brand new github action self hosted runner is ready and waiting to pickup jobs.

Action runner systemd service comes enabled by default so it will automatically start after a system reboot.

Step 5: Using selfhosted runner in Github action workflow

To use this runner in workflow , set runs-on in your github actions workflow file to selfhosted

Example workflow file below.

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the workflow will run
on:
  # Triggers the workflow on push or pull request events but only for the "main" branch
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: self-hosted

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v4

      # Runs a single command using the runners shell
      - name: Run a one-line script
        run: echo Hello, world!

      # Runs a set of commands using the runners shell
      - name: Run a multi-line script
        run: |
          echo Add other actions to build,
          echo test, and deploy your project.

in action workflow we can see job ran on the runner we setup.

In our runner logs we can see job completed successfully

Self-hosted GitHub runners on Ubuntu VMs put you in control—locking down sensitive workflows and integrating seamlessly with private infrastructure. With this setup, your pipelines stay fast, secure, and entirely within your environment. Will this be your team’s next step toward DevOps adventure? 🚀

Thank you for reading.

Access EC2 Instances Privately Using AWS Systems Manager

CodeHiRise — Sat, 30 Dec 2023 15:24:49 +0000

Hi Everyone,

In this article, we will learn how to access AWS EC2 instances using AWS Systems Manager. This is quite useful,

when you don't want to expose your instance to public internet via a public ip.
when you want to manage a large number of instances without sharing ssh keys.
run commands in many instances without logging in to them.

but this does come with a few prerequisites.

SSM agent should be up and running to connect to this instance (SSM agent comes with Amazon Linux and Ubuntu amis by default).
The instance should have access to the public internet (or access to SSM endpoints via AWS private endpoints).
The instance should have a role with the required permissions.

to continue with this post I will assume that you have access to an AWS account and have a basic idea of AWS.

keep in mind that if you are not in the AWS free tier this will incure you a cost.

With the above in mind let's get started.

First, we will create an aws instance role which is required to grant permissions to the aws instance so that the ssm agent in the ec2 instance can connect with the aws systems manager.

navigate into aws iam roles and click create a role as below.

in next page add trusted entity EC2 and select EC2 Role for AWS Systems Manager

next in permissions since we chose EC2 Role for AWS Systems Manager it will automatically set AmazonSSMManagedInstanceCore policy which is the required permission for this use case.

after hitting next add a name for your role and a description.

and review whether all settings are mentioned below and click Create role.

and we are done with iam role for our instance. now let's create an ec2 instance.

navigate to the ec2 menu and click launch instance

I will add an instance name as shown below but this is optional.

next select ami for this instance I will choose the default Amazon Linux 2023 image for this purpose.

next, specify the instance type and key pair for login. for this example, I will add a keypair since it will be useful if there is an issue with agent connectivity although we are not using it in this tutorial.

next under the network setting, I will remove all inbound access to this instance to demonstrate we can connect without public access. but keep in mind that outbound access is required for ssm agent connectivity, which is added by default when creating a security group.

I will keep storage as default as below.

in the advanced setting we need to add iam role(IAM instance profile) we created in the previous steps.

next hit the launch instance button.

and you will get a similar output as below.

click on the instance id to navigate to the instance page. Here we can recognize the newly created instance. click on the instance id to view the instance details page.

click on Connect to navigate to the Connect page.

in the connect page navigate to the session manager tab and press connect

if you receive an error as below, check the troubleshooting section below.

if everything goes well new tab will be created with a terminal displayed as below. now you are connected to the instance with a terminal.

note user is ssm user.

Troubleshooting.

check if the instance has outbound internet access in security group rules.

make sure the instance has iam instance role attached.

check iam instance role permissions.
only require permissons is AmazonSSMManagedInstanceCore policy.

check if the instance ssm agent is reporting to the aws ssm fleet manager.
it should be listed as online.

If everything above is in place and you still cannot get ssm connectivity you will need to further troubleshoot issues in ssm agent.
a good place to start troubleshooting is by checking logs.

check ssm agent logs under /var/log/amazon/ssm

here we can see there is an access denied error in errors.log

here we can see there is an access denied error in amazon-ssm-agent.log and it is sleeping for 30 minutes.
this could happen if we attach the role after the instance starts,
so ssm agent checked to authenticate and failed then the next retry will be in 30 minutes.

we can restart ssm agent so it will try to reauthenticate and succeed this time.

This is just one scenario of debugging and your specific scenario might change but going through logs will give an insight into what the problem.

Thank you for reading. Share your thoughts in comments section.

An Introduction To CAP Theorem In Distributed Systems

CodeHiRise — Mon, 11 Sep 2023 14:12:47 +0000

Hello everyone,

Managing distributed computing systems across multiple nodes is a complex challenge. The CAP theorem highlights the intricate trade-offs involved in designing and managing such systems. This theorem centers around three essential attributes:

Consistency
Availability
Partition Tolerance

Collectively referred to as the CAP triad.

This blog post aims to provide an overview of the CAP theorem, including its significance, implications, and real-world applications.

The three properties of the CAP theorem are:

Consistency

Consistency, in the context of the CAP theorem, refers to the idea that all nodes within a distributed system should have access to the same data at the same time. In other words, regardless of the node you interact with, you should observe the most recent data update. To maintain the same date in all nodes upon an update that data should be replicated to all nodes before the write is considered successful. Maintaining consistency becomes crucial in scenarios where data integrity and correctness are paramount, such as financial systems or databases for critical applications.

Availability

Availability ensures that every request made to the system should receive a response. However, the response might not always include the most recent data. Availability ensures that a system remains operational and responsive even when individual nodes experience failures or slowdowns. This attribute is crucial in systems where downtime or unresponsiveness is unacceptable, such as e-commerce platforms or real-time analytics systems.

Partition Tolerance

Partition tolerance refers to the system's ability to continue functioning despite arbitrary network communication failures between nodes. A partition communication error occurs when there are temporary delays or loss of connections between nodes in the distributed system.

CAP theorem proof

According to the CAP theorem, it is impossible for a distributed system to simultaneously achieve all three properties. As a result, system designers must make trade-offs and prioritize certain properties based on the specific requirements of their applications.

Let's assume that we have a 2-node system with consistent, available, and partition tolerance. which means this system can fulfill all three properties.

At the start, both servers s1 and s2 have the same variable with value v1.

now client will update v1 to v2 value on this variable connecting to s1 server.

now if the client reads the same variable value from s2 it will return v1 which was the previous value breaking consistency in the system.

This happens because the system cannot communicate between two nodes so s1 cannot replicate changes made to s2 node.

from this we can conclude that we cannot fulfill a consistent, available, partition tolerant system. In this example, we broke consistency over availability and partition tolerance.

CA Systems:

Some systems prioritize Consistency and Availability (CA) over Partition Tolerance. Traditional relational databases often fall into this category, where maintaining data integrity and guaranteeing real-time updates are key objectives. However, these systems might struggle during network failures between partitions, potentially leading to downtime.

Examples

postgresql

CP Systems:

Other systems emphasize Consistency and Partition Tolerance (CP). Such systems ensure data integrity and are designed to function well even in the presence of network partitions. Distributed databases like MongoDB follow this model, delivering consistency at the expense of potential unavailability during connection failures between partitions.

Examples

mongodb

MongoDB has tunable levels of consistency for reads and writes. reference

AP Systems:

Finally, some systems prioritize Availability and Partition Tolerance (AP). These systems are designed to remain operational even in the face of network disruptions, responding even if the data might be temporarily inconsistent across nodes. NoSQL databases like Apache Cassandra exemplify this approach.

Examples

cassandra

Cassandra makes some guarantees about its scalability, availability, and reliability. reference

PACELC Theorem: Extending the CAP Theorem

PACELC theorem extends the CAP theorem by introducing extra factors in to consideration, including latency and consistency, to offer a more comprehensive perspective on distributed systems.

It states that in case of network partitioning (P) in a distributed computer system, one has to choose between availability (A) and consistency (C) (as per the CAP theorem), but else (E), even when the system is running normally in the absence of partitions, one has to choose between latency (L) and consistency (C). wikipedia

According to the PACELC theorem, when a network partition occurs, a trade-off must be made between availability and consistency. In the absence of a partition, another trade-off arises between latency and consistency. This extended framework acknowledges the importance of latency in real-world applications and highlights the need to consider trade-offs beyond the traditional CAP properties.

Thank you for reading. Share if you loved it.

Drop your thoughts in comment section

Execute Terminal Commands And Receive Live Output With React+SSE

CodeHiRise — Sat, 09 Sep 2023 19:01:33 +0000

Hello everyone,

This is the second part of Node.js + SSE if you haven't checked out part one where we create SSE server I highly recommend you to check it out first.

Client

Here i will go with react + vite setup with tailwind as UI, but you can use any framework/library you want since these basics will stay the same.

I assume that you have installed nodejs and have a code editor setup, and you have previously created SSE server running on localhost:3000

mkdir ssedemoclient

yarn create vite

i will also install tailwind ui setup. check out tailwind official doc for setting up for setting up with react+vite.

Add the below code snippet to app.js.

import { useState } from "react";
import "./App.css";

let events = null;

function App() {
  const [commandOut, setCommandOut] = useState([]);
  const [isActive, setisActive] = useState(false);

  const sendCommand = (command) => {
    // close current event if user submit another command
    if (events) {
      console.log("close current event");
      setCommandOut([]);
      events.close();
    }

    if (!command) {
      console.log("Empty command");
      return;
    }

    // create event source
    events = new EventSource(`http://localhost:3000/run?command=${command}`);

    // add eventlistner for message output from command
    events.addEventListener(
      "stdout",
      function (e) {
        let parsedData = JSON.parse(e.data);
        console.log(parsedData);
        setCommandOut((pre) => [parsedData, ...pre]);
      },
      false
    );

    events.addEventListener(
      "stderr",
      function (e) {
        let parsedData = JSON.parse(e.data);
        console.log(parsedData);
        setCommandOut((pre) => [parsedData, ...pre]);
      },
      false
    );

    events.addEventListener(
      "err",
      function (e) {
        let parsedData = JSON.parse(e.data);
        console.log(parsedData);
        setCommandOut((pre) => [parsedData, ...pre]);
      },
      false
    );

    events.addEventListener(
      "open",
      function (e) {
        console.log("open");
        setisActive(true);
      },
      false
    );
    // event handler for command exit to stop retry
    events.addEventListener(
      "exit",
      function (e) {
        console.log("exited");
        events.close();
        setisActive(false);
      },
      false
    );

    events.addEventListener(
      "error",
      function (e) {
        if (e.readyState == EventSource.CLOSED) {
          // Connection was closed.
          setisActive(false);
        }
      },
      false
    );

    events.addEventListener("error", (e) => {
      console.log("An error occurred while attempting to connect.");
      setisActive(false);
      events.close();
    });
  };

  return (
    <div className="bg-slate-800 h-screen">
      <div className="container mx-auto ">
        <div className="h-[20vh]">
          <h1 className="text-3xl text-cyan-50 pt-10 font-bold text-center">
            Run command with SSE Demo
          </h1>
          <form
            onSubmit={(e) => {
              e.preventDefault();
              console.log(e.target[0].value);
              sendCommand(e.target[0].value);
            }}
          >
            <div className="flex flex-row justify-between align-middle self-center content-center pt-10 w-full">
              <input
                type="text"
                id="last_name"
                className="bg-gray-50 border border-gray-300 placeholder-gray-600 text-gray-900 text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-[90%] p-2.5"
                placeholder="Enter command to run"
                required
              />
              {/* show only when listening to messeges */}
              <div
                className={`content-center flex flex-wrap ${
                  isActive ? "" : "hidden"
                }`}
              >
                {/* spinner animation on active conenction */}
                <svg
                  aria-hidden="true"
                  role="status"
                  className="inline w-6 h-6 mx-3 text-blue-600 animate-spin"
                  viewBox="0 0 100 101"
                  fill="none"
                  xmlns="http://www.w3.org/2000/svg"
                >
                  <path
                    d="M100 50.5908C100 78.2051 77.6142 100.591 50 100.591C22.3858 100.591 0 78.2051 0 50.5908C0 22.9766 22.3858 0.59082 50 0.59082C77.6142 0.59082 100 22.9766 100 50.5908ZM9.08144 50.5908C9.08144 73.1895 27.4013 91.5094 50 91.5094C72.5987 91.5094 90.9186 73.1895 90.9186 50.5908C90.9186 27.9921 72.5987 9.67226 50 9.67226C27.4013 9.67226 9.08144 27.9921 9.08144 50.5908Z"
                    fill="#a6c0d7"
                  />
                  <path
                    d="M93.9676 39.0409C96.393 38.4038 97.8624 35.9116 97.0079 33.5539C95.2932 28.8227 92.871 24.3692 89.8167 20.348C85.8452 15.1192 80.8826 10.7238 75.2124 7.41289C69.5422 4.10194 63.2754 1.94025 56.7698 1.05124C51.7666 0.367541 46.6976 0.446843 41.7345 1.27873C39.2613 1.69328 37.813 4.19778 38.4501 6.62326C39.0873 9.04874 41.5694 10.4717 44.0505 10.1071C47.8511 9.54855 51.7191 9.52689 55.5402 10.0491C60.8642 10.7766 65.9928 12.5457 70.6331 15.2552C75.2735 17.9648 79.3347 21.5619 82.5849 25.841C84.9175 28.9121 86.7997 32.2913 88.1811 35.8758C89.083 38.2158 91.5421 39.6781 93.9676 39.0409Z"
                    fill="currentColor"
                  />
                </svg>
              </div>
              {/* show connection close button */}
              {isActive && (
                <button
                  type="button"
                  className="text-white bg-slate-500 hover:bg-slate-600 p-2 font-medium rounded-lg text-sm px-5 mr-4"
                  onClick={() => {
                    console.log("event listen stopped");
                    events.close();
                    setisActive(false);
                  }}
                >
                  Stop
                </button>
              )}
              <button
                type="submit"
                className="text-white bg-blue-600 hover:bg-blue-800 p-2 font-medium rounded-lg text-sm px-5 "
              >
                Run
              </button>
            </div>
          </form>
        </div>
        {/* list down commands with timestamp */}
        <div className="text-white pt-10 whitespace-pre h-[80vh] overflow-y-auto">
          {commandOut.map((data, index) => (
            <div className="flex" key={index}>
              <div>
                <p className="mr-2 text-sky-400">{`${new Date()
                  .toLocaleString()
                  .replace("T", " ")} : `}</p>
                <p>{data.type}</p>
              </div>
              <p>{data.out}</p>
            </div>
          ))}
        </div>
      </div>
    </div>
  );
}

export default App;

EventSource is where the server SSE integrates with the Client end.

EventSource is a javascript API that enables the client side to receive SSE from a server over a single, long-lived HTTP connection.

Then we need to add event listers to EventSource. here we need to listen to the events we send from the server end which are stdout,stderr, err, and exit events.

We maintain one useState for managing command outputs and one for keeping the command execution state since we can execute commands like tail to get realtime updates of a file that follow file changes realtime.

exit EventListener will close the connection from the client end so it is not automatically retried.

after we update the state to commandOut, we need to render ui with event type, time, and command output using the array map method.

I have added additional functionality cancel where we can close the current connection to stop listening to events. in this case, the client will close the connection and the command executing on the server will also be killed.

If everything goes well, you will get below UI.

Git repo

codehirise / nodejs-react-sse-code-executor

Execute Terminal Commands And Receive Live Output With Nodejs + SSE + React

Read full article on

build backend : https://codehirise.com/post/building-a-real-time-command-execution-app-with-node-js-sse

build frontend :https://codehirise.com/post/execute-terminal-commands-and-receive-live-output-with-react-sse

View on GitHub

Thank you for reading. Share if you loved it.

Building A Real-Time Command Execution App With Node.Js + SSE

CodeHiRise — Sat, 09 Sep 2023 18:56:37 +0000

Hello everyone,

Real-time communication protocols are essential for applications that require realtime data updates. Two popular options for achieving real-time communication are Server-Sent Events (SSE) and WebSockets. In this two part series, we'll dive into SSE ie Server sent events and build SSECommander, An application that execute terminal commands sent via HTTP and delivers the command output stream real-time through Server-Sent Events (SSE) using Node.js.

With server-sent events, a server can push data to the client without the client needing to poll the server to get data once in a while. SSE has some similarities with WebSockets but also has distinct features that make it better for many use cases.

Web sockets

Bi-directional i.e Full duplex
support Binary and text data
Uses WebSocket protocol
Scaling is more complex than SSE

Server sent events

Uni directional i.e Half duplex
support only text data
Uses HTTP protocol
Simple and user friendly

WebSockets are a perfect fit for applications that require real-time collaboration, such as a collaborative drawing app, where both the client and server need to send data realtime. However, it's worth noting that implementing WebSockets can introduce a notable level of complexity to your application and infrastructure. This complexity can add extra development and maintenance efforts.

On the other hand, if your use case primarily involves the server pushing data to the client without requiring bidirectional communication, Server-Sent Events (SSE) are an excellent choice. SSE offers a simpler and more streamlined design, allowing you to focus more on your business logic instead of spending time on debugging complex WebSocket-related issues. SSE provides a straightforward approach to delivering continuous updates from the server to the client, resulting in an efficient and straightforward implementation process.

So without further ado let's get started.

Server

First, we will start with developing the server.

I assume that you have installed nodejs and have a code editor setup.

Let's create a directory and init npm

mkdir ssedemoserver
npm init -y

let's install express and cors to handle cross-origin.

npm I express cors

create app.js file

touch app.js

This will add required libraries. child_process is required to execute commands in the host machine aka our server. we can also use exec over spawn but here I will use spawn since it will stream data instead of buffering. refere nodejs official doc for more info on that.

const express = require("express");
const cors = require("cors");

// setup cors
const app = express();
app.use(cors());

// required to execute commands on host
const { spawn } = require("child_process");

const PORT = 3000;

Now let's create the route for sse events. this is a GET route and the command is sent via http query parameters.

SSE requires setting some headers to begin.

Headers

`Content-Type: text/event-stream`

This header specifies the content type of the response as "text/event-stream". It indicates that the server will be sending an SSE stream, which is a text-based format for sending a continuous stream of events to the client.

`Cache-Control: no-cache`

This header instructs the client and any intermediary caches not to cache the SSE response. SSE is designed for real-time communication, and caching the response would defeat the purpose of receiving continuous updates.

`Connection: keep-alive`

This header enables a persistent connection between the server and the client. By setting the "Connection" header to "keep-alive", the server informs the client to keep the connection open for further events. This allows the server to send events to the client without having to establish a new connection for each event, improving efficiency and reducing overhead.

Data format

`Event Type`

This line starts with event: and specifies the type of event being sent. It is optional but can be useful for client-side event handling.

`Data`

This line starts with data: and contains the actual data being sent in the event. Multiple lines can be used for multi-line data. Each line should start with data: and end with \n. To end message append \n\n at the end

`ID`

This line starts with id: and provides a unique identifier for the event.

`Retry`

Retry: This line starts with retry: and specifies the reconnection time in milliseconds if the connection is closed. It allows the client to automatically reconnect after a specified time defaulting to 3s.

in the below, i will not set retry since default retry is suitable for this implementation

I have used stdout,stderr, err, and exit event types to emphasize the usage of events but you can always simplify that design for better use cases.

once a command is exited we will close the connection since there will be no more data to send from the server end. after the response id close from the server client will retry to connect, to eliminate this behavior we will send an exit event so the client will close the connection.

in the event of a client connection closing, kill the process as we don't want to keep any dangling processes.

// run route handler
app.get("/run", async (request, response) => {
  console.log(`command : ${request.query.command}`);

  // set headers required for sse `"Content-Type": "text/event-stream",`is where the magic happens
  const headers = {
    "Content-Type": "text/event-stream",
    Connection: "keep-alive",
    "Cache-Control": "no-cache",
  };

  // send 200 response to client
  response.writeHead(200, headers);

  // break down command to command and arguments as requried by spaw methode
  const runCommand = spawn(
    request.query.command.split(" ")[0],
    request.query.command.split(" ").slice(1)
  );

  console.log(`pid : ${runCommand.pid}`);

  // send stdout encoding to uft8 since sse only support text
  runCommand.stdout.setEncoding("utf8");

  // add event listener to list to data on stdout
  runCommand.stdout.on("data", (chunk) => {
    console.log("stdout : " + String(chunk));

    // create it for event
    let id = new Date().getTime();
    // write id to response
    response.write(`id: ${id}\n`);
    // set event
    response.write(`event: stdout\n`);
    // write data to response
    response.write(
      `data: ${JSON.stringify({ type: "stdout", out: String(chunk) })}\n\n`
    );
  });

  // add event listener to list to data on stderr
  runCommand.stderr.on("data", (data) => {
    console.error(`stderr : ${String(data)}`);

    let id = new Date().getTime();
    // write id to response
    response.write(`id: ${id}\n`);
    // set event
    response.write(`event: stderr\n`);
    // write data to response
    response.write(
      `data: ${JSON.stringify({ type: "strerr", out: String(data) })}\n\n`
    );
  });

  // add event listener to list to errors
  runCommand.once("error", (err) => {
    console.log(`Error executing command exited : ${err}`);
    // set event
    response.write(`event: err\n`);
    // write data to response
    response.write(
      `data: ${JSON.stringify({ type: "error", out: String(err) })}\n\n`
    );
    response.emit("close");
    response.end();
  });

  // when command is exit close connection by sending
  runCommand.once("exit", (code) => {
    console.log(`command exited code : ${code}`);
    // set event
    response.write(`event: exit\n`);
    response.write(`data: exited\n\n`);
    // set event
    response.end();
  });
  // kill long running command id client connection is close
  request.on("close", () => {
    console.log(`${request.socket.remotePort} Connection closed`);
    runCommand.kill();
  });
});

setup http server listen with express

// http server listen on given port
app.listen(PORT, () => {
  console.log(`App is listening on http://localhost:${PORT}`);
});

now lets run the application using

node app.js

for testing, I am using Postman.

SSE support for postman was introduced on version v10.10 and above.

postman blog

if everything goes well you will get below output. here we can see event data.

Important Note: It's crucial to understand that executing commands on a server without proper input sanitization can pose significant security risks.The code provided in this example is meant for educational purposes only.

Just like that, we build a server with SSE support.

Fun fact: Did you know that ChatGPT uses `SSE` to stream responses?

Git repo

codehirise / nodejs-react-sse-code-executor

Execute Terminal Commands And Receive Live Output With Nodejs + SSE + React

Read full article on

build backend : https://codehirise.com/post/building-a-real-time-command-execution-app-with-node-js-sse

build frontend :https://codehirise.com/post/execute-terminal-commands-and-receive-live-output-with-react-sse

View on GitHub

In the second part of this tutorial, we will build a client ui using react to create a realtime command execute application.

Aaaaaaand that's a wrap.