DEV Community: Precious Ogundipe

Setting Up Self-Hosted Runners As Pods

Precious Ogundipe — Mon, 09 Mar 2026 17:45:01 +0000

Automation is a big part of modern technology, and for DevOps engineers, managing CI/CD pipelines is a key responsibility. Our goal is to make sure these pipelines run fast, cost-efficiently, and securely when deploying applications.

Kubernetes pods can be used as self-hosted runners in CI/CD pipelines. Instead of relying entirely on shared runners or maintaining a dedicated server just for CI/CD tasks, this approach lets you control how build environments are created, scaled, and destroyed.

This improves scalability, resource efficiency and also helps ensure our applications are deployed more securely.

In this hands on project, ill be walking you through how to setup a self hosted runner as a pod on kubernetes using kind.

Prerequisites

Github Repository
Actions Runner Controller (ARC)
Kind
Kubectl
Cert Manager
Helm

🟣Kubectl

This is a Kubernetes command-line tool, that allows you to run commands against Kubernetes clusters.

You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. To install kubectl run this command on your terminal

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

## Install the binary file
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

## To check the kubectl has been installed 
kubectl version --client

🟣Helm

Helm is the package manager for Kubernetes, that helps you install applications using pre-packaged templates called charts. To install helm follow this step

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-4  
chmod 700 get_helm.sh  
./get_helm.sh

🟣Kind

This is a tool for running local Kubernetes clusters using Docker container as nodes. It is used for testing kubernetes applications locally. You can install kind following the steps here

🟣Cert Manager

This helps adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing and using those certificates.

It supports issuing certificates from a variety of sources, including Let's Encrypt (ACME), HashiCorp Vault, and CyberArk Certificate Manager, as well as local in-cluster issuance.

This is here because the runner controller uses admission webhooks, which requires valid TLS certificates for secure communication with the Kubernetes API server. cert-manager automatically generates and manages these certificates, ensuring the webhooks function properly without manual certificate management. We can install cert-manager on the cluster using the kubectl command

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.4/cert-manager.yaml

🟣Setting Up Actions Runner Controller (ARC)

This is a Kubernetes operator that orchestrates and scales self-hosted runners for GitHub Actions.

With ARC, you can create runner scale sets that automatically scale based on the number of workflows running in your repository, organization, or enterprise. To set this up on kind, we'll use helm to install the chart on our local node

NAMESPACE="github-runner"
helm install arc \
  --namespace "${NAMESPACE}" \
  --create-namespace \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set-controller

🟣Configure GitHub Authentication

To do this step, there are two options: you can either use a Personal Access Token or set up a GitHub App.

Going the Personal Access Token route requires rotating the token regularly for security purposes, so using a GitHub App is generally recommended for production environments. In this hands-on guide, we will demonstrate both methods, starting with the Personal Access Token approach then Github Apps

Personal Access Token

To set this up on your Github profile, go to Settings → Developer Settings → Personal access tokens → Tokens (classic), click on generate new tokens. Select the name of the token, For repository runners you usually need permissions like: repo; admin:repo_hook, then generate the token

Github Apps

To configure this, go to GitHub Settings → Developer Settings, select GitHub Apps, create a new GitHub Apps. Fill the necessary information like, App name; arc-runner-controller, Homepage URL; in this case https://github.com/PreciousDipe/Self-Hosted-Runner-On-Pods, uncheck the active button for Webhook URL, under permissions select repository permissions and choose these; Actions → Read, Metadata → Read, Administration → Read & Write, Click on create Github App. After creation you will get: App ID, Client ID.

To complete the step, generate a private key, This downloads a .pem file. That we will be used for authentication. Install the github app, ensure the app is installed

🟣Create a Secret

The next thing we need to do is to use the credentials we've created in the previous step to create a secret in kubernetes.

PAT Method

RUNNER_NS="arc-runners"
GITHUB_PAT="Your_Generated_PAT"

kubectl create secret generic arc-github-config \
  --namespace ${RUNNER_NS} \
  --from-literal=github_token="${GITHUB_PAT}"

Github Apps Method

RUNNER_NS="arc-runners"
APP_ID="Generated-App-ID"
INSTALLATION_ID="Generated-Installation-ID" (Click on your app on the installation page, and the installation ID will be the number at the end of the URL)
PATH_TO_PRIVATE_KEY=./path-to-pem-file

kubectl create secret generic controller-manager \
  --namespace ${NAMESPACE} \
  --from-literal=github_app_id="${APP_ID}" \
  --from-literal=github_app_installation_id="${INSTALLATION_ID}" \
  --from-file=github_app_private_key="${PATH_TO_PRIVATE_KEY}"

Create a Runner Scale Set

This tells Kubernetes how to spin up pods when jobs are queued, scale the number of runners up or down based on workload, and destroy pods once the jobs finish. we will be using helm to install the runner set from the gha-runner-scale-set helm chart

PAT Method

INSTALLATION_NAME="runner-set"   
NAMESPACE="${RUNNER_NS}"
GITHUB_CONFIG_URL="https://github.com/PreciousDipe/Self-Hosted-Runner-On-Pods"

helm install "${INSTALLATION_NAME}" \
  --namespace "${NAMESPACE}" \
  --create-namespace \
  --set githubConfigUrl="${GITHUB_CONFIG_URL}" \
  --set githubConfigSecret.github_token="${GITHUB_PAT}" \
  oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set

Github Apps Method

INSTALLATION_NAME="runner-set-github-apps"   
NAMESPACE="${RUNNER_NS}"
GITHUB_CONFIG_URL="https://github.com/PreciousDipe/Self-Hosted-Runner-On-Pods"

helm repo add actions-runner-controller https://actions-runner-controller.github.io/actions-runner-controller
"actions-runner-controller" has been added to your repositories

helm install "${INSTALLATION_NAME}" \
  actions-runner-controller/actions-runner-controller \
  --version 0.23.7 \
  --namespace "${NAMESPACE}" \
  --create-namespace \
  --set githubConfigUrl="${GITHUB_CONFIG_URL}" \
  --set githubConfigSecret.name=controller-manager

🟣Create the workflow file to test your configurations

name: Self Hosted Runners on Pods

on:
  push:
    branches: [ main ]

jobs:
  build:
    runs-on: runner-set-github-apps
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Show runner info
        run: |
          echo "This job runs on a self-hosted runner managed by actions runner controller"
          hostname
          cat /etc/os-release || true

Depending on the method you want to use, either it’s the PAT or GitHub Apps, just replace the runs-on with your Helm installation name. Here, we’re using the GitHub Apps method.

looking at the image above we can see 2 pods running in the cluster, that is the runner-set-dlxqc-runner-vkm65 pod also know as the controller pod, this pod manages the runner lifecycle, while the other pod runner-set-github-actions-runner-controller-7bc8bd578xxlxf is the listener pod, and this pod helps to establish a connection to GitHub, watches for new jobs in your repo, and requests an ephemeral runner pods when jobs arrive. These pods are "always-on" components.

Conclusion

We can see that the runners picked up the jobs on the pipeline and executed the workflow successfully. GitHub App configuration worked, Kubernetes spun up an ephemeral pod as a runner, which picked up the job from GitHub and executed it, achieving dynamic, secure CI/CD without relying on traditional self-hosted servers. Click here to check out the project on Github

References

Nice

Precious Ogundipe — Fri, 24 Oct 2025 14:02:11 +0000

Jess Lee for The DEV Team

Oct 22 '25

Join the Agentic Postgres Challenge with Tiger Data: $3,000 in Prizes!

#agenticpostgreschallenge #devchallenge #postgres #agents

130

Comments 19

3 min read

Automating User Creation with a Bash Script

Precious Ogundipe — Fri, 24 Oct 2025 13:59:46 +0000

As a DevOps Engineer, writing automation scripts is an essential part of your job. These scripts help streamline repetitive tasks and increase efficiency. In this task, we must create a bash script that reads a text file containing employees' usernames and group names, where each line is formatted as user; groups. The script must create the specified users and groups, set up home directories with the appropriate permissions and ownership, generate random passwords for each user, and log all actions to /var/log/user_management.log. Additionally, it should securely store the generated passwords in /var/secure/user_passwords.txt and implement error handling to manage scenarios where users already exist, all while providing clear documentation and comments within the script.

Requirements

Linux Machine
Bash Script

Steps

sudo nano create_users.sh

creates a file to write the script

#!/bin/bash

# Ensure script is run as root
if [[ $EUID -ne 0 ]]; then
    echo "This script must be run as root" >&2
    exit 1
fi

# Check if input file is provided
if [[ -z "$1" ]]; then
    echo "Usage: $0 <user-list-file>"
    exit 1
fi

USER_FILE="$1"
LOG_FILE="/var/log/user_management.log"
PASSWORD_FILE="/var/secure/user_passwords.csv"

# Ensure log and password directories exist
mkdir -p /var/secure
touch "$LOG_FILE" "$PASSWORD_FILE"
chmod 600 "$PASSWORD_FILE" 

# Process each line in the file
while IFS=';' read -r username groups; do
    username=$(echo "$username" | xargs)  
    groups=$(echo "$groups" | xargs)  

    # Skip empty lines
    [[ -z "$username" ]] && continue

    # Creates groups if it doesn't exist
    if ! getent group "$username" >/dev/null; then
        groupadd "$username"
        echo "Created group: $username" | tee -a "$LOG_FILE"
    fi

    # Creates a user with and adds them to their group
    if ! id "$username" &>/dev/null; then
        useradd -m -g "$username" -s /bin/bash "$username"
        echo "Created user: $username" | tee -a "$LOG_FILE"
    else
        echo "User $username already exists, skipping..." | tee -a "$LOG_FILE"
        continue
    fi

    # Assign users to additional groups
    if [[ -n "$groups" ]]; then
        IFS=',' read -ra GROUPS_ARRAY <<< "$groups"
        for group in "${GROUPS_ARRAY[@]}"; do
            group=$(echo "$group" | xargs)  
            if ! getent group "$group" >/dev/null; then
                groupadd "$group"
                echo "Created group: $group" | tee -a "$LOG_FILE"
            fi
            usermod -aG "$group" "$username"
            echo "Added $username to group: $group" | tee -a "$LOG_FILE"
        done
    fi

    # Generate a random password
    password=$(openssl rand -base64 8)
    echo "$username,$password" >> "$PASSWORD_FILE"
    echo "Generated password for $username" | tee -a "$LOG_FILE"

    # Set password
    echo "$username:$password" | chpasswd

done < "$USER_FILE"

echo "User creation process completed." | tee -a "$LOG_FILE"
exit 0

explaining each section of the bash script

#!/bin/bash

This section specifies that the script should be executed using the Bash shell

if [[ $EUID -ne 0 ]]; then
    echo "This script must be run as root" >&2
    exit 1
fi

This section ensures the script is executed with root privileges. Without root, actions like creating users and groups will fail.

if [[ -z "$1" ]]; then
    echo "Usage: $0 <user-list-file>"
    exit 1
fi

This section, verifies that the user provided the input file which contains the usernames and groups.

USER_FILE="$1"
LOG_FILE="/var/log/user_management.log"
PASSWORD_FILE="/var/secure/user_passwords.csv"

mkdir -p /var/secure
touch "$LOG_FILE" "$PASSWORD_FILE"
chmod 600 "$PASSWORD_FILE"

This section sets file paths for the input file, log file, and password file, creates necessary directories and files, and sets secure permissions on the password file.

while IFS=';' read -r username groups; do
    username=$(echo "$username" | xargs)  
    groups=$(echo "$groups" | xargs)      

    # Skip empty lines
    [[ -z "$username" ]] && continue

This section reads the input file line by line, using ; as a delimiter to separate the username from the groups, and trims any extra whitespace.

    if ! getent group "$username" >/dev/null; then
        groupadd "$username"
        echo "Created group: $username" | tee -a "$LOG_FILE"
    fi

This section checks if a group with the same name as the username exists. If not, it creates the group and logs the action.

    if ! id "$username" &>/dev/null; then
        useradd -m -g "$username" -s /bin/bash "$username"
        echo "Created user: $username" | tee -a "$LOG_FILE"
    else
        echo "User $username already exists, skipping..." | tee -a "$LOG_FILE"
        continue
    fi

This section creates the user with the home directory, assigning them to their group. If the user already exists, it displays a message that the user already exists and skips further processing for that user.

    if [[ -n "$groups" ]]; then
        IFS=',' read -ra GROUPS_ARRAY <<< "$groups"
        for group in "${GROUPS_ARRAY[@]}"; do
            group=$(echo "$group" | xargs) 
            if ! getent group "$group" >/dev/null; then
                groupadd "$group"
                echo "Created group: $group" | tee -a "$LOG_FILE"
            fi
            usermod -aG "$group" "$username"
            echo "Added $username to group: $group" | tee -a "$LOG_FILE"
        done
    fi

This section processes additional groups by splitting the string on commas, creates them if they don't exist, adds the user to each, and logs the operations

    password=$(openssl rand -base64 12)
    echo "$username,$password" >> "$PASSWORD_FILE"
    echo "Generated password for $username" | tee -a "$LOG_FILE"

    # Set password
    echo "$username:$password" | chpasswd

This section generates a secure random password, appends the username and password to the secure password file, and sets the generated password for the user.

echo "User creation process completed." | tee -a "$LOG_FILE"
exit 0

This section logs a message that the process is finished and exits the script successfully.

sudo nano users.txt

This command helps opens a file where the script will be written.

light; sudo,dev,www-data
idimma; sudo
mayowa; dev,www-data
john; admin,dev
emma; sudo,www-data
alice; dev
bob; admin,sudo
charlie; www-data
dave; dev,admin
eve; sudo

Input this in the file

sudo bash create_users.sh users.txt

Use this command to run the script

sudo cat /var/log/user_management.log
sudo cat /var/secure/user_passwords.csv

Use this command to view the content of the files created to ensure the script worked

Building a Nodejs API using the Twelve-Factor App Principles

Precious Ogundipe — Sun, 13 Apr 2025 22:33:38 +0000

I recently started learning to write APIs following the best practices and design. This research led me to The Twelve-Factors, which was introduced by Heroku co-founder Adam Wiggins in 2011. The concept focuses on the best practices for writing SaaS applications, which involve building application systems that are easy to set up, portable across environments, consistent from development to production, scalable, and cloud-ready.
Knowing this concept makes it easier for developers to plan how to build their applications using a tested blueprint. It can also be applied to applications written in any programming language.

The Twelve-Factors

Codebase
Dependencies
Config
Backing Services
Build, release, run
Processes
Port binding
Concurrency
Disposability
Dev/prod parity
Logs
Admin processes

To explain all these concepts, I'll be using a student-api project I created using Nodejs and express.js to explain all these concepts.

Codebase: Version control systems, such as Git, should always be used to track applications. This idea only clarifies that a version control system should be used to maintain a single codebase for every application. For this part, set up a new repository for your application and, if you don't already have one, register a GitHub account.
Dependencies: This concept entails declaring all packages and dependencies in a packaging system, specifically the Node Package Manager (NPM). It reduces the need for system-wide installations and makes it easier for new developers to set up the application..

### Create a new directory for your project and initialize the project
mkdir student-api 
cd student-api 
npm init -y

### Install the required packages
npm install express sqlite3 knex dotenv joi morgan
npm install --save-dev jest supertest nodemon

The npm init—y command simply creates a package.json file, where all our dependencies and their specific versions will be declared.

An example of how your package.json file should look

Config: Using configuration files such as API keys, database URLs, and other sensitive data as constants in your code is against the Twelve-Factor rules. Instead, keep them as environment variables and use them in your code. In this case, I utilized the dotenv library to access the '.env' file. touch index.js

const express = require('express');
require('dotenv').config();
const morgan = require('morgan');
const studentRouter = require('./routes/students');

const app = express();

app.use(morgan('dev'));
app.use(express.json());

app.use('/api/v1/students', studentRouter);

app.get ('/api/v1/healthcheck', (req, res) => {
    res.json({
        status: 'ok'
    });
});

module.exports = app;

if (require.main === module) {
    const PORT = process.env.PORT || 3000;
    app.listen(PORT, () => {
        console.log(`Server is running on port ${PORT}`);
    });
}

Backing Services: These services are treated as attached resources configured to be loosely coupled. They include databases, messaging/queuing, caching, and SMTP services. For my student-api, I utilized SQLite as a database and [Knex](https://knexjs.org/guide/migrations.html) for database migrations.

### create the database configuration file
touch knexfile.js
### create a database connection
touch db.js
#### defines CRUD operations for the students table
mkdir models
cd models
touch students.js
#### defines the REST API endpoints with validation
mkdir routes
cd routes
touch students.js
#### create a migration to set up the students' table
npx knex migrate:make create_students_table
#### run the migration
npx knex migrate:latest

Knex is a database query builder and migration tool in Node.js.
knex.js

require('dotenv').config();

module.exports = {
  development: {
    client: 'sqlite3',
    connection: {
      filename: process.env.DB_FILE || './students.db'
    },
    useNullAsDefault: true
  }
};

db.js

const knex = require('knex');
const config = require('./knexfile');

const env = process.env.NODE_ENV || 'development';
const db = knex(config[env]);

module.exports = db;

models/students.js

const db = require('../db');

const Student = {
  create: async (data) => db('students').insert(data).returning('*'),
  findAll: async () => db('students').select('*'),
  findById: async (id) => db('students').where({ id }).first(),
  update: async (id, data) => db('students').where({ id }).update(data).returning('*'),
  delete: async (id) => db('students').where({ id }).del()
};

module.exports = Student;

routes/students.js

const express = require('express');
const router = express.Router();
const Student = require('../model/students');
const Joi = require('joi');

const studentSchema = Joi.object({
  name: Joi.string().required(),
  age: Joi.number().integer().required(),
  grade: Joi.string().required()
});

// Create a Student
router.post('/', async (req, res) => {
    try {
      console.log('Request body:', req.body); // Debug log
      const { error } = studentSchema.validate(req.body);
      if (error) {
        console.log('Validation error:', error.details); // Debug validation
        return res.status(400).json({ error: error.details[0].message });
      }
      const student = await Student.create(req.body);
      res.status(201).json(student[0]);
    } catch (error) {
      console.error('Error in POST /students:', error); // Detailed error log
      res.status(500).json({ error: error.message });
    }
  });

// Get all Students
router.get('/', async (req, res) => {
  try {
    const student = await Student.findAll();
    res.json(student);
  } catch (error) {
    console.error(error); // log the error
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

// Get a student by ID
router.get('/:id', async (req, res) => {
  try {
    const student = await Student.findById(req.params.id);
    if (student) res.json(student);
    else res.status(404).json({ error: 'Student not found' });
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});


// Update a student
router.put('/:id', async (req, res) => {
  try {
    if (!req.params.id) return res.status(400).json({ error: 'Missing ID parameter' });
    const { error } = studentSchema.validate(req.body, { allowUnknown: true });
    if (error) return res.status(400).json({ error: error.details[0].message });
    const student = await Student.update(req.params.id, req.body);
    if (student.length > 0) res.json(student[0]);
    else res.status(404).json({ error: 'Student not found' });
  } catch (error) {
    console.error(error); // log the error
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

// Delete a student
router.delete('/:id', async (req, res) => {
  try {
    if (!req.params.id) return res.status(400).json({ error: 'Missing ID parameter' });
    const student = await Student.delete(req.params.id);
    if (student > 0) res.status(204).end();
    else res.status(404).json({ error: 'Student not found' });
  } catch (error) {
    console.error(error); // log the error
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

module.exports = router;

Joi is a JavaScript library that is used to validate and sanitize data input from clients.

migrations/_create_students_table.js, paste the code below in that file to create the student table in the database

exports.up = function(knex) {
  return knex.schema.createTable('students', (table) => {
    table.increments('id').primary();
    table.string('name').notNullable();
    table.integer('age').notNullable();
    table.string('grade').notNullable();
  });
};

exports.down = function(knex) {
  return knex.schema.dropTable('students');
};

migrated the studentsdb using knex

Build, release, run: This concept describes how programs should keep the build, release, and run stages separate. In the student-api, I used 'Make' for the build/run phases; however, for the release, you can create a CI/CD pipeline.

### install make using npm
npm install make
### create a makefile
touch makefile

install:
    npm install

migrate:
    npx knex migrate:latest

run:
    npm start

test:
    npm test

Processes: Any data that must be persistent must be kept in a stateful backing service, such as a database; application processes should be stateless and not share anything. Our application stores data in a SQLite database, so our application can safely be restored without losing data.
Port binding: Expose applications on a dedicated port, ensuring that it is self-contained and easy to deploy. In our case, our app is exposed on port 3000 using Express.
Concurrency: This process talks about how applications should be able to handle traffic by running multiple instances of the application. This can be done using tools like Kubernetes or PM2.
Disposability: Twelve-factor applications are architected to handle unexpected, non-graceful terminations. This means they can be started or stopped at a moment’s notice.
Dev/prod parity: Keep development, staging, and production as similar as possible. This means that applications used in different environments should be the same or very similar to avoid bugs.
Logs: Treat logs as event streams and output them to stdout. Use log aggregation and monitoring tools to analyze logs and detect issues. For the API application, I used Morgan to log HTTP requests to the console, which can be captured and analyzed separately.

Morgan outputting the application's Logs

Admin processes: Admin processes should be run in an identical environment as the regular long-running processes of the application. This includes processes like database migration, running one-time scripts, and all. For the API application, database migrations are handled via [knex](https://knexjs.org/guide/migrations.html), separate from the main application process.

Deploying a FastAPI Application with CI/CD Pipeline: HNG Task 2

Precious Ogundipe — Tue, 11 Feb 2025 06:58:28 +0000

Introduction

In this article, I will walk you through how I deployed a FastAPI application with continuous integration (CI) and continuous deployment (CD).

Requirements

Python
FastAPI Application
Docker
GitHub Actions Secrets
AWS Linux server

Step 1: Clone the Repository

The first step is to fork the repo and clone the repository to your local device
git clone https://github.com/hng12-devbotops/fastapi-book-project.git
cd fastapi-book-project/

Step 2: Setting Up a Virtual Environment

I am testing the application locally before deploying, using a virtual environment is the best practice to manage your Python dependencies. To do this you need to create and activate a virtual environment.
python3 -m venv hng
source hng/bin/activate

install the required dependency using this command

pip install -r requirements.txt

Step 3: Implementing the Missing Endpoint

Our API is currently unable to fetch a single book using its ID. In this step, I added a new endpoint to the FastAPI router that handles a GET request to retrieve a book based on its provided ID.
Add this code to the books.pyfile

@router.get("/{book_id}", response_model=Book, status_code=status.HTTP_200_OK)  # New endpoint
async def get_book(book_id: int):
    book = db.books.get(book_id)
    if not book:
        raise HTTPException(
            status_code=status.HTTP_404_NOT_FOUND, 
            detail="Book not found"
        )
    return book

Ensure to import HTTPException from fastapi

Step 4: Testing the Endpoint

Once you've implemented the endpoint, it's important to test it to ensure it behaves as expected, I did this by running this command
pytest

I then started the FastAPI application using Uvicorn uvicorn main:app --reload

The applications seem to be running fine on our localhost

Step 5: Dockerizing Our Application

create a Dockerfile in the project directory

# Use an official lightweight Python image
FROM python:3.9-slim

# Set work directory
WORKDIR /app

# Install dependencies
COPY requirements.txt .
RUN pip install --upgrade pip
RUN pip install -r requirements.txt

# Copy project files
COPY . .

# Expose the port FastAPI runs on
EXPOSE 8000

# Run the FastAPI application using Uvicorn
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Step 6: Integrating Nginx as a Reverse Proxy

To make things easier for me, I used a docker-compose.yml file to configure Nginx as a reverse proxy. To do this I created an nginx.conf file with the configurations for my application. The purpose of this configuration is to have Nginx listen on port 80 and forward incoming HTTP requests to the FastAPI application running on port 8000 inside its container.
nginx.conf

server {
    listen 80;
    server_name your_domain_or_IP;
    location / {
        proxy_pass http://app:8000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For 
        $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

listen 80: Nginx listens for incoming HTTP requests on port 80.
server_name: Replace with your domain or IP address.
proxy_pass: Directs traffic to the service named app (defined in the docker-compose file) on port 8000.

docker-compose.yml

version: '3.8'
services:
  app:
    build: .
    container_name: fastapi-app
    ports:
      - "8000:8000"

  nginx:
    image: nginx:latest
    container_name: fastapi-nginx
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - "80:80"
    depends_on:
      - app

The app service builds and runs our FastAPI application.
The Nginx service uses an Nginx image, mounts the custom configuration file, and maps port 80 of the container to port 80 on the host.
The depends_on command ensures that the FastAPI container starts before Nginx.
Then I used this command to build the containers
docker-compose up --build

As you can see, our containers are up and running and Nginx is successfully routing incoming traffic to our FastAPI application.

Step 7: Create a Linux Server

For hosting and deploying our FastAPI application, I set up a Linux server on AWS. This server serves as the backbone of our deployment environment, where our Dockerized application will run and be accessible to users.

Step 8: Set Up the CI Pipeline

I created a GitHub Actions Workflow File for this step to automatically run tests on every pull request targeting the main branch. This ensures that any new changes are validated before they can be merged.
mkdir -p .github/workflows/
cd .github/workflows/
touch ci.yml

name: ci pipeline

on:
  pull_request:
    branches: [ main ]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.9'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run tests
      run: pytest

push your changes to GitHub and check the status of your pipeline

Step 9: Set Up the Deployment Pipeline

For continuous deployment, I created another GitHub Actions Workflow File named deploy.yml. This deployment pipeline is triggered whenever changes are pushed to the main branch, automating the process of updating the live application.
touch deploy.yml

name: CD Pipeline
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3

      - name: Deploy via SSH
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.SSH_HOST }}
          username: ${{ secrets.SSH_USERNAME }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            # Update package index and install dependencies
            sudo apt-get update -y
            sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common

            # Add Docker's official GPG key
            curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
            sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

            # Install Docker
            sudo apt-get update -y
            sudo apt-get install -y docker-ce docker-ce-cli containerd.io

            # Add the SSH user to the Docker group
            sudo usermod -aG docker ${{ secrets.SSH_USERNAME }}

            # Install Docker Compose
            sudo curl -L "https://github.com/docker/compose/releases/download/v2.20.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
            sudo chmod +x /usr/local/bin/docker-compose

            # Verify installations
            docker --version
            docker-compose --version

            # Navigate to the project directory and deploy
            cd /home/ubuntu/
            git clone <your-github-repo>
            cd fastapi-book-project/
            git fetch origin main
            git reset --hard origin/main
            docker-compose up -d --build

Ensure your secrets are stored: Make sure SSH_HOST, SSH_USERNAME, and SSH_PRIVATE_KEY are added to your GitHub repository's secrets.
Push your changes to GitHub and check the status of your pipeline.
Post-Deployment Verification: Once the pipeline completes, verify that your application is running as expected.

Step 10: Repository Access Requirement

Before submission, we are supposed to invite the hng12-devbotops to our GitHub account as a collaborator in our repository. To do this;

Open the repository you want to add a collaborator
Click on the Settings tab, which is usually located on the right side of the repository menu.
In the left sidebar, under access, select Collaborators.
Click on add people, add hng12-devbotops

Thanks for reading, don't forget to drop a like and comment 😄😄
👩🏻‍💻👩🏻‍💻

Checkout my other article on how to create a Linux Server on AWS

Building a Number Classification API on Lambda

Precious Ogundipe — Sun, 02 Feb 2025 23:03:05 +0000

Introduction

In this, tutorial I will walk you through how I built a Number Classification API. This API allows users to input a number and get details about the number. I'll cover both the frontend and backend parts of the project, explaining how each piece of the system works and how they interact.

Technical Architecture

Step 1: Creating the Frontend for Input

The first thing I did was create the frontend UI to input the number. The frontend is essential because users need an interface to interact with the API. I used HTML and JavaScript to build a simple form where users can enter numbers.

You can check out my frontend code here

Step 2: Hosted My Frontend on GitHub Pages

For this project, I wanted a seamless user experience by allowing users to input a number and see the results from the API. This is where GitHub Pages came into play. GitHub Pages is a free service that allows you to host static websites directly from a GitHub repository. Here's how I used it for the frontend part of my API.

Steps for hosting on GitHub Pages:

Create a GitHub repository for the frontend files.
Push the HTML, CSS, and JavaScript files to the repository.
Enable GitHub Pages under the repository settings.
Choose the branch (main) and the root directory for serving the files.
GitHub will provide a URL to access the hosted frontend.

Step 3: Setting Up the Backend with AWS Lambda

Once the frontend was ready, I moved on to the backend. For the backend, the first thing I did was to create a Lambda function called numberapi, which uses node.js version 22 as runtime which would be used to process the request from the frontend. The Lambda function accepts the number, performs calculations, and returns the results.

here is my lambda function code;

// Utility function: Check if a number is prime
function isPrime(num) {
  if (num < 2) return false;
  for (let i = 2; i <= Math.sqrt(num); i++) {
      if (num % i === 0) return false;
  }
  return true;
}

// Utility function: Check if a number is perfect
function isPerfect(num) {
  if (num < 1) return false;
  let sum = 0;
  for (let i = 1; i <= num / 2; i++) {
      if (num % i === 0) sum += i;
  }
  return sum === num;
}

// Utility function: Check if a number is an Armstrong number
function isArmstrong(num) {
  const digits = num.toString().split('').map(Number);
  const power = digits.length;
  const sum = digits.reduce((acc, digit) => acc + Math.pow(digit, power), 0);
  return sum === num;
}

// Utility function: Calculate the sum of digits
function digitSum(num) {
  // Handle negative numbers by using the absolute value
  num = Math.abs(num);
  return num.toString().split('').reduce((acc, digit) => acc + Number(digit), 0);
}

// AWS Lambda handler function
export const handler = async (event) => {
  const query = event.queryStringParameters || {};
  const numberParam = query.number;
  const parsedNumber = parseInt(numberParam, 10);

  const headers = {
      "Content-Type": "application/json",
      "Access-Control-Allow-Origin": "*"
  };

  // Validate input: if not a valid integer, return 400 with error JSON
  if (isNaN(parsedNumber)) {
      return {
          statusCode: 400,
          headers,
          body: JSON.stringify({ number: numberParam, error: true })
      };
  }

  const prime = isPrime(parsedNumber);
  const perfect = isPerfect(parsedNumber);
  const armstrong = isArmstrong(parsedNumber);
  const parity = (parsedNumber % 2 === 0) ? "even" : "odd";

  let properties = [];
  if (prime) {
      properties.push("prime");
  }
  if (perfect) {
      properties.push("perfect");
  }
  if (armstrong) {
      properties.push("armstrong");
  }

  // Add parity (odd/even)
  properties.push(parity);

  const sum = digitSum(parsedNumber);

  let fun_fact = "";
  try {
      const response = await fetch(`http://numbersapi.com/${parsedNumber}/math`);
      fun_fact = await response.text();
  } catch (error) {
      fun_fact = "Fun fact unavailable at this time.";
  }

  const result = {
      number: parsedNumber,
      is_prime: prime,
      is_perfect: perfect,
      properties: properties,
      digit_sum: sum,
      fun_fact: fun_fact
  };

  return {
      statusCode: 200,
      headers,
      body: JSON.stringify(result)
  };
};

Step 4: Deploying the Lambda Function with API Gateway

After creating the Lambda function, the next step was to set up an API Gateway to allow communication between the frontend and the Lambda function.

Create a REST API in API Gateway.
Add a GET endpoint (/classify-number) that triggers the Lambda function when called.
Set up CORS so that the frontend can make requests to the API.

If you go back to your lambda function you'll see that API GATEWAY has been added automatically

You can check the API app here

Setting Up Nginx on AWS EC2

Precious Ogundipe — Wed, 29 Jan 2025 15:34:33 +0000

As a DevOps Engineer, one of the first things you should learn is how to run applications on a server, using webservers like Nginx or Apache. These tools are essential for serving web content, managing traffic, and ensuring your applications are accessible to users. In this article, I will walk you through configuring Nginx to serve applications, a critical skill for anyone starting their DevOps journey.
Nginx is a web server that helps handle traffic loads while maintaining low resource usage, it can also function as a reverse proxy, load balancer, and HTTP cache.

Technical Architecture

Requirements

Linux Server
SSH key
Nginx
Web Application Code

Steps

Create a Linux Server on any cloud provider of your choice, for this article, I used the AWS cloud provider and launched an EC2 instance running on Ubuntu Server 24.04 LTS.
When the instance is provisioned, ssh into the instance following these steps;

mv keys.pem ~/.ssh
chmod 400 ~/.ssh/keys.pem
ssh -i ~/.ssh/keys.pem ubuntu@<server-ip>

The next thing I did was to update the packages and install NGINX on the server using these commands;

sudo apt update
sudo apt install nginx -y

I then moved to the directory my application will be running in to edit the code using these commands;

cd /var/www/html
sudo nano index.html

Nginx configuration files are located in this directory '/etc/nginx/' in this file 'nginx.conf', but we will be using the configuration of the file in this directory '/etc/nginx/sites-available/default'

cd /etc/nginx/sites-available
sudo nano default

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
</head>
<body>
<h1>Welcome to DevOps Stage 0 -_ [Your Name]/[SlackName]</h1>
</body>
</html>

I used these commands to ensure nginx is enabled and running on the server;

sudo systemctl start nginx
sudo systemctl enable nginx
sudo systemctl status nginx

Then to ensure my application was running properly on the server, I went to the IP address of the server

http://<your-server-ip>/

Personal Growth

This task was pretty straightforward for me and I'll say the only challenge I faced was writing this article, I sort of overthought how I'd approach writing this article, but after doing a bit of research and reading some other people's articles, I found my direction and wrote it confidently. At the end of the day, it's all about how you approach a problem, break it down, and find the right way to solve it. This experience reinforced the importance of adaptability and learning from others while staying true to my thought process.

Looking to Hire the Best Engineers
HNG connects companies with the best engineers suited to their needs. Looking for top-tier talent to handle your DevOps, cloud infrastructure, and CI/CD tasks, HNG has the best developers available for hire and freelance jobs anywhere in the world.

Explore specialized talents available for hire:
DevOps Engineers
Cloud Engineers

DEV Community: Precious Ogundipe

Setting Up Self-Hosted Runners As Pods

Prerequisites

🟣Kubectl

🟣Helm

🟣Kind

🟣Cert Manager

🟣Setting Up Actions Runner Controller (ARC)

🟣Configure GitHub Authentication

Personal Access Token

Github Apps

🟣Create a Secret

Create a Runner Scale Set

🟣Create the workflow file to test your configurations

Conclusion

References

Nice

Join the Agentic Postgres Challenge with Tiger Data: $3,000 in Prizes!

Automating User Creation with a Bash Script

Requirements

Steps

Building a Nodejs API using the Twelve-Factor App Principles

The Twelve-Factors

Further Reading

Deploying a FastAPI Application with CI/CD Pipeline: HNG Task 2

Introduction

Requirements

Step 1: Clone the Repository

Step 2: Setting Up a Virtual Environment

Step 3: Implementing the Missing Endpoint

Step 4: Testing the Endpoint

Step 5: Dockerizing Our Application

Step 6: Integrating Nginx as a Reverse Proxy

Step 7: Create a Linux Server

Step 8: Set Up the CI Pipeline

Step 9: Set Up the Deployment Pipeline

Step 10: Repository Access Requirement

Building a Number Classification API on Lambda

Introduction

Technical Architecture

Step 1: Creating the Frontend for Input

Step 2: Hosted My Frontend on GitHub Pages

Steps for hosting on GitHub Pages:

Step 3: Setting Up the Backend with AWS Lambda

Step 4: Deploying the Lambda Function with API Gateway

Setting Up Nginx on AWS EC2

Technical Architecture

Steps

Personal Growth