DEV Community: EphraimX

How to Track and Manage Your Cloud Infrastructure Costs With Infracost

EphraimX — Mon, 18 Aug 2025 02:36:36 +0000

It's the beginning of a month, and you receive an email in your inbox detailing your AWS bill for the previous month. You see how much you're charged, and you're shocked. You begin to ask several questions: How did I spend this much? What services did I use that cost this much? Did I forget to run terraform destroy? Which you’re pretty sure you did… but the bill says otherwise. You sigh, still puzzled over how you ended up with such a huge charge.

If this sounds familiar, you’re not alone. Whether you’re a beginner, a seasoned cloud engineer, or somewhere in between, almost everyone has faced this kind of surprise at least once. You can handle this surprise by using various cost monitoring tools, either provided by the cloud vendor or third parties, but in this article, we will focus on cost monitoring using Infracost.

What is Infracost

Infracost is a cost monitoring solution that helps teams and companies shift cost left. In simple terms, Infracost gives you visibility into the cost of your infrastructure before you deploy to the cloud, instead of waiting until the end of the month to see what your infrastructure costs you.

This means you manage your cloud costs proactively rather than reactively. This allows you to balance your infrastructure needs with cost right from the onset, rather than deploying and waiting to find out your cost before attempting to optimize. And more importantly, this approach means you can make informed decisions, and this helps you to be more holistic in your approach, balancing both technical and business needs.

Why Use Infracost

The major challenge with cloud costs is visibility. When working with cloud infrastructure, you often don't know how much something will cost until it's deployed; by then, it's already too late, as the resources are running, and the bill is counting.

On reading this, you might say, you'll just make estimates using AWS or Azure calculator. And while you're right on this, it's a manual process and consumes valuable time that can be spent on activities much more profitable. Also, when using these tools, chances are you might leave out a resource or not account for a "hidden" charge. By hidden charge, I mean a charge that can go unnoticed.

A good example of this would be in a previous project where I built an end-to-end DevOps pipeline. In this project, I needed to use a NAT gateway to allow the resources in the private subnets to connect to the internet. I did my research and saw what fees I would incur for including it in my architecture. Imagine my surprise when I saw my bill and discovered I would also have to pay for IPv4-related charges, which I did not initially account for.

These reasons, among many others, are the reasons why Infracost matters. With Infracost, you see the cost of infrastructure ahead of time, so no surprises. You weigh tradeoffs between instance types, storage classes, or scaling strategies, allowing you to make smarter decisions. Lastly, costs show up in pull requests so everyone sees the financial impact of infrastructure changes.

Getting Started With Infracost

Installing Infracost

Installing Infracost depends on your operating system, but if you use a Linux operating system (like me), you can install Infracost by running the command below:

# Downloads the CLI based on your OS/arch and puts it in /usr/local/bin
curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh

For other operating systems, you can find your respective commands here.

Creating an Account on Infracost

After installing Infracost, you need to create an account with Infracost. You can sign up here, either with GitHub, Google, or email and password.

On signing up, select your "Source control integrations", depending on your Version Control System (VCS), and click on "Connect".

You will be directed to install Infracost on your repository, either all or some. For security purposes, it is advisable to select only the repositories that you want Infracost to have access to. Once done, you confirm access by inputting your MFA code or password.

N.B.: If you do not have a Terraform project to practice with, you can fork an already existing Terraform project in my repo here

Back to Infracost, the next step is to decide how you want to "Add your code repos". You could choose to add all current and future Terraform repos automatically, which is recommended. You could choose to add it based on a wildcard, or finally add it manually. I'll go with the recommended version, but if you're hard on security, add your code repos manually.

In the next sections, you'll see how to work with Infracost in pull requests and when working with Terraform.

How Infracost Works in Pull Requests

For this section, create a new branch in your repository; a test repo will work fine for this, or clone the repository linked above. Ensure this repository is accessible by Infracost.

Next, create a file called infracost_test.tf and paste in the following code:

provider "aws" {
  region = "us-east-1"
  skip_credentials_validation = true
  skip_requesting_account_id = true
  access_key = "mock_access_key"
  secret_key = "mock_secret_key"
}

resource "aws_instance" "my_web_app" {
  ami = "ami-005e54dee72cc1d00"

  instance_type = "m3.xlarge" # <<<<<<<<<< Try changing this to m5.xlarge to compare the costs

  tags = {
    Environment = "production"
    Service = "web-app"
  }

  root_block_device {
    volume_size = 1000 # <<<<<<<<<< Try adding volume_type="gp3" to compare costs
  }
}

resource "aws_lambda_function" "my_hello_world" {
  runtime = "nodejs12.x"
  handler = "exports.test"
  image_uri = "test"
  function_name = "test"
  role = "arn:aws:ec2:us-east-1:123123123123:instance/i-1231231231"

  memory_size = 512
  tags = {
    Environment = "Prod"
  }
}

N.B.: This code is from Infracost's official "Get Started"

Once created, commit your changes, and head to the branch's main page.

Next, open a pull request by clicking on "Compare and Pull Request" or "Contribute" on the right-hand side of the page.

Put in your desired description and create your pull request

On creating a pull request, Infracost scans your Terraform files and points out inconsistencies such as tagging policies and guardrails, as seen below.

How Infracost Works with Terraform

For this section, create or clone a repository with Terraform files available, and ensure you have Terraform installed on your local machine.

To start, change into your project's root directory:

cd end-to-end-devops-pipeline-using-aws-docker-and-terraform

Next, get your Infracost API key. If you don't have it, you can generate it by clicking "Settings", then clicking on "Org Settings".

Then, to get your API, copy the already generated token for CLI and CI/CD operations.

After getting your API key, the next step is to configure your CLI. You can do this by running:

infracost configure set api_key {API_KEY}

Replace {API_KEY} with your actual API key.

Once completed, you can get your Terraform infrastructure costs by running:

infracost breakdown --path .

And here's the result:


Project: main

 Name                                                          Monthly Qty  Unit                        Monthly Cost   

 aws_instance.my_web_app                                                                                               
 ├─ Instance usage (Linux/UNIX, on-demand, m3.xlarge)                  730  hours                            $194.18   
 └─ root_block_device                                                                                                  
    └─ Storage (general purpose SSD, gp2)                            1,000  GB                               $100.00   

 aws_lambda_function.my_hello_world                                                                                    
 ├─ Requests                                           Monthly cost depends on usage: $0.20 per 1M requests            
 ├─ Ephemeral storage                                  Monthly cost depends on usage: $0.0000000309 per GB-seconds     
 └─ Duration (first 6B)                                Monthly cost depends on usage: $0.0000166667 per GB-seconds     

 Project total                                                                                               $294.18   

──────────────────────────────────
Project: terraform
Module path: terraform

 Name                                                                   Monthly Qty  Unit              Monthly Cost   

 aws_instance.roi_calculator_bastion_host_ec2_public_subnet_one                                                       
 ├─ Instance usage (Linux/UNIX, on-demand, t3.xlarge)                           730  hours                  $121.47   
 └─ root_block_device                                                                                                 
    └─ Storage (general purpose SSD, gp2)                                         8  GB                       $0.80   

 aws_instance.roi_calculator_bastion_host_ec2_public_subnet_two                                                       
 ├─ Instance usage (Linux/UNIX, on-demand, t3.xlarge)                           730  hours                  $121.47   
 └─ root_block_device                                                                                                 
    └─ Storage (general purpose SSD, gp2)                                         8  GB                       $0.80   

 aws_instance.roi_calculator_production_host_ec2_private_subnet_one                                                   
 ├─ Instance usage (Linux/UNIX, on-demand, t3.xlarge)                           730  hours                  $121.47   
 └─ root_block_device                                                                                                 
    └─ Storage (general purpose SSD, gp2)                                         8  GB                       $0.80   

 aws_instance.roi_calculator_production_host_ec2_private_subnet_two                                                   
 ├─ Instance usage (Linux/UNIX, on-demand, t3.xlarge)                           730  hours                  $121.47   
 └─ root_block_device                                                                                                 
    └─ Storage (general purpose SSD, gp2)                                         8  GB                       $0.80   

 aws_nat_gateway.roi_calculator_ngw_private_subnet_one                                                                
 ├─ NAT gateway                                                                 730  hours                   $32.85   
 └─ Data processed                                                   Monthly cost depends on usage: $0.045 per GB     

 aws_nat_gateway.roi_calculator_ngw_private_subnet_two                                                                
 ├─ NAT gateway                                                                 730  hours                   $32.85   
 └─ Data processed                                                   Monthly cost depends on usage: $0.045 per GB     

 aws_lb.roi_calculator_aws_lb                                                                                         
 ├─ Application load balancer                                                   730  hours                   $16.43   
 └─ Load balancer capacity units                                     Monthly cost depends on usage: $5.84 per LCU     

 aws_db_instance.roi_calculator                                                                                       
 ├─ Database instance (on-demand, Single-AZ, db.t3.micro)                       730  hours                   $13.14   
 └─ Storage (general purpose SSD, gp2)                                            5  GB                       $0.58   

 Project total                                                                                              $584.93   

 OVERALL TOTAL                                                                                             $879.11 

*Usage costs can be estimated by updating Infracost Cloud settings; see docs for other options.

──────────────────────────────────
49 cloud resources were detected:
∙ 10 were estimated
∙ 39 were free

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Project                                            ┃ Baseline cost ┃ Usage cost* ┃ Total cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━╋━━━━━━━━━━━━┫
┃ main                                               ┃          $294 ┃           - ┃       $294 ┃
┃ terraform                                          ┃          $585 ┃           - ┃       $585 ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━┻━━━━━━━━━━━━┛

From the example above, you see that Infracost breakdown estimates $879/month, mostly from EC2 instances ($680) and NAT gateways ($65), with smaller costs from storage, RDS, and load balancers. This clearly shows how compute and networking dominate cloud bills in this setup, and why Infracost is useful for spotting optimizations like right-sizing instances or reducing NAT gateways before deploying.

Conclusion

Getting hit with an unexpected cloud bill is one of those experiences that stays with you, and not in a good way. Throughout this article, we've explored how Infracost can transform the way you handle cloud costs by giving you the information you need upfront, rather than leaving you to discover it in your monthly bill. The beauty of Infracost lies in its simplicity; you get real-time cost estimates integrated right into your development workflow, whether through pull requests or command-line breakdowns.

The shift from reactive to proactive cost management isn't just about saving money; it's about making better decisions with complete information. Tools like Infracost help ensure that cost visibility becomes a natural part of your workflow, balancing both technical requirements and business realities from day one. At the end of the day, the best way to avoid bill shock is to eliminate the shock entirely.

Dev, Test, Stage, Prod: How Applications Are Deployed in the Real World

EphraimX — Thu, 14 Aug 2025 09:08:02 +0000

Many developers and DevOps engineers begin building their projects in a similar manner. They have a project idea, create a GitHub or GitLab repository, and then they get to building out the project. This process is ideal when you're a beginner or working on a side project that you plan to present to recruiters to increase your chances of getting hired.

However, this process is not sufficient when building for end-users and deploying in production, as you will immediately notice upon being onboarded to the company's version control system. You get to see a standard way of shipping to production, and if this is your first time seeing this, you might be wondering what's going on.

That's why I'm writing this article. The goal is to present a high-level explanation of how startups and organizations deploy their code to production, which is to be used by their end-users. If you're just starting in your career or you know how to code but haven't worked in this sort of environment, this article is for you.

Dev, Test, Stage, Prod: What Do They Mean?

From the article topic, you might have guessed or known that dev stands for development, test for testing, stage for staging, and prod for production. So what are they? They are environments. And what is an environment in this context? An environment is a space that is built and tailored to serve a specific interest or set of interests. In this case, the interest of ensuring your code or application is built and deployed to the highest standard possible.

When building applications for thousands to millions to billions of people, each developer cannot just build on their local machine, push to GitHub, merge it (that's if it does), and push straight to production. This is a true and tested recipe for chaos, as you can imagine. What you get from this is conflicting, unstandardized code and production environments. This inevitably leads to a terrible product, a bad reputation, and a declining customer base. That's why no one does it.

Dev, Test, Stage, Prod: Who Uses What and Why?

One thing to keep in mind is that this setup changes from company to company. So don't expect to see this setup everywhere you go. What you will find is that some companies use just Dev and Prod, some Test and Prod, others Dev, Stage, Prod, and others commit to the full process.

What is generally seen is that startups tend to favour the Dev, Stage, Prod method, where speed is prioritized over process. They get less maintenance overhead as there are fewer environments, and it works well when deployments are frequent, and also, teams can quickly rollback if needed. Also, for this setup, developers can do some testing locally or in the staging environment before deploying to production.

On the other hand, companies that are mid to large in terms of size, work in regulated industries like finance, healthcare, aerospace, or the military, and have dedicated QA teams, go for the Dev, Test, Stage, and Prod environment setup. Why? The first reason is that more layers between development and production reduce the risk of pushing untested code to production. Also, companies in these categories mostly prioritize process over speed, and they like to get it right once, no matter how long it takes.

Dev, Test, Stage, Prod: Features And Differences

Here, you'll understand the details behind these various environments and what they entail, starting with the development environment

Development Environment

This is the initial environment where developers first check in their new code. It's primarily for developers to see if their code changes are working on a basic level. This environment is very messy, often corrupted, and goes down a lot due to developers frequently checking in new code, and they sometimes do this without any testing or just minimal tests. In summary, little to no testing goes on in this environment

Testing Environment

The test environment is where the code goes after some initial checks take place in Dev, and it's the primary place for Quality Assurance and testing activities. Unlike the dev environment, it's a more controlled environment for functional, integration, and regression testing.

This environment is usually used by dedicated QA teams within these organizations. It's more like the quality control department of the code process, where tests like smoke tests, integration tests, regression tests, automated tests, and all manual tests are carried out. Any bug found within this environment is reported back to the dev team, and the code is tagged so that defects can be traced to specific builds.

Pre-prod or Staging Environment

The staging or pre-production environment is a near-identical replica of production for final acceptance testing. By final acceptance testing, I mean this is the last checkpoint before software or a system goes live.

It generally involves a formal review process where stakeholders like business owners, product managers, or beta testers verify that the product meets the agreed requirements and specifications. In the two previous stages, the focus was mainly on the technical aspect of the product; in this stage, the focus is on ensuring it works in real-world scenarios.

As mentioned initially, the staging environment is very similar to the production environment. It uses the same size machines, same hardware specs, software versions, configurations, integrations, and sometimes even the production database, just like the production environment. You also find that load and stress tests occur in this stage to mimic live production usage. All this is done to ensure the system doesn't fail in production.

Production Environment

This is the final environment where the code is live and accessible to the public. It is the version that your customers interact with. In terms of stability, it is the most stable environment, as any issues can have severe consequences. The code in this environment is a result of thorough checks and is constantly monitored for any issues that may come up.

In this stage, you validate if the deployment was successful, but compared to other stages, you do not run as many tests, and tests here are generally limited to non-destructive tests (like loading a page without changing data), as creating fake users or generating fake data can interfere with analytics or violate security/government controls. At this final stage, scaling, caching, and reliability are the top priorities of the organization. At this point, the deployment pipeline comes to an end.

Conclusion

I hope this article aided your understanding of what deployment looks like in the real world. If you're like me, who was previously confused about what these stages mean, I hope this article clears up your doubt. If you need more resources, you can check out the following:

Thank you for reading to the end, and if you're interested, you can read more DevOps-related content on my page.

End-to-End DevOps: Deploying and Monitoring a Full-Stack App with Docker, Terraform, and AWS

EphraimX — Wed, 13 Aug 2025 14:25:20 +0000

In this article, you’ll learn how to build and deploy a production-ready full-stack application on AWS using Docker, Terraform, Prometheus, and Grafana for containerization, infrastructure provisioning, monitoring, and visualizing application metrics, respectively.

This project reflects how full-stack applications are deployed in real-world environments, from startups to large organizations. It’s ideal for full-stack developers looking to deploy their apps to the cloud, or aspiring DevOps engineers who want a hands-on understanding of how infrastructure, deployment, and monitoring all come together.

By the end of this article, you’ll walk away with both the practical skills and theoretical knowledge needed to:

Containerize a full-stack app
Provision and manage cloud infrastructure using Terraform
Deploy services to AWS
Set up full observability using Prometheus and Grafana

Prerequisites and Setup

Before you begin, ensure you have the following tools and resources configured on your local machine:

Docker & Docker Compose These are required to containerize and run the application services locally before deployment. Follow the official installation guide based on your operating system here.
Terraform Terraform will be used to provision all necessary infrastructure on AWS. You can install it by following the instructions on the official HashiCorp website.
AWS CLI Required to configure and authenticate your AWS credentials locally. Install it using the official AWS CLI installation guide.
AWS Account + Access Keys You’ll need an AWS account with programmatic access enabled. If you don’t have one, sign up here. Then generate your Access Key ID and Secret Access Key by following this step-by-step guide.
Git and Project Repository Clone the GitHub repository containing all the code and infrastructure configuration for this project:

  git clone https://github.com/EphraimX/aws-devops-fullstack-pipeline.git

Once all tools are installed and your credentials are set, you're ready to begin deploying and monitoring the application.

Understanding the Application

The application you're deploying is a classic three-tier architecture consisting of:

Frontend (Client-side) – A simple interface that collects user input.
Backend (Server-side) – A FastAPI application that handles computation and database operations.
Database – A PostgreSQL instance hosted on AWS RDS.

The application functions as an ROI (Return on Investment) calculator. It accepts:

The total cost of an investment,
The expected monthly profit, and
The number of months the profit is expected to come in.

Once the user submits these values, the frontend sends a request to the backend via the /api/calculate-roi endpoint. The backend calculates the ROI percentage and the time to break even, then sends the result back to the frontend.

Here’s the logic behind the ROI calculation:

class RoiCalculationRequest(BaseModel):
    cost: float
    revenue: float
    timeHorizon: int

@app.post("/api/calculate-roi")
async def calculate_roi(request: RoiCalculationRequest):
    """
    Calculates Return on Investment (ROI) and break-even time.
    """
    current_cost = request.cost
    current_revenue = request.revenue
    current_time_horizon = request.timeHorizon

    if any(val is None for val in [current_cost, current_revenue, current_time_horizon]) or current_time_horizon <= 0:
        raise HTTPException(status_code=400, detail="Please enter valid numbers for all fields.")

    total_revenue = current_revenue * current_time_horizon
    net_profit = total_revenue - current_cost

    calculated_roi_percent = 0.0
    if current_cost > 0:
        calculated_roi_percent = (net_profit / current_cost) * 100
    elif net_profit > 0:
        calculated_roi_percent = float('inf')

    calculated_break_even_months: Union[str, float] = "N/A"
    if current_revenue > 0:
        calculated_break_even_months = round(current_cost / current_revenue, 2)
    elif current_cost > 0:
        calculated_break_even_months = "Never"
    else:
        calculated_break_even_months = "0"

    roi_percent = "Infinite" if calculated_roi_percent == float('inf') else f"{calculated_roi_percent:.2f}%"
    break_even_months = str(calculated_break_even_months)

    return {"roiPercent": roi_percent, "breakEvenMonths": break_even_months}

Once the frontend receives the ROI and break-even results, it makes another API request to /api/recordEntry. This second endpoint records both the user inputs and the computed results into the PostgreSQL database for tracking and analysis.

Here’s the relevant backend logic:

class PaymentRecordRequest(BaseModel):
    cost: float
    revenue: float
    time_horizon: int = Field(..., alias="timeHorizon")
    roi_percent: str = Field(..., alias="roiPercent")
    break_even_months: str = Field(..., alias="breakEvenMonths")
    date: str

    class Config:
        validate_by_name = True 

@app.post("/api/recordEntry")
def record_payment(request: PaymentRecordRequest, db: Session = Depends(get_db)):
    try:
        new_record = PaymentRecord(
            cost=request.cost,
            revenue=request.revenue,
            time_horizon=request.time_horizon,
            roi_percent=request.roi_percent,
            break_even_months=request.break_even_months,
            date=request.date
        )
        db.add(new_record)
        db.commit()
        db.refresh(new_record)

        return {"success": True, "message": "Payment record saved to RDS."}
    except Exception as e:
        logging.exception(e)
        raise HTTPException(status_code=500, detail="Failed to save payment record.")

Below is a short demo that shows the flow of the application in action:

Application Containerization

Containerization allows developers to build and ship their applications on any machine or server without running into a myriad of issues like installation errors or dependency problems. It creates its operating system for applications to run on.

The process of containerizing an application depends mainly on how the application runs, the packages it needs, and the overall architecture of your system. For this application, the frontend tier runs on Next.js, and the backend runs on FastAPI.

For the frontend tier, here's the Dockerfile for it:

# ---------- Builder Stage ----------
FROM node:21.5-bullseye-slim AS builder

WORKDIR /usr/src/app

# Copy package files and install dependencies
COPY package*.json ./
RUN npm install --force

# Copy application code
COPY . .

# Build-time environment variable
ARG NEXT_PUBLIC_APIURL
ENV NEXT_PUBLIC_APIURL=$NEXT_PUBLIC_APIURL

# Build the app
RUN npm run build

# ---------- Production Stage ----------
FROM node:21.5-bullseye-slim AS runner

WORKDIR /usr/src/app

# Install only production dependencies
COPY package*.json ./
RUN npm install --omit=dev --force

# Install PM2 globally
RUN npm install -g pm2

# Copy built app from builder stage
COPY --from=builder /usr/src/app/.next ./.next
COPY --from=builder /usr/src/app/public ./public
COPY --from=builder /usr/src/app/node_modules ./node_modules
COPY --from=builder /usr/src/app/package.json ./package.json
COPY --from=builder /usr/src/app/next.config.mjs ./next.config.mjs

# Expose app port
EXPOSE 3000

# Set runtime env (can override with --env at runtime)
ENV NEXT_PUBLIC_APIURL=$NEXT_PUBLIC_APIURL

# Start Next.js using PM2
CMD ["pm2-runtime", "start", "npm", "--", "start"]

From the Dockerfile above, you can see you have two stages: the builder stage and the production stage. Dockerfiles like this are referred to as multi-stage Dockerfiles, and they're important because they help reduce the size of the image and make it more secure by using only the necessary components of the application, thereby reducing the total surface area for an attack.

In the builder stage of the frontend Dockerfile, you start by defining the base image, which in this case is an operating system with a version of Node.js installed. Next, you define the working directory and copy all variations of the package.json file, before running npm install --force to force install all the packages. You then set the API URL as a build-time argument and set it as an environment variable that the application can reference during build. Finally, you run npm run build to build the application.

Once that's done, you move to the production stage, where you choose a similar base image and working directory. You then copy all variations of the package.json file again and run npm install --omit=dev --force to install only production dependencies. Next, you install pm2, which is what you'll use to serve the application. You then copy all the relevant folders from the previous stage into your working directory, expose port 3000, set your runtime environment variable, and pass in the entrypoint command to start the application with PM2.

Once this is done, you can move on to your backend Dockerfile. Here, you follow the same principle of a multi-stage Dockerfile, for the added reasons of security and image size efficiency.

Here's the Dockerfile for the backend system:

# =================================
# Stage 1: Base Dependencies
# =================================
FROM python:3.11-slim AS base

# Install system dependencies
RUN apt-get update && apt-get install -y \
    curl \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# =================================
# Stage 2: Dependencies Builder
# =================================
FROM base AS deps-builder

# Install build dependencies
RUN apt-get update && apt-get install -y \
    build-essential \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

# Create virtual environment
RUN python -m venv /venv
ENV PATH="/venv/bin:$PATH"

# Install Python dependencies
COPY requirements.txt .
RUN pip install --upgrade pip && \
    pip install --no-cache-dir -r requirements.txt

# =================================
# Stage 3: Production
# =================================
FROM base AS production

# Install runtime dependencies only
RUN apt-get update && apt-get install -y \
    libpq5 \
    && rm -rf /var/lib/apt/lists/* \
    && groupadd -r fastapi && useradd -r -g fastapi fastapi

# Copy virtual environment
COPY --from=deps-builder /venv /venv
ENV PATH="/venv/bin:$PATH"

# Copy application
COPY . /app/

# Make scripts executable
RUN chmod +x /app/scripts/entrypoint.sh
RUN chmod +x /app/scripts/healthcheck.sh

# Set ownership
RUN chown -R fastapi:fastapi /app

USER fastapi

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
    CMD /app/scripts/healthcheck.sh

EXPOSE 8000

ENTRYPOINT ["/app/scripts/entrypoint.sh"]

From your backend Dockerfile, you have 3 stages. The first stage is where you install the base dependencies, more like the fundamental packages needed for the image OS to run properly. You also set the working directory in this stage.

Next is the dependencies builder stage, where you use the base stage as your image and install the Python dependencies. The last stage is the production stage, where you copy the environment packages from the deps stage, set your Python path to /venv/bin, and copy the entire application to your /app directory.

Next, you make both the entrypoint and healthcheck scripts executable. The healthcheck script is what your Dockerfile runs to ensure the backend system is working as expected.

Here's the content of the file:

#!/bin/bash

set -e
set -x

# Check application viability
curl --fail http://localhost:8000/api/healthcheck || exit 1

# Check database viability
curl --fail http://localhost:8000/api/dbHealth || exit 1

Here you set -e for the script to cancel if any command fails, and also -x to print the processes as they run. Then you make requests to both the application healthcheck and the database healthcheck endpoints to ensure the main components of the backend infrastructure are running as required.

Once that’s done, you set fastapi as the owner of the system, switch to that user, and run the healthcheck script as described above. Finally, you expose port 8000 and then run the entrypoint script.

Here’s the entrypoint script:

#!/bin/bash

set -e
set -x

alembic -c /app/alembic.ini upgrade head

exec uvicorn main:app --host 0.0.0.0 --port 8000 --workers 4

Similar to the healthcheck script, you set it to fail on error and print logs as the application runs. Then you run migrations using Alembic to essentially create the tables in your PostgreSQL database, before starting the FastAPI application on host 0.0.0.0 and port 8000, with 4 workers to run concurrent processes.

Monitoring Setup With Prometheus and Grafana

Monitoring is an important aspect of every application structure. It is often said that you can’t improve what you don’t measure, hence why monitoring is a core part of every application that makes it to production.

One thing to note is that there are two major components of monitoring: logs and metrics. Logs are information collected or inputted by the different services that are running, while metrics are thresholds that are generally measured from actions or events.

There are different tools used within the industry to monitor systems, but two of the most popular tools used in tandem are Prometheus and Grafana. Prometheus is a monitoring application that collects information and stores it in a central database. It gets this information by pulling and scraping metrics from different targets and servers.

Grafana, on the other hand, is a visualization platform that uses PromQL (Prometheus Query Language) to query this information from Prometheus. This queried data can be visualized via dashboards, either created by the engineer or sourced from the community. It's generally advisable to first search if the dashboard you need already exists on the Grafana Community Dashboards before going ahead to build a custom solution.

For this article, you will monitor two important sets of metrics. First, you will monitor the host machine metrics using node_exporter, and also monitor container resources using cadvisor. Monitoring these two gives you a perspective into how your system is performing. With node_exporter, you get to see how your host machine is behaving, how your CPU and RAM are responding to the system’s workload. With cadvisor, you monitor your container resources to keep an eye on how they’re performing with the applications running in them.

To get started with monitoring, navigate to the monitoring folder and take a look at the monitoring-docker-compose.yml file, as shown below:

volumes:
  prometheus-data:
    driver: local
  grafana-data:
    driver: local

networks:
  monitoring:
    driver: bridge

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    ports:
      - "9090:9090"
    volumes:
      - "./prometheus.yaml:/etc/prometheus/prometheus.yml"
      - "prometheus-data:/prometheus"
    restart: unless-stopped
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
    networks:
      - monitoring

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    ports:
      - "3000:3000"
    volumes:
      - "grafana-data:/var/lib/grafana"
    restart: unless-stopped
    networks:
      - monitoring

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.52.1
    container_name: cadvisor
    ports:
      - "8085:8080"
    volumes:
      - /:/rootfs:ro
      - /run:/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    devices:
      - /dev/kmsg
    privileged: true
    restart: unless-stopped
    networks:
      - monitoring

  node_exporter:
    image: quay.io/prometheus/node-exporter:v1.9.1
    container_name: node_exporter
    command: "--path.rootfs=/host"
    pid: host
    restart: unless-stopped
    volumes:
      - /:/host:ro,rslave
    networks:
      - monitoring

For your monitoring file, you start out by defining the volumes for both Prometheus and Grafana, as these services require persistent volumes to store configurations and data. Next, you define the different services, including the image, container name, the host, and container ports, volumes, and startup commands where necessary.

For Prometheus to know what services to scrape, they need to be included as a target in the prometheus.yml file, which you can see below:

---
global:
  scrape_interval: 15s  

scrape_configs:
  # Monitor Prometheus
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']

  # Monitor EC2 Instances
  - job_name: 'node_exporter'
    static_configs:
      - targets: ['node_exporter:9100']

  # Monitor All Docker Containers
  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8085']

Here, you set Prometheus up to scrape its own metrics on port 9090, node_exporter on port 9100, and cadvisor on port 8085

Once this is done, you can now build and start your monitoring stack by running:

sudo docker compose -f monitoring-docker-compose.yml up -d prometheus cadvisor node_exporter grafana

Once successful, you should find a similar printout in your terminal:

[+] Running 30/30
 ✔ node_exporter Pulled                                                  132.6s 
 ✔ prometheus Pulled                                                     129.6s 
 ✔ cadvisor Pulled                                                        20.9s 
 ✔ grafana Pulled                                                        129.4s 


[+] Running 7/7
 ✔ Network monitoring_default           Cr...                              0.1s 
 ✔ Volume "monitoring_prometheus-data"  Created                            0.0s 
 ✔ Volume "monitoring_grafana-data"     Created                            0.0s 
 ✔ Container cadvisor                   Started                           13.6s 
 ✔ Container grafana                    Started                           13.6s 
 ✔ Container node_exporter              Start...                          13.3s 
 ✔ Container prometheus                 Started                           13.4s

Once your monitoring services are running, the next step is to set up your Grafana dashboard. You can do this by heading to localhost:3000, where you’ll be met with the Grafana login page. Proceed to log in with the default username and password: admin. For security purposes, it is advisable to change this on first login.

Once logged in, the first step would be to set Prometheus as your data source:

![Select Prometheus as your Data Source]

Next, input the Prometheus URL, which, if you're running on localhost, should be http://localhost:9090 or http://prometheus:9090.

Once successful, you should see the success message at the bottom of the page.

All that’s left to do is create the dashboards to visualize the scraped data. To do that, navigate to the /grafana-dashboards folder in monitoring and copy the contents of the node-exporter.json file in the repository. Once done, click on the “Dashboards” option on the left-hand side of the page, and click on the “New” button at the right-hand side of the screen. You will find three ways to create a dashboard — select the Import method.

![Choose the Import method to create the dashboard]

Proceed to paste the content of the node-exporter.json into the "Import via dashboard JSON model" section and click on the “Load” button.

On the next page, select “Prometheus” as the data source for your dashboard. Once done, you should be met with this dashboard displaying your host machine metrics.

From the image above, you can see metrics like “System Load” based on time averages, the “Root FS” usage, and a host of other insights.

For cadvisor, follow the same process to import the dashboard using the cadvisor.json, and you should get something like this:

The result above shows the CPU, Memory, Network, and Misc metrics of the different running containers.

Understanding the Architecture

Before deploying any application to the cloud, it's important to have a laid-out architecture or system design for how the application will be deployed. This matters because it helps you understand what works and what doesn't, along with the trade-offs between different architectural setups. Doing this allows you to optimize your deployment based on your specific use case and take into account cost, reliability, and scalability.

Architectural Diagram

Represented below is the application's architectural diagram to be deployed on AWS:

From the diagram above, you can see the different AWS services that come together to make the application production-ready. The diagram really just has two sets of services: network and compute.

Under the network layer, you have:

VPC – This isolates your application from the world.
Internet Gateway – Attached to the VPC so it can communicate with the outside world.
Public Subnets – Connected to the Internet Gateway and used to house public-facing resources like your Application Load Balancer (ALB), Public EC2 instance, and NAT Gateways.
NAT Gateways – These allow resources in your private subnet to send traffic to the internet, while blocking incoming traffic.
Private Subnets – These house private services that don’t need to communicate directly with the outside world.

For your compute resources, you have:

Application Load Balancer (ALB) – Lives in the public subnet, provides an entry point for external users, and routes requests to the appropriate targets.
Bastion Host – Also in the public subnet. This allows SSH access to the private EC2 instance and also doubles as the server running the Grafana container.
Private EC2 Instance – Hosts both the frontend and backend containers, and also runs monitoring tools like Prometheus, Node Exporter, and cAdvisor.
RDS (PostgreSQL) – This is your managed database instance that holds all the application data.

Implementing the Architecture with Terraform

Terraform is an Infrastructure as Code (IaC) tool, which is used for automatically provisioning resources in the cloud. Its need stems from a place of developers not wanting to "point and click" their way to the creation of resources, but rather have them automatically provisioned, while having a record of what changed and what did not for effective management.

To provision your resources with Terraform, you begin by defining your providers.tf file, where you specify the providers or "packages" terraform should install for your project. Here's the providers.tf file for this project.

terraform {
  required_providers {
    aws = {
      syource = "hashicorp/aws"
      version = "6.5.0"
    }
  }
}


provider "aws" {
  region = var.aws_region
}

In your provider's Terraform file, you specify AWS as a required provider and assign the region from the variables Terraform file.

The variables Terraform file is used to prevent repetition of variables across different files in your Terraform configuration.

In the variables.tf file for this project, as seen below, you will specify a couple of variables that will be used throughout the project.

variable "aws_region" {
  type = string
  default = "us-east-2"
}


variable "tags" {
  type = object({
    name = string
    environment = string
    developer = string
    team = string
  })

  default = {
    name = "EphraimX"
    developer = "TheJackalX"
    environment = "Production"
    team = "Boogey Team"
  }
}


variable "DB_PORT" {
  type = number
  sensitive = true
}


variable "DB_NAME" {
  type = string
  default = "roi_calculator"
}


variable "DB_USER" {
  type = string
  sensitive = true
}


variable "DB_PASSWORD" {
  type = string
  sensitive = true
}


variable "DB_IDENTIFIER" {
  type = string
  default = "roi-calculator"
}


variable "DB_INSTANCE_CLASS" {
  type = string
  default = "db.t3.micro"
}


variable "DB_ENGINE" {
  type = string
  default = "postgres"
}


variable "DB_TYPE" {
  type = string
  default = "postgresql"
}


# variable "NEXT_PUBLIC_APIURL" {
#   type = string
#   sensitive = true
# }


variable "CLIENT_URL" {
  type = string
  default = "*"
}

Specified in the variables.tf are different variables such as the AWS region you'll be deploying your resources to, the tags for your resources, the database port, and other database values such as the name, the user, the password, instance identifier, instance class, database engine, and the database type. Also included is the client URL, which is added to the list of allowed origins in the backend code to prevent a CORS error,r as seen in the snippet below:

# CORS Settings
CLIENT_URL = os.getenv("CLIENT_URL", "http://localhost:3000")
ALLOyouD_ORIGINS = [
    "http://localhost",
    "http://localhost:3000",  # or the port your frontend uses
    CLIENT_URL
]

With this in place, you can go on to build your main.tf, the file that will handle all your provisioning and the configurations.

Starting with the VPC and Subnets:

#################################################
## AWS VPC and Subnets
#################################################

resyource "aws_vpc" "roi_calculator_vpc" {
  cidr_block = "10.10.0.0/16"
  tags = var.tags
}


resyource "aws_subnet" "roi_calculator_public_subnet_one" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  cidr_block = "10.10.10.0/24"
  availability_zone = "us-east-2a"
  tags = var.tags
}


resyource "aws_subnet" "roi_calculator_public_subnet_two" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  cidr_block = "10.10.20.0/24"
  availability_zone = "us-east-2b"
  tags = var.tags
}


resyource "aws_subnet" "roi_calculator_private_subnet_one" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  cidr_block = "10.10.30.0/24"
  availability_zone = "us-east-2a"
  tags = var.tags
}


resyource "aws_subnet" "roi_calculator_private_subnet_two" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  cidr_block = "10.10.40.0/24"
  availability_zone = "us-east-2b"
  tags = var.tags
}

Here, you define your VPC and the CIDR block you want it to operate on. CIDR blocks are how you define IP address ranges in your VPC; they tell AWS which IPs your network will own. Instead of assigning individual IPs, you just define a range like 10.10.0.0/16, and AWS handles the rest. With this in place, you define your subnets, both public and private, deploying them in different availability zones in order to ensure high availability.

Next, you define your internet gateway and route tables for your public subnets.

#################################################
## Internet Gateway and Route Tables - Public 
#################################################


resyource "aws_internet_gateway" "roi_calculator_igw" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  tags = var.tags
}


resyource "aws_route_table" "roi_calculator_route_table" {
  vpc_id = aws_vpc.roi_calculator_vpc.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.roi_calculator_igw.id
  }

  tags = var.tags 
}


resyource "aws_route_table_association" "roi_calculator_route_table_association_public_subnet_one" {
  route_table_id = aws_route_table.roi_calculator_route_table.id
  subnet_id = aws_subnet.roi_calculator_public_subnet_one.id
}


resyource "aws_route_table_association" "roi_calculator_route_table_association_public_subnet_two" {
  route_table_id = aws_route_table.roi_calculator_route_table.id
  subnet_id = aws_subnet.roi_calculator_public_subnet_two.id
}

An internet gateway is important when working in a VPC as it allows resources within it to access the internet. Without it, resources in your VPC will be completely shut off from the internet. You also define a route table here, which routes all incoming and outgoing traffic from the internet through your internet gateway. And lastly, for your public subnets to be truly public, you have to associate them with the route table connected to the internet gateway.

Next, you define your Elastic IP address, NAT gateway, and corresponding route tables for your private subnets.

#############################################################
## EIP, NAT Gateway, Route Tables - Private Subnet One
#########################################################


resyource "aws_eip" "roi_calculator_ngw_eip_private_subnet_one" {
  domain = "vpc"
  tags = var.tags
}


resyource "aws_nat_gateway" "roi_calculator_ngw_private_subnet_one" {
  subnet_id = aws_subnet.roi_calculator_public_subnet_one.id
  allocation_id = aws_eip.roi_calculator_ngw_eip_private_subnet_one.id
  tags = var.tags
  depends_on = [ aws_internet_gateway.roi_calculator_igw ]
}


resyource "aws_route_table" "roi_calculator_route_table_private_subnet_one" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  route {
    cidr_block = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.roi_calculator_ngw_private_subnet_one.id
  }
  tags = var.tags
}


resyource "aws_route_table_association" "roi_calculator_route_table_association_private_subnet_one" {
  subnet_id = aws_subnet.roi_calculator_private_subnet_one.id
  route_table_id = aws_route_table.roi_calculator_route_table_private_subnet_one.id
}


#############################################################
## EIP, NAT Gateway, Route Tables - Private Subnet Two
#########################################################


resyource "aws_eip" "roi_calculator_ngw_eip_private_subnet_two" {
  domain = "vpc"
  tags = var.tags
}


resyource "aws_nat_gateway" "roi_calculator_ngw_private_subnet_two" {
  subnet_id = aws_subnet.roi_calculator_public_subnet_two.id
  allocation_id = aws_eip.roi_calculator_ngw_eip_private_subnet_two.id
  tags = var.tags
  depends_on = [ aws_internet_gateway.roi_calculator_igw ]
}


resyource "aws_route_table" "roi_calculator_route_table_private_subnet_two" {
  vpc_id = aws_vpc.roi_calculator_vpc.id
  route {
    cidr_block = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.roi_calculator_ngw_private_subnet_two.id
  }
  tags = var.tags
}


resyource "aws_route_table_association" "roi_calculator_route_table_association_private_subnet_two" {
  subnet_id = aws_subnet.roi_calculator_private_subnet_two.id
  route_table_id = aws_route_table.roi_calculator_route_table_private_subnet_two.id
}

It is generally expected that resources in your private subnet not have any form of communication to the internet, as it hosts very sensitive applications. But what happens in production is that some of these resources need to access the internet for things like patches and updates.

To solve this, a NAT gateway is placed in a public subnet and attached to the private subnet through a route table. And for the NAT gateway to be able to communicate with the internet, an Elastic IP address (EIP) is attached to provide it with a static public IPv4 address for this purpose.

All these define your network resources for the project. For the compute resources, the first set of resources to define is the security groups for the EC2 servers.

#################################################
## Bastion Host Security Group
#################################################


resyource "aws_security_group" "roi_calculator_bastion_host_sg" {
  name = "roi-calculator-bastion-host-sg"
  vpc_id = aws_vpc.roi_calculator_vpc.id
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_grafana_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_bastion_host_sg.id
  description = "grafana"
  cidr_ipv4 = "0.0.0.0/0"
  from_port = 3000
  to_port = 3000
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_bastion_ssh_ingress" {
  security_group_id = aws_security_group.roi_calculator_bastion_host_sg.id
  description = "ssh"
  cidr_ipv4         = "0.0.0.0/0"   # Open to the world – for production, restrict this!
  from_port         = 22
  to_port           = 22
  ip_protocol       = "tcp"
}


resyource "aws_vpc_security_group_egress_rule" "bastion_host_allow_all_traffic_ipv4" {
  security_group_id = aws_security_group.roi_calculator_bastion_host_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}


#################################################
## Production Host Security Group
#################################################


resyource "aws_security_group" "roi_calculator_production_host_sg" {
  name = "roi-calculator-production-host-sg"
  vpc_id = aws_vpc.roi_calculator_vpc.id
  tags = var.tags
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_production_ssh_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "ssh"
  cidr_ipv4         = aws_vpc.roi_calculator_vpc.cidr_block   # Open to the world – for production, restrict this!
  from_port         = 22
  to_port           = 22
  ip_protocol       = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_http_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "http-and-also-for-next-js"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 80
  to_port = 80
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_https_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "https"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 443
  to_port = 443
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_fastapi_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "fastapi"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 8000
  to_port = 8000
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_next_js_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "next_js"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 8081
  to_port = 8081
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_prometheus_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "prometheus"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 9090
  to_port = 9090
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_cadvisor_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "cadvisor"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 8085
  to_port = 8085
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_node_exporter_sg_ingress" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  description = "node_exporter"
  cidr_ipv4 = aws_vpc.roi_calculator_vpc.cidr_block
  from_port = 9100
  to_port = 9100
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_egress_rule" "production_host_allow_all_traffic_ipv4" {
  security_group_id = aws_security_group.roi_calculator_production_host_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}


#################################################
## Application Load Balancer Security Group
#################################################


resyource "aws_security_group" "roi_calculator_alb_sg" {
  name = "roi-calculator-alb-sg"
  vpc_id = aws_vpc.roi_calculator_vpc.id
  tags = var.tags
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_alb_sg_http" {
  security_group_id = aws_security_group.roi_calculator_alb_sg.id
  description = "alb_http"
  cidr_ipv4 = "0.0.0.0/0"
  from_port = 80
  to_port = 80
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_ingress_rule" "roi_calculator_alb_sg_https" {
  security_group_id = aws_security_group.roi_calculator_alb_sg.id
  description = "alb_https"
  cidr_ipv4 = "0.0.0.0/0"
  from_port = 443
  to_port = 443
  ip_protocol = "tcp"
}


resyource "aws_vpc_security_group_egress_rule" "alb_allow_all_traffic_ipv4" {
  security_group_id = aws_security_group.roi_calculator_alb_sg.id
  cidr_ipv4         = "0.0.0.0/0"
  ip_protocol       = "-1"
}

A VPC security group is responsible for determining what traffic comes in and out, from what source, and also what port the traffic should be directed to. For the bastion host in this project, you defined two ingress rules, ingress means what traffic should be allowed into the instance.

These rules allow traffic from any IP address to port 3000 for accessing Grafana, and port 22 for accessing the system via SSH. In the example above, access is granted from all IP addresses, but in production, limit SSH access to just your IP address. Lastly, on the bastion host, you allow traffic out of the instance to all IP addresses.

For the production host, you have a similar setup, just with more ports. For ingress rules, you have 22 for SSH, 80 & 8081 for the frontend application, depending on the configuration, 8000 for the backend application, 443 for HTTPS, 9090 for Prometheus, 8085 for cAdvisor, 9100 for the Node Exporter, and lastly, for the egress rules, you allow traffic to all IP addresses.

Next, you define the EC2 instance for both the bastion host and production host.

#################################################
## Bastion Host 
#################################################


resyource "aws_instance" "roi_calculator_bastion_host_ec2_public_subnet_one" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.xlarge"
  vpc_security_group_ids = [aws_security_group.roi_calculator_bastion_host_sg.id]
  subnet_id = aws_subnet.roi_calculator_public_subnet_one.id
  key_name = "rayda-application"
  associate_public_ip_address = true
  user_data = file("scripts/bastion-host.sh")
  tags = var.tags
}


resyource "aws_instance" "roi_calculator_bastion_host_ec2_public_subnet_two" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.xlarge"
  vpc_security_group_ids = [aws_security_group.roi_calculator_bastion_host_sg.id]
  subnet_id = aws_subnet.roi_calculator_public_subnet_two.id
  key_name = "rayda-application"
  associate_public_ip_address = true
  user_data = file("scripts/bastion-host.sh")
  tags = var.tags
}


#################################################
## Production Host
#################################################


resyource "aws_instance" "roi_calculator_production_host_ec2_private_subnet_one" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.xlarge"
  vpc_security_group_ids = [aws_security_group.roi_calculator_production_host_sg.id]
  subnet_id = aws_subnet.roi_calculator_private_subnet_one.id
  key_name = "rayda-application"
  associate_public_ip_address = false
  user_data = templatefile("${path.module}/scripts/production-host.sh", {
    db_host            = aws_db_instance.roi_calculator.address
    db_port            = var.DB_PORT
    db_name            = var.DB_NAME
    db_user            = var.DB_USER
    db_password        = var.DB_PASSWORD
    db_type            = var.DB_TYPE
    client_url         = var.CLIENT_URL
    MY_IP              = "127.0.0.1"
  })
  tags = var.tags
}


resyource "aws_instance" "roi_calculator_production_host_ec2_private_subnet_two" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.xlarge"
  vpc_security_group_ids = [aws_security_group.roi_calculator_production_host_sg.id]
  subnet_id = aws_subnet.roi_calculator_private_subnet_two.id
  key_name = "rayda-application"
  associate_public_ip_address = false
  user_data = templatefile("${path.module}/scripts/production-host.sh", {
    db_host            = aws_db_instance.roi_calculator.address
    db_port            = var.DB_PORT
    db_name            = var.DB_NAME
    db_user            = var.DB_USER
    db_password        = var.DB_PASSWORD
    db_type            = var.DB_TYPE
    client_url         = var.CLIENT_URL
    MY_IP              = "127.0.0.1"
  })
  tags = var.tags
}

For both these hosts, you define the properties of the EC2 instances, such as the instance class, which is set to "t3.xlarge", and the VPC security groups are set to the security group you defined initially. You create two instances and place them in two different subnets in different availability zones. You set the key name for SSH access, and you associate a public IP in case of a bastion host, but not for the production host. There's also the AMI or Amazon Machine Image, which provides the required software to set up and boot an Amazon EC2 instance. For the AMI, you get the ID of the image you want by retrieving the image properties from AWS in data.tf as shown below:

data "aws_ami" "ubuntu" {
  most_recent = true

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["099720109477"] # Canonical
}

As seen in the config above, you tell AWS to get you the most recent image running Ubuntu Jammy 22.04. You select virtualization to Hardware Virtual Machine (HVM) and select the owners to be canonical.

Another important configuration of both the production and bastion host is the user_data. The user_data is are set of commands you give to AWS to run at boot time. For an EC2 instance, this allows you to predefine installation steps without having to access these servers and run the installation manually.

For the bastion host, the user_data is using the file function to load the bastion-host.sh file in scripts. Here is the contents of the bastion scripts file:

#!/bin/bash

set -e
set -x


sudo apt update
sudo apt install -y unzip curl

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Update and install Docker prerequisites
sudo apt update -y
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common git

# Add Docker GPG key and repo
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"

# Install Docker
sudo apt update -y
sudo apt install -y docker-ce

# Verify Docker is running
sudo systemctl is-active --quiet docker && echo "Docker is running" || echo "Docker is not running"

# Allow current user to run Docker without sudo
sudo usermod -aG docker ubuntu

# Clone your repo
git clone https://github.com/EphraimX/roi-calculator.git

# Change directory
cd roi-calculator/monitoring

# Build and start the Grafana service from your Docker Compose file
sudo docker compose -f monitoring-docker-compose.yml up -d grafana

In the user_data for the bastion host, you set your initial parameters, such as set -e for exit on failure and set -x to log your process as it runs. Next, you update your packages and install much-needed packages such as the AWS CLI and Docker. Then you verify Docker is running and allow the current user to run Docker without sudo. Lastly, you clone the repository, navigate to the monitoring folder, and start the Grafana service.

The process is also very similar to the user_data of the production host, as seen below.

#!/bin/bash

set -e
set -x

sudo apt update
sudo apt install -y unzip curl

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

# Step 1: Request a metadata session token (valid for 6 hours)
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

# Step 2: Use that token to access instance metadata securely
MY_IP=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/local-ipv4)

# Set environment variables
# export NEXT_PUBLIC_APIURL="__next_public_apiurl__"
export NEXT_PUBLIC_APIURL="http://$(curl -H "X-aws-ec2-metadata-token: $(curl -X PUT http://169.254.169.254/latest/api/token \
                                                                              -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")" \
                                                                              http://169.254.169.254/latest/meta-data/local-ipv4):8000/api"
export DB_HOST="${db_host}"
export DB_PORT="${db_port}"
export DB_NAME="${db_name}"
export DB_USER="${db_user}"
export DB_PASSWORD="${db_password}"
export DB_TYPE="${db_type}"
export CLIENT_URL="${client_url}"

# echo "NEXT_PUBLIC_APIURL=http://${MY_IP}:8000/api" >> /etc/environment
echo "DB_HOST=$DB_HOST"
echo "DB_PORT=$DB_PORT"
echo "DB_NAME=$DB_NAME"
echo "DB_USER=$DB_USER"
echo "DB_PASSWORD=$DB_PASSWORD"
echo "DB_TYPE=$DB_TYPE"
echo "CLIENT_URL=$CLIENT_URL"

# Install Docker
sudo apt update -y
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common git
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository -y "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
sudo apt update -y
sudo apt install -y docker-ce
sudo systemctl is-active --quiet docker && echo "Docker is running" || echo "Docker is not running"
sudo usermod -aG docker ubuntu

# Clone repo
git clone https://github.com/EphraimX/roi-calculator.git
cd roi-calculator


# Create Docker Network
sudo docker network create roi-calculator-network

# Frontend
cd client-side
sudo docker build \
  -f Dockerfile.dev \
  --build-arg NEXT_PUBLIC_APIURL=$NEXT_PUBLIC_APIURL \
  -t roi-calculator-frontend .
sudo docker run -d --name roi-calculator-frontend --network roi-calculator-network -p 80:3000 roi-calculator-frontend

# Backend
cd ../server-side
sudo docker run -d \
  --name roi-calculator-backend \
  --network roi-calculator-network \
  -e DB_HOST=$DB_HOST \
  -e DB_PORT=$DB_PORT \
  -e DB_NAME=$DB_NAME \
  -e DB_USER=$DB_USER \
  -e DB_PASSWORD=$DB_PASSWORD \
  -e DB_TYPE=$DB_TYPE \
  -e CLIENT_URL=$CLIENT_URL \
  -p 8000:8000 \
  ephraimx57/roi-calculator-backend

# Monitoring
cd ../monitoring
sudo docker compose -f monitoring-docker-compose.yml up -d prometheus cadvisor node_exporter

Here, you perform the initial process involved for the bastion-host.sh. Past there, you generate the NEXT_PUBLIC_APIURL, which is the backend API to communicate with the frontend. Then you export the environment variables, perform a couple of installations including Docker, and then launch the frontend and backend of the.

To launch the frontend application, you pull the repository, navigate to the "client-side" directory, and build and run the Dockerfile with the appropriate parameters. For the backend, you run the already published image at "ephraimx57/roi-calcylator-backend" with the environment variables that you're initially passed to the script. Lastly, navigate to the monitoring directory to start Prometheus, cAdvisor, and Node Exporter.

After the production host, the next resource for you to implement is the RDS Postgres database, which is displayed below:

#################################################
## AWS RDS
#################################################


resyource "aws_db_subnet_group" "roi_calculator_rds_db_subnet_group" {
  name       = "roi-calculator-rds-db-subnet-group"
  subnet_ids = [aws_subnet.roi_calculator_private_subnet_one.id, aws_subnet.roi_calculator_private_subnet_two.id]
  tags = var.tags
}


resyource "aws_security_group" "rds_sg" {
  name        = "rds-sg"
  description = "Allow Postgres inbound traffic"
  vpc_id      = aws_vpc.roi_calculator_vpc.id
  ingress {
    description = "Allow Postgres from my IP"
    from_port   = 5432
    to_port     = 5432
    protocol    = "tcp"
    cidr_blocks = [aws_subnet.roi_calculator_private_subnet_one.cidr_block, aws_subnet.roi_calculator_private_subnet_two.cidr_block]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = [aws_subnet.roi_calculator_private_subnet_one.cidr_block, aws_subnet.roi_calculator_private_subnet_two.cidr_block]
  }
}


resyource "aws_db_instance" "roi_calculator" {
  identifier = var.DB_IDENTIFIER
  allocated_storage = 5
  db_name = var.DB_NAME
  engine = var.DB_ENGINE
  engine_version = "17.5"
  instance_class = var.DB_INSTANCE_CLASS
  username = var.DB_USER
  password = var.DB_PASSWORD
  skip_final_snapshot = true
  port = var.DB_PORT
  publicly_accessible = false
  db_subnet_group_name = aws_db_subnet_group.roi_calculator_rds_db_subnet_group.name
  vpc_security_group_ids = [aws_security_group.rds_sg.id]
}

The RDS instance is pretty straightforward to set up. You define a "db_subnet_group", which allows for the high availability of the database. Next, you have the security group, which allows the database to be reached on port 5432 from resources within the VPC. And finally, you create the instance itself with some parameters already specified in your variables.tf file.

Your last resource to create is the Application Load Balancer (ALB).

#################################################
## Application Load Balancer
#################################################


resyource "aws_lb" "roi_calculator_aws_lb" {
  name               = "roi-calculator-aws-lb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [aws_security_group.roi_calculator_alb_sg.id]
  subnets            = [aws_subnet.roi_calculator_public_subnet_one.id, aws_subnet.roi_calculator_public_subnet_two.id]
  tags = var.tags
}


resyource "aws_lb_target_group" "roi_calculator_aws_lb_target_group" {
  name     = "roi-calc-aws-lb-tg-prisub-one"
  port     = 80
  protocol = "HTTP"
  target_type = "instance"
  health_check {
    path                = "/"
    protocol            = "HTTP"
    healthy_threshold   = 2
    unhealthy_threshold = 2
    timeout             = 5
    interval            = 30
  }
  vpc_id   = aws_vpc.roi_calculator_vpc.id
}


resyource "aws_lb_listener" "roi_calculator_alb_sg_listener_private_subnet_one" {

  load_balancer_arn = aws_lb.roi_calculator_aws_lb.arn
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.roi_calculator_aws_lb_target_group.arn
  }
}


resyource "aws_lb_target_group_attachment" "roi_calculator_aws_lb_target_group_attachment_private_subnet_one" {
  target_group_arn = aws_lb_target_group.roi_calculator_aws_lb_target_group.arn
  target_id        = aws_instance.roi_calculator_production_host_ec2_private_subnet_two.id
  port             = 80
}


resyource "aws_lb_target_group_attachment" "roi_calculator_aws_lb_target_group_attachment_private_subnet_two" {
  target_group_arn = aws_lb_target_group.roi_calculator_aws_lb_target_group.arn
  target_id        = aws_instance.roi_calculator_production_host_ec2_private_subnet_two.id
  port             = 80
}

The main purpose of the ALB is to balance load across different prospective loads. For this application, the ALB is also preferred because it provides a URL to easily access internal-facing applications. To define your ALB, you need to specify the target group. A target group describes the nature of the target machine network. You will also require a listener to listen to all the requests made and forward them to the right target group. Finally, you attach the target group to the private subnet, which enables it to reach the resources hidden behind this private subnet.

##CI/CD With GitHub Actions

Once your Terraform infrastructure is done, you are now ready to automate the deployment using GitHub Actions. GitHub Actions is a Continuous Integration and Continuous Deployment tool that is used by companies to automate the testing, integration, and deployment of their code. It helps streamline development and prevents the deployment of code that does not pass certain criteria or tests.

Using CI/CD in this project allows you to build, test, and automatically effect a change once you push to the Version Control System, which is GitHub in this case.

To set up GitHub Actions, simply create a .github/workflows folder, and create the file deploy.yml in it and paste in the following code:

 name: Terraform Deploy

on:
  push:
    branches:
      - main  # or your deployment branch

jobs:
  terraform:
    name: Terraform Apply
    runs-on: ubuntu-latest

    defaults:
      run:
        working-directory: terraform

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v3
      with:
        terraform_version: 1.8.0  # or your preferred version

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v2
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

    - name: Terraform Init
      run: terraform init

    - name: Terraform Validate
      run: terraform validate

    - name: Terraform Plan
      run: |
        terraform plan \
          -var="db_user=${{ secrets.DB_USER }}" \
          -var="db_password=${{ secrets.DB_PASSWORD }}" \
          -var="db_port=${{ secrets.DB_PORT }}"

    - name: Terraform Apply
      if: github.ref == 'refs/heads/main'  # protect applies to main branch
      run: |
        terraform apply -auto-approve \
          -var="db_user=${{ secrets.DB_USER }}" \
          -var="db_password=${{ secrets.DB_PASSWORD }}" \
          -var="db_port=${{ secrets.DB_PORT }}"

The GitHub Actions code above performs a set of actions once a push is made to the main branch. It first sets the runner of the GitHub Actions, which is ubuntu-latest. Next, it sets the folder "terraform" as the default working directory, checks out the code, and sets up (installs) Terraform on the runner.

To be able to deploy to AWS, you configure your running using your ACCESS_KEY and SECRET_KEY that can be gotten from your AWS "Security Credentials" page. Next, it runs terraform init, which initializes the Terraform environment. Then it validates the code, ensuring the syntax is correct. Next, it runs terraform plan, which creates a plan of the resources to be created using variables from GitHub secrets. And finally, it applies the code with the same variables from GitHub Secrets.

For this to work, you have to have your secret set up. To do that, click on the "Settings" tab of your repository:

Under "Security," click on "Secrets and variables" and select "Actions"

Lastly, create repository secrets for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, assigning values from your AWS account, and DB_USER, DB_PASSWORD, DB_PORT, assigning values of your choice. For DB_PORT, it is advisable to use "5432" as that is the default port of PostgreSQL.

Verifying the Deployment

After completing the deployment, you need to verify that everything is running as expected.

Start by connecting to the Bastion Host using SSH:

eval "$(ssh-agent -s)"
ssh-add ~/path/to/key-pair
ssh -A ubuntu@<bastion-ip>

Once inside the Bastion Host, check that the monitoring containers are running:

docker ps

You should see the container for Grafana running.

Next, open your browser and navigate to:

http://<bastion-public-ip>:3000

This will take you to the Grafana dashboard. Log in using the default credentials (admin / admin), then go to Settings → Data Sources, and add a new Prometheus data source with the following URL:

http://<private-ec2-ip>:9090

This links Grafana to the Prometheus instance running on the private EC2 instance.

Once that's done, return to your terminal and SSH into the private EC2 instance:

ssh ubuntu@<private_ip>

Check that the frontend and backend services are running:

docker ps

Lastly, open the application in your browser by visiting the ALB DNS URL. This is the public-facing entry point to your full-stack application. You should be able to load the frontend and confirm that it connects properly to the backend.

Everything should now be live and fully functional.

Conclusion

This guide walks through how to deploy a full-stack Dockerized application to AWS using Terraform. You set up the infrastructure from scratch, including VPCs, subnets, security groups, and EC2 instances, and made sure your application could run securely inside a private subnet. You also added a bastion host to help us reach the private instances when needed.

On top of that, you configured a load balancer for public access and set up monitoring using Prometheus and Grafana. With this in place, you’re able to track system metrics and confirm that everything’s running as expected.

There are several ways in which this setup can be improved, one of which is including extensive logging within the system and application, but for now, this is a good start. If you enjoyed this article, you can head on to my page to read more DevOps-related articles.

How to Deploy a Full Stack App to Koyeb Using Docker Compose and CircleCI

EphraimX — Tue, 12 Aug 2025 08:54:17 +0000

CircleCI offers a developer-friendly and efficient CI/CD experience, making it an excellent choice for automating deployment workflows for containerized applications. In this guide, you'll learn how to deploy a full-stack Docker Compose application to Koyeb using CircleCI.

We’ll configure a custom pipeline that uses the Koyeb CLI and leverages the Dockerfile-based deployment mechanism. This approach mirrors our previous GitHub Actions, GitLab CI/CD, and Jenkins guides but focuses specifically on CircleCI's environment and capabilities.

If you haven’t yet reviewed the foundational article that walks through the project setup, Dockerfiles, and Docker Compose configuration, please start there: Deploying a Full Stack App to Koyeb Using Docker Compose and GitHub Actions

Once you're familiar with the application structure and build context, proceed with this article to integrate CircleCI into your workflow.

Prerequisites
Obtaining Your Koyeb API Token
Configuring CircleCI Environment Variables
CircleCI Pipeline Configuration
Deploying to Koyeb Using CircleCI
Conclusion

Prerequisites

Before diving into the CircleCI setup and deployment process, make sure you have the following:

Familiarity with the Full Stack Application Setup: This article builds on the foundation laid in the previous article, How to Deploy a Full Stack App to Koyeb Using Docker Compose and GitHub Actions. It’s recommended to review that article first for context on the application structure and Docker setup.
CircleCI Account: You need an active CircleCI account connected to your GitHub repository where the application code resides. If you don’t have one, sign up at circleci.com.
Koyeb Account: A Koyeb account is required to deploy your application. Simply create an account at koyeb.com if you don’t have one.
Koyeb API Token: You will need a Koyeb API token for authentication when deploying via CircleCI. Instructions for obtaining this token will be covered in the next section.
Basic Understanding of CircleCI Pipelines: If you are new to CircleCI pipelines or configuration, consider reviewing the official CircleCI documentation or refer to my article on setting up CI/CD pipelines with CircleCI.

Obtaining Your Koyeb API Token

To deploy your application to Koyeb using CircleCI, you'll need a Koyeb API token for authentication. Follow these steps to generate one:

Log in to your Koyeb account: Visit app.koyeb.com and sign in.
Access Account Settings: Click on your profile icon in the top-right corner and select "Account Settings".
Navigate to API Access Tokens: In the left sidebar, click on "API Access Tokens".
Create a New Token:
- Click the "Create API Access Token" button.
- Provide a name and description for the token to help identify its purpose.
- Click "Create".
Save the Token: Once generated, copy the token immediately. For security reasons, you won't be able to view it again.

Adding the Koyeb API Token to CircleCI

To securely store and use your Koyeb API token in your CircleCI pipeline, add it as an environment variable:

Log in to CircleCI: Go to circleci.com and sign in.
Navigate to Your Project: From the dashboard, select the project where your application resides.
Access Project Settings:
- Click on the gear icon next to your project name.
- Select "Project Settings".
Add an Environment Variable:
- In the left sidebar, click on "Environment Variables".
- Click the "Add Variable" button.
- In the "Name" field, enter KOYEB_API_TOKEN.
- In the "Value" field, paste the API token you copied earlier.
- Click "Add Variable" to save.

This environment variable will now be accessible in your CircleCI jobs and can be used to authenticate with the Koyeb API.

Setting Up CircleCI Configuration

First, create a .circleci directory at the root of your project if it doesn't already exist:

mkdir -p .circleci

Next, inside this directory, create a file named config.yml and paste the following code:

version: 2.1
jobs:
  deploy:
    docker:
      - image: ubuntu:24.04
    steps:
      - checkout
      - run:
          name: Install Curl and Koyeb CLI
          command: |
            apt-get update
            apt-get install -y curl
            curl -fsSL https://raw.githubusercontent.com/koyeb/koyeb-cli/master/install.sh | sh
            export PATH=$HOME/.koyeb/bin:$PATH
            export KOYEB_TOKEN=${KOYEB_API_TOKEN}
            koyeb app create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb || echo "App already exists"
            koyeb service create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb \
              --app glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb \
              --git github.com/EphraimX/glowberry-global-tax-structure-simulator-gha-docker-compose-koyeb \
              --instance-type free \
              --git-builder docker \
              --git-docker-dockerfile Dockerfile.koyeb \
              --port 3000:http \
              --route /:3000 \
              --privileged

workflows:
  build-and-deploy-to-koyeb:
    jobs:
      - deploy

Explanation of the CircleCI Configuration

version: 2.1 Specifies the CircleCI configuration version used.
jobs: Defines a set of jobs CircleCI will run. Here, we have a single job named deploy.
deploy job:
- docker: Runs the job inside an Ubuntu 24.04 Docker container.
- steps:
- checkout: Checks out your repository code so the job can use it.
- run step named Install Curl and Koyeb CLI:
  - Updates package lists and installs curl.
  - Downloads and installs the Koyeb CLI tool.
  - Updates the PATH to include the Koyeb CLI.
  - Sets the environment variable KOYEB_TOKEN from the CircleCI environment variable ${KOYEB_API_TOKEN}.
  - Runs koyeb app create to create the app if it doesn't already exist (ignores error if it does).
  - Runs koyeb service create with detailed parameters to deploy the app and service to Koyeb.
workflows: Defines the workflow that triggers jobs. Here, it runs the deploy job under the name build-and-deploy-to-koyeb.

Pushing to GitHub and Monitoring Your Deployment

Once your .circleci/config.yml file is ready and committed to your project repository, you can trigger the deployment process by pushing your code to GitHub.

Steps to Trigger Deployment

Commit your changes:

   git add .circleci/config.yml
   git commit -m "Add CircleCI config for Koyeb deployment"

Push to your repository:

   git push origin main

Replace main with your default branch if different.

Monitoring the Deployment on CircleCI

After pushing, CircleCI will automatically detect the changes and start the pipeline based on your config file.
To monitor the deployment progress:
1. Log in to your CircleCI dashboard at https://app.circleci.com/.
2. Navigate to your project’s pipeline page.
3. Click on the running job (usually labeled deploy under the workflow name build-and-deploy-to-koyeb).
4. View detailed logs of each step in the job, including:
  - Checkout your repo code.
  - Installation of curl and the Koyeb CLI.
  - Koyeb app and service creation commands.
Any errors during the deployment will appear here, allowing you to troubleshoot quickly.

Conclusion

In this guide, you’ve learned how to automate the deployment of a full-stack application to Koyeb using CircleCI and Docker Compose. By configuring your CircleCI pipeline with the Koyeb CLI, you ensure that every push to your repository triggers a seamless build and deployment process.

This approach eliminates manual deployment steps, reducing errors and accelerating your development workflow. With the environment variables securely stored in CircleCI and the deployment status easily monitored via the CircleCI dashboard, managing your application lifecycle becomes efficient and transparent.

If you want to explore other CI/CD tools, consider our articles on GitHub Actions, GitLab CI/CD, and Jenkins integrations with Koyeb.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Frontend App to Vercel Using Jenkins

EphraimX — Tue, 12 Aug 2025 08:17:49 +0000

While most frontend teams rely on Vercel’s default Git integrations for automatic deployments, there are times when you need more control. Jenkins gives you that power—with fully customizable pipelines, conditional logic, and integrations tailored to your workflow.

In this article, you’ll learn how to set up a Jenkins pipeline that automatically builds and deploys a frontend application to Vercel whenever changes are pushed to your main branch.

We’ll cover the entire workflow, including securely managing your Vercel token, writing your Jenkinsfile, and verifying the deployment on the Vercel dashboard—all without relying on Vercel’s default GitHub or GitLab integrations.

Prerequisites
Setting Up Jenkins for CI/CD
- Installing Jenkins via Docker (Recommended)
- Installing Jenkins via Script (Linux)
- First Time Setup
- Creating a Jenkins Pipeline with Git Polling
Creating the Jenkinsfile for Vercel Frontend Deployment
- Explaining the Jenkinsfile
Setting Up the Vercel Token in Jenkins
Deploying and Testing the Frontend Application
Conclusion

Prerequisites

Before diving in, ensure you’ve got the following ready:

A Vercel account with an existing project.
A Jenkins setup (self-hosted or via CI providers).
A Vercel token with project-level scope and an appropriate expiry set. (If you haven’t generated one before, refer to the A Practical Guide to Deploying Frontend Apps on Vercel Using GitHub Actions article for a detailed walkthrough.
Node.js installed in your Jenkins environment.
Familiarity with the structure of your frontend app.

We’re using the same TypeScript-based frontend application introduced in the A Practical Guide to Deploying Frontend Apps on Vercel Using GitHub Actions. Read through that article first to get a solid understanding of the app layout before continuing.

Setting Up Jenkins for CI/CD

Before we can deploy anything with Jenkins, we need to make sure it's up and running correctly. Below is a guide on how to install Jenkins, configure your admin account, and create a pipeline that polls a Git repository.

Option 1: Installing Jenkins via Docker (Recommended)

If you’re comfortable with Docker, this is the fastest way to get started:

docker run -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

Once running, open http://localhost:8080 in your browser.

Option 2: Installing Jenkins via Script (Linux)

If you're installing Jenkins natively:

# Add Jenkins repository and key
curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

# Update and install
sudo apt update
sudo apt install jenkins -y

# Start Jenkins
sudo systemctl start jenkins

Jenkins will be accessible at http://localhost:8080.

N.B: You have to have Java installed on your machine.

First Time Setup

After installation:

Unlock Jenkins: Visit http://localhost:8080. You’ll see a screen asking for an admin password. Run:

   sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Install Plugins: Choose Install Suggested Plugins.
Create Admin User: Set up your Jenkins username, password, and full name.
Confirm URL: You can leave the Jenkins URL as http://localhost:8080 unless you’re using a custom domain.

Creating a Jenkins Pipeline with Git Polling

Create a New Jenkins Pipeline Job:

From your Jenkins dashboard, click New Item.*
Enter a name for your job (e.g., frontend-deploy-vercel) and select Pipeline.
Click OK

Configure the Pipeline:

In the Pipeline job config, scroll to Pipeline → Definition, and set it to Pipeline script from SCM
Set SCM to Git
Paste your repository URL (GitHub or GitLab)
(Optional but recommended) Add credentials if the repo is private

Polling the Repository:

Scroll down to Build Triggers
Tick Poll SCM
Use the schedule H/5 * * * * to check for changes every 5 minutes (You can adjust this later for performance)

Once this setup is complete, Jenkins will automatically poll your repo and run the defined pipeline when changes are pushed to the main branch.

Creating the Jenkinsfile for Vercel Frontend Deployment

To automate your frontend deployments to Vercel, create a Jenkinsfile at the root of your repository and include the following configuration:

pipeline {
  agent any

  environment {
    VERCEL_TOKEN = credentials('VERCEL_TOKEN')
  }

  stages {
    stage('Build and Deploy Frontend Service') {
      steps {
        dir('frontend') {
          sh 'npm install'
          sh 'npm run build'
          sh 'npx vercel --prod --token $VERCEL_TOKEN --yes'
        }
      }
    }
  }

  post {
    success {
      echo 'Frontend deployed successfully to Vercel.'
    }
    failure {
      echo 'Frontend deployment failed.'
    }
  }
}

Note: This assumes the vercel CLI is already installed globally in your Jenkins agent. If not,add npm install -g vercel before the deploy step.

Explaining the Jenkinsfile

pipeline { ... }: Defines the structure of the Jenkins pipeline in a declarative format. This setup helps automate and manage CI/CD tasks in a readable and maintainable way.
agent any: Specifies that this pipeline can run on any available Jenkins agent. This provides flexibility for most general-purpose use cases.
environment { VERCEL_TOKEN = credentials('VERCEL_TOKEN') }: Loads the VERCEL_TOKEN from Jenkins’ secure credentials store. This token is necessary for authenticating with Vercel’s CLI. By storing it as a secret credential in Jenkins, you keep your deployment pipeline secure and avoid hardcoding sensitive data.
stages { ... }: Defines the main steps in the pipeline.
stage('Build and Deploy Frontend Service')
- dir('frontend'): Changes directory into the frontend folder, which contains the codebase for the frontend application.
- sh 'npm install': Installs all required dependencies.
- sh 'npm run build': Compiles the application into production-ready static assets.
- sh 'npx vercel --prod --token $VERCEL_TOKEN --yes': Deploys the compiled application to Vercel using the CLI. The --prod flag ensures it's a production deployment, --yes bypasses interactive prompts, and the --token ensures authenticated access.
post { ... }: Handles actions that should run after the pipeline completes:
- success: Logs a confirmation message if the deployment is successful.
- failure: Outputs an error message to aid in troubleshooting if the deployment fails.

Once this is in place, every time your pipeline runs, Jenkins will automatically build and deploy your frontend service to Vercel. Next, we’ll cover how to securely add your Vercel token to Jenkins using the credentials store.

Setting Up the Vercel Token in Jenkins

For Jenkins to deploy your frontend application to Vercel, you'll need to store your Vercel token securely using Jenkins' credentials manager.

Step 1: Access Jenkins Credentials

From your Jenkins dashboard, go to Manage Jenkins → Credentials → (global) → Add Credentials.

Step 2: Add the Vercel Token

In the Add Credentials dialog:
- Kind: Choose Secret text.
- Secret: Paste your Vercel token.
- ID: Enter VERCEL_TOKEN (ensure it matches what you’ve referenced in your Jenkinsfile).
- Description: (Optional) e.g., Vercel Deployment Token.
Click OK to save the token.

You can obtain your token from the Vercel dashboard:
Click your profile avatar → Account Settings → Tokens, then create a new one with a defined name, scope (recommended: project-specific), and expiration date.

Once stored, Jenkins will automatically inject the token into your pipeline when referenced using credentials('VERCEL_TOKEN').

Deploying and Testing the Frontend Application

After setting up the Jenkinsfile and securely storing your Vercel token, you can move ahead with deploying your frontend application.

Step 1: Execute the Build

Click Build Now to kick off the process.
Monitor progress in the Build History → select the build → Console Output.
Jenkins will:

Install frontend dependencies in the frontend/ directory.
Build the project using npm run build.
Deploy to Vercel using the CLI and the stored token.

Step 2: Confirm Deployment

Head to the Vercel dashboard and locate your project.
Confirm the successful deployment and access the production URL.
You can also inspect the deployment logs on Vercel for any additional diagnostics.

Conclusion

Using Jenkins to deploy frontend applications to Vercel brings structure and consistency to your release process. While Vercel’s Git integration works out of the box, Jenkins offers additional room to scale: conditional builds, preview environments, and integration with testing or linting tools.

In this article, we walked through setting up Jenkins, creating a pipeline job, storing Vercel tokens securely, and automating deployments with a structured Jenkinsfile. You also learned how to monitor builds and verify your production release via the Vercel dashboard.

This setup is a solid starting point for teams that want tight control over their CI/CD flows while still leveraging powerful hosting platforms like Vercel.

Found this guide useful? Follow EphraimX for more practical DevOps guides and automation tips. You can also connect with me on LinkedIn or explore more of my projects on my portfolio.

How to Deploy a Python Backend to Fly.io Using Jenkins

EphraimX — Tue, 12 Aug 2025 08:15:57 +0000

Jenkins is one of the most trusted CI/CD tools for developers who want full control over their automation workflows. In this guide, you’ll learn how to set up a Jenkins pipeline that automatically deploys a Python backend application to Fly.io anytime there’s a change in your main branch.

This process is ideal if you want to maintain complete control—no hidden processes, no magic buttons—just a clear, auditable, and customizable CI/CD pipeline.

In this guide, you’ll walk through the exact setup, from creating your Jenkinsfile to storing sensitive tokens securely, and finally testing your deployment live on Fly.io.

Prerequisites
Setting Up Jenkins for CI/CD
- Option 1: Installing Jenkins via Docker (Recommended)
- Option 2: Installing Jenkins via Script (Linux)
- First Time Setup
- Creating a Jenkins Pipeline with Git Polling
Creating the Jenkinsfile for Fly.io Deployment
- Explaining the Jenkinsfile
Setting Up the Fly.io API Token in Jenkins
Deploying and Testing the Backend Application
Conclusion

Prerequisites

To follow along with this tutorial, make sure you have the following:

A Fly.io account.
A Jenkins instance up and running (locally, on a server, or via a CI service).
A valid Fly.io API token. You can create one from your Fly.io dashboard under Access Tokens.
Basic understanding of your application's structure.

This article uses the same Python backend from our How to Set Up CI/CD for a Python Backend Application on Fly.io Using GitHub Actions. If you haven’t already, go through that article to understand how the application works and how the Dockerfile is structured. That knowledge will be important when setting up Jenkins correctly.

Setting Up Jenkins for CI/CD

Option 1: Installing Jenkins via Docker (Recommended)

If you’re comfortable with Docker, this is the fastest way to get started:

docker run -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts

Once running, open http://localhost:8080 in your browser.

Option 2: Installing Jenkins via Script (Linux)

If you're installing Jenkins natively:

# Add Jenkins repository and key
curl -fsSL https://pkg.jenkins.io/debian/jenkins.io-2023.key | sudo tee \
  /usr/share/keyrings/jenkins-keyring.asc > /dev/null

echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] \
  https://pkg.jenkins.io/debian binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null

# Update and install
sudo apt update
sudo apt install jenkins -y

# Start Jenkins
sudo systemctl start jenkins

Jenkins will be accessible at http://localhost:8080.

N.B.: You have to have Java installed on your machine.

First Time Setup

After installation:

Unlock Jenkins: Visit http://localhost:8080. You’ll see a screen asking for an admin password. Run:

   sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Install Plugins: Choose Install Suggested Plugins.
Create Admin User: Set up your Jenkins username, password, and full name.
Confirm URL: You can leave the Jenkins URL as http://localhost:8080 unless you’re using a custom domain.

Creating a Jenkins Pipeline with Git Polling

Create a New Jenkins Pipeline Job:
- From your Jenkins dashboard, click New Item.*
- Enter a name for your job (e.g., backend-deploy-flyio) and select Pipeline.
- Click OK
Configure the Pipeline:
- In the Pipeline job config, scroll to Pipeline → Definition, and set it to Pipeline script from SCM
- Set SCM to Git
- Paste your repository URL (GitHub or GitLab)
- (Optional but recommended) Add credentials if the repo is private
Polling the Repository:
- Scroll down to Build Triggers
- Tick Poll SCM
- Use the schedule H/5 * * * * to check for changes every 5 minutes (You can adjust this later for performance)

Once this setup is complete, Jenkins will automatically poll your repo and run the defined pipeline when changes are pushed to the main branch.

Creating the Jenkinsfile for Fly.io Deployment

To configure Jenkins to deploy your Python backend service to Fly.io, you need to create a file named Jenkinsfile in the root directory of your repository. This file defines the steps Jenkins will execute as part of the pipeline.

Paste the following content into your Jenkinsfile:

pipeline {
  agent any

  environment {
    FLY_API_TOKEN = credentials('FLY_API_TOKEN')
  }

  stages {
    stage('Build and Deploy Backend Service') {
      steps {
        dir('backend') {
          sh 'flyctl deploy --remote-only'
        }
      }
    }
  }

  post {
    success {
      echo 'Backend deployed successfully to Fly.io.'
    }
    failure {
      echo 'Backend deployment failed.'
    }
  }
}

Explaining the Jenkinsfile

pipeline { ... }: This is a declarative Jenkins pipeline. It provides a clear structure for defining the build, test, and deployment lifecycle of your application.
agent any: Specifies that the pipeline can run on any available Jenkins agent. This is the default setting and is appropriate unless you are working with a customized or restricted build environment.
environment { FLY_API_TOKEN = credentials('FLY_API_TOKEN') }: This section declares environment variables required during the pipeline execution. The credentials() function pulls the FLY_API_TOKEN from Jenkins' credentials store, ensuring that your deployment token is not exposed in plain text. You will need to configure this credential in Jenkins, which we will cover later.
stages { ... }: Defines the sequential stages that Jenkins will execute.
stage('Build and Deploy Backend Service'): This stage handles the core deployment process.
- dir('backend'): Navigates into the backend directory of your repository, where the backend application code resides.
- sh 'flyctl deploy --remote-only': Executes the Fly.io deployment command using the flyctl CLI. The --remote-only flag ensures that the application is packaged locally but built and deployed on Fly.io’s infrastructure, reducing dependencies on your local build environment.
post { ... }: This block defines actions that should be executed after the pipeline has finished.
- success: Prints a message to the console when the deployment succeeds.
- failure: Prints a message when the deployment fails, which can assist in diagnosing issues during execution.

With this setup in place, Jenkins will be able to automate the process of deploying your backend application to Fly.io whenever a new change is pushed to your repository. In the next section, we will walk you through the secure process of adding the FLY_API_TOKEN to Jenkins.

Setting Up the Fly.io API Token in Jenkins

To enable automated deployments to Fly.io from Jenkins, you need to securely add your Fly.io API token to the Jenkins credentials system. Follow these steps:

Step 1: Access the Credentials Manager

Open your Jenkins dashboard.
Navigate to Manage Jenkins → Credentials → (global) → Add Credentials.

Step 2: Add the Fly.io Token

In the Add Credentials form:
- Kind: Select Secret text.
- Secret: Paste your Fly.io API token.
- ID: Enter FLY_API_TOKEN (make sure it matches the ID used in your Jenkinsfile).
- Description: (Optional) Add something like Fly.io API Token.
Click OK to save the token.

You can retrieve your Fly.io token by running fly auth token in your terminal after authenticating via fly auth login, or you can generate one from your Fly.io dashboard under Account Settings → Access Tokens.

This credential will be securely accessed in the Jenkins pipeline using the credentials('FLY_API_TOKEN') directive inside the environment block.

Deploying and Testing the Backend Application

Once you’ve created your Jenkinsfile at the root of your repository and configured your Fly.io token, you’re ready to trigger the deployment.

Step 1: Run the Pipeline

Click Build Now to trigger your first run.
Navigate to the Build History and click the build number to view logs.
If configured correctly, you’ll see Jenkins:
1. Install Fly CLI
2. Enter the backend/ directory
3. Deploy your backend service to Fly.io using flyctl deploy --remote-only

Step 2: Verify Deployment

Visit your Fly.io dashboard and confirm the deployment under the targeted app.
You can also run fly status locally to inspect the deployment.

Conclusion

Integrating Jenkins into your Fly.io deployment pipeline gives you more than just automation—it offers control. By managing your own CI/CD infrastructure, you can introduce custom build logic, stage-specific testing, and gated deploy conditions, all without compromising flexibility.

In this guide, you learned how to install Jenkins, connect your Git repository, and configure a Jenkinsfile that handles your backend deployment with Fly.io. You also saw how to monitor builds directly from the Jenkins dashboard and verify successful deployments via the Fly.io console.

With this foundation in place, you're better equipped to scale your CI/CD setup, customize build stages, and adopt a more production-grade engineering workflow.

If this helped you, consider following EphraimX on GitHub for more articles and open-source automation projects. You can also follow me on LinkedIn and explore more of my work on my portfolio.

How to Deploy a Full Stack Application to Koyeb Using Docker Compose, Terraform, and Jenkins

EphraimX — Mon, 23 Jun 2025 13:39:00 +0000

In this article, we will walk you through the process of automating the deployment of a full-stack application on Koyeb using Terraform for infrastructure as code, combined with Jenkins as the continuous integration and continuous deployment (CI/CD) tool. Jenkins offers a flexible and widely adopted platform for orchestrating pipelines, enabling teams to build, test, and deploy applications efficiently.

Before proceeding, it is recommended to review the previous article on deploying with Docker Compose and Terraform using GitHub Actions to understand the Terraform configuration and Docker setup. This article will build upon that foundation by focusing on Jenkins for managing the CI/CD pipeline.

We will cover how to configure Jenkins to execute Terraform commands that provision and manage your Koyeb infrastructure, ensuring your application environment is consistent and up-to-date. Whether you’re new to Jenkins or Terraform, this guide provides a clear, step-by-step approach to integrating these powerful tools for seamless cloud deployments.

Prerequisites
Obtaining Your Koyeb API Token and Setting Up Jenkins Credentials
Creating the Jenkins Pipeline (Jenkinsfile)
Pushing Code and Monitoring Deployment
Conclusion

Prerequisites

Before proceeding with this guide, ensure you have the following:

Familiarity with Jenkins pipelines and how to create and manage Jenkins jobs. If you are new to Jenkins pipelines, please refer to this comprehensive Jenkins pipeline tutorial for step-by-step guidance.
Access to a Jenkins server with permission to create pipelines and manage credentials.
A Koyeb account with a valid API token. This token is necessary to authenticate Jenkins with Koyeb for deployment. Instructions for obtaining this token are provided in the next section.
The repository containing the Terraform configuration files for deploying the application to Koyeb. If you haven’t set this up yet, see the supporting article on deploying with Terraform and GitHub Actions for detailed configuration.
A basic understanding of Terraform and Docker Compose is helpful, though the Terraform configuration is covered in the referenced article above.

Obtaining Your Koyeb API Token and Configuring Jenkins Credentials

To enable Jenkins to deploy your application to Koyeb, you need to generate a Koyeb API token and securely store it in Jenkins as a credential.

How to Get Your Koyeb API Token

Log in to your Koyeb dashboard at https://app.koyeb.com.
Navigate to Account Settings (usually accessible from the user profile menu).
Select the API Tokens section.
Click Create New Token.
Give your token a descriptive name (e.g., Jenkins Deployment Token) and set appropriate permissions.
Generate the token and copy it immediately — you won’t be able to view it again.

Adding the API Token to Jenkins Credentials

Log in to your Jenkins server.
Go to Manage Jenkins > Manage Credentials.
Select the appropriate domain (usually Global).
Click Add Credentials.
For Kind, select Secret text.
In the Secret field, paste your copied Koyeb API token.
Give it an ID (for example, KOYEB_API_TOKEN) and optionally a description.
Save the credential.

Creating the Jenkinsfile for Terraform Deployment

To automate the deployment of your full-stack application to Koyeb using Terraform and Jenkins, you need to create a Jenkinsfile in the root directory of your repository.

Steps:

In your project’s root directory, create a file named Jenkinsfile.
Paste the following pipeline script into the Jenkinsfile:

pipeline {
  agent any

  environment {
    KOYEB_TOKEN = credentials('KOYEB_API_TOKEN')
  }

  stages {
    stage('Install Terraform & Deploy') {
      steps {
        dir('terraform') {
          sh '''
            #!/bin/bash
            set -e
            if [ ! -f terraform ]; then
              echo "Downloading Terraform binary..."
              curl -fsSL https://releases.hashicorp.com/terraform/1.11.4/terraform_1.11.4_linux_amd64.zip -o terraform.zip
              unzip terraform.zip
              chmod +x terraform
              rm terraform.zip
            fi

            ./terraform version
            ./terraform init
            ./terraform fmt
            ./terraform validate
            ./terraform plan
            ./terraform apply --auto-approve
          '''
        }
      }
    }
  }
}

Explanation:

agent any: Runs the pipeline on any available Jenkins agent.
environment: The KOYEB_TOKEN environment variable is set using the Jenkins credential ID KOYEB_API_TOKEN. Make sure this matches the ID you configured in Jenkins credentials.
stage ‘Install Terraform & Deploy’:
- The script moves into the terraform directory where your Terraform configuration files reside.
- Checks if Terraform binary exists; if not, downloads and extracts Terraform 1.11.4.
- Runs Terraform commands in sequence:
- terraform version to confirm the version,
- terraform init to initialize the working directory,
- terraform fmt to check formatting,
- terraform validate to validate the configuration,
- terraform plan to preview changes,
- terraform apply to apply infrastructure changes automatically.

Pushing Code and Monitoring Jenkins Pipeline Deployment

Pushing Your Code to Prepare for Deployment

After creating and committing the Jenkinsfile along with your Terraform configuration files in the terraform directory, push the changes to your remote Git repository:

git add Jenkinsfile terraform/
git commit -m "Add Jenkins pipeline for Terraform deployment to Koyeb"
git push origin main

Note: Depending on your Jenkins setup, pushing your code may not automatically trigger the pipeline. You might need to manually start the build by navigating to your Jenkins job and clicking Build Now.

Monitoring the Jenkins Pipeline

Access Jenkins Dashboard: Log in to your Jenkins instance and go to the dashboard.
Locate Your Pipeline Job: Find the pipeline job configured for this repository (e.g., glowberry-terraform-deploy).
Start the Build (If Needed): If the pipeline doesn’t start automatically, click Build Now to initiate the deployment.
View Build History: Once started, your build will appear in the build history list.
Open Build Details: Click on the latest build number for detailed information.
Follow Console Output: Click Console Output to monitor the pipeline logs in real time, including:
- Terraform download and setup
- Terraform init, fmt, validate
- Terraform plan and apply
- Deployment progress and errors (if any)
Confirm Deployment Success: The build is successful if all steps complete without error, and Terraform apply confirms resource creation or updates.

Conclusion

In this article, you learned how to automate the deployment of a full-stack application to Koyeb using Terraform within a Jenkins pipeline. By integrating Infrastructure as Code with your CI/CD process, you achieve consistent, repeatable, and manageable deployments.

With the provided Jenkinsfile and Terraform configuration, you can efficiently provision and update your cloud resources while maintaining full control through Jenkins. This approach enhances deployment reliability and streamlines your development workflow.

For a deeper understanding of the Terraform configuration used here, refer to the supporting article linked in the prerequisites section.

Feel free to explore additional enhancements such as pipeline notifications, environment-specific deployments, or secret management integrations to optimize your deployment process further.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Full Stack App to Koyeb Using Docker Compose and Jenkins

EphraimX — Fri, 20 Jun 2025 13:37:40 +0000

Jenkins remains one of the most powerful and flexible tools in the CI/CD space—widely adopted for its extensibility and tight control over build pipelines. In this article, we’ll explore how to deploy a full-stack Docker Compose application to Koyeb using a Jenkins pipeline.

Unlike managed CI/CD platforms like GitHub Actions or GitLab CI/CD, Jenkins gives you complete control over the environment and pipeline steps. This makes it a great choice when you need custom configurations, self-hosted agents, or integration with complex systems.

We’ll skip the broader details of the application setup itself. If you haven’t already, you should read the foundational GitHub Actions article for insights into the Dockerfiles, Docker Compose configuration, and the overall structure of the Glowberry Global Tax Structure Simulator app. This guide will focus entirely on the Jenkins pipeline configuration and how to automate the deployment to Koyeb from there.

Let’s get started.

Prerequisites
Configuring Koyeb API Token in Jenkins
Jenkins Pipeline Setup
Understanding the Deployment Steps
Executing the Pipeline
Conclusion

Prerequisites

Before proceeding with this Jenkins-specific guide, ensure the following prerequisites are met:

Read the Supporting Article: This article builds on concepts covered in How to Deploy a Full Stack App to Koyeb Using Docker Compose and GitHub Actions. That guide explains the structure of the application, Docker setup, and how Docker Compose and Dockerfile.koyeb drive the deployment. Please review it first to understand the foundational components.
Jenkins Installed and Running: You should have Jenkins installed and accessible on your system or through a server (such as localhost or a cloud VM).
Jenkins User with Required Permissions: Ensure your Jenkins user has the necessary permissions to manage credentials and execute pipelines.
Install Required Jenkins Plugins:
- Pipeline
- GitHub Integration
- Git Plugin
- Docker Pipeline
- Environment Injector (for managing environment variables)
Koyeb Account: If you haven’t already, create a Koyeb account. This guide assumes you have access to it and can generate an API token.
Koyeb API Token: You’ll need a valid Koyeb API token for deployment. We'll walk through storing it securely in Jenkins shortly.
GitHub Repository: Your application should be stored in a GitHub repository accessible to Jenkins.

Absolutely. Here’s a concise section explaining how to retrieve your Koyeb API key before using it in Jenkins:

Retrieving Your Koyeb API Token

To interact with Koyeb from CI/CD tools like Jenkins, you'll need a Koyeb API token. Here's how to get it:

How to Generate a Koyeb API Token

Log In to Your Koyeb Account Visit https://app.koyeb.com and sign in
Go to Account Settings Click on your profile icon at the top-right corner and select Account Settings.
Navigate to the API Tab In the settings sidebar, click on API.
Generate a New API Token
- Click the Generate API Token button.
- Give your token a descriptive name (e.g., jenkins-deployment-token).
- Choose the appropriate scope (typically full access for CI/CD).
- Click Create Token and copy the token immediately—this is the only time it will be fully visible.

Once retrieved, you can securely store the token in Jenkins as described in the next section.

Configuring Koyeb API Token in Jenkins

To allow Jenkins to securely deploy your application to Koyeb, you'll need to store your Koyeb API token as a Secret Text credential in Jenkins. Follow these steps carefully:

Step-by-Step: Store Koyeb API Token in Jenkins

Access Jenkins Dashboard Log into your Jenkins instance.
Navigate to Credentials From the sidebar, go to: Manage Jenkins → Credentials → (select a domain or Global) → Add Credentials
Select Credential Type
- Kind: Choose Secret text
- Secret: Paste your Koyeb API token here
- ID: Use a simple identifier like KOYEB_API_TOKEN
- Description: Optional, e.g., “Koyeb token for CI/CD deployment”
Save Click OK to save the credential.

Jenkins will now securely reference this token using the ID KOYEB_API_TOKEN in your pipeline.

New to Jenkins Pipelines?

If you're unfamiliar with how to create or run a Jenkins pipeline, I’ve written a companion article to guide you through the process. It covers creating a Jenkinsfile, setting up a pipeline job, and connecting it to a GitHub repository. Refer to the Jenkins Pipeline Setup Guide

Creating the `Jenkinsfile` for Your Pipeline

With your Koyeb API token securely stored in Jenkins credentials (as KOYEB_API_TOKEN), you’re ready to define your CI/CD pipeline.

Follow these steps:

1. Create the `Jenkinsfile`

In the root directory of your project repository, create a file named Jenkinsfile with no extension.

2. Paste the Following Code into the `Jenkinsfile`

pipeline {

  agent any

  environment {
    KOYEB_API_TOKEN = credentials('KOYEB_API_TOKEN')
  }

  stages {

    stage('Koyeb Setup and Deploy Job') {
      steps {
        // sh 'apt install curl' // Uncomment if curl is not installed on your Jenkins agent
        sh 'curl -fsSL https://raw.githubusercontent.com/koyeb/koyeb-cli/master/install.sh | sh'
        sh '''
          export PATH="/var/jenkins_home/.koyeb/bin:$PATH"
          export KOYEB_TOKEN=$KOYEB_API_TOKEN
          koyeb app create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb
          koyeb service create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb \
            --app glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb \
            --git github.com/EphraimX/glowberry-global-tax-structure-simulator-gha-docker-compose-koyeb \
            --instance-type free \
            --git-builder docker \
            --git-docker-dockerfile Dockerfile.koyeb \
            --port 3000:http \
            --route /:3000 \
            --privileged
        '''
      }
    }

  }
}

pipeline: Declares a declarative Jenkins pipeline.
agent any: Runs the pipeline on any available agent.
environment: Injects the stored Koyeb API token from Jenkins credentials into the environment.
stage('Koyeb Setup and Deploy Job'): This stage does the heavy lifting:
- Installs the Koyeb CLI via curl.
- Updates the PATH so the CLI is accessible.
- Sets the KOYEB_TOKEN environment variable.
- Uses the Koyeb CLI to:
  - Create the app (if it doesn’t already exist).
  - Deploy the service using:
  - Dockerfile.koyeb as the deployment entry point.
  - The docker Git builder.
  - Port 3000 exposed publicly.
  - Route all root requests to port 3000.
  - The --privileged flag (important for some internal service behavior).

If curl is not installed on your Jenkins agent, simply uncomment the apt install curl line near the top.

Deploying to Koyeb with Jenkins

Once your Jenkinsfile is in place and committed to your repository, deploying your application becomes a matter of triggering a Jenkins pipeline build. Here's how to proceed:

1. Push Your Code to the Repository

Commit and push your updated project — including the new Jenkinsfile — to the main branch (or the branch your Jenkins job is configured to track).

git add Jenkinsfile
git commit -m "Add Jenkins pipeline for Koyeb deployment"
git push origin main

Make sure your Jenkins job is configured to monitor the correct repository and branch.

2. Trigger the Jenkins Job

If your Jenkins project is configured for polling SCM or webhook-based triggers, Jenkins will automatically detect the new commit and run the job.
If not, you can manually trigger the job from the Jenkins dashboard by clicking "Build Now".

3. Monitor the Build

Navigate to your Jenkins dashboard.
Click on your project, then click on the build under the "Build History" section.
Use the "Console Output" tab to monitor the step-by-step progress of your pipeline.

If everything is configured correctly:

The pipeline will install the Koyeb CLI.
Authenticate with the Koyeb API using your secret token.
Create the app and service.
Deploy your application using the Dockerfile.koyeb and the docker-compose.yml file.

4. Verify the Deployment

After a successful build:

Visit your Koyeb dashboard.
Locate the app and service by name.
Open the service URL to access your deployed frontend application running on port 3000.

Conclusion

Integrating Jenkins into your CI/CD workflow to deploy a full-stack Docker Compose application to Koyeb gives you fine-grained control over your build and release processes. By leveraging the Dockerfile.koyeb, automating deployment through the Koyeb CLI, and managing secrets securely within Jenkins, you can achieve repeatable and consistent deployments across environments.

If you're already maintaining infrastructure with Jenkins, this method fits seamlessly into your pipeline architecture. For developers transitioning from other CI/CD systems, Jenkins remains a powerful and flexible alternative, especially when paired with a platform like Koyeb that abstracts infrastructure complexity.

With this foundation in place, you can now iterate faster, maintain confidence in your deployments, and explore more advanced Jenkins features such as parallel builds, test automation, and staged rollouts.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Full-Stack Application to Koyeb Using Docker Compose, Pulumi and GitLab CI/CD

EphraimX — Mon, 16 Jun 2025 13:28:07 +0000

This article demonstrates how to deploy your full-stack application to Koyeb by leveraging Pulumi infrastructure-as-code alongside GitLab CI/CD pipelines.

Building on the foundational Pulumi setup detailed in the previous GitHub Actions Pulumi article, this guide focuses on integrating Pulumi deployment workflows within GitLab CI/CD. We won't dive deep into Pulumi code here, so if you haven't set up your Pulumi infrastructure yet, please refer to the earlier article for full context.

By the end of this tutorial, you'll have a clear understanding of configuring GitLab pipelines to automate infrastructure provisioning and application deployment using Pulumi on Koyeb.

Prerequisites
Setting Up GitLab CI/CD
Pushing Code and Triggering Deployment
Monitoring Deployment Progress
Conclusion

Prerequisites

Complete the foundational Pulumi setup as detailed in the Deploying a Full Stack Application with Pulumi on Koyeb Using GitHub Actions article.
Ensure your Pulumi project directory contains these files: Pulumi.yaml, __main__.py, requirements.txt, and .gitignore.
Have a GitLab repository with your Pulumi project committed.
Enable GitLab CI/CD on your repository.
Add the following environment variables in your GitLab project settings under Settings > CI/CD > Variables:
- KOYEB_TOKEN
- PULUMI_ACCESS_TOKEN
Basic understanding of GitLab pipelines and YAML configuration.
Confirm your Pulumi stack is initialized and properly configured as described in the foundational article.

Setting Up GitLab CI/CD

To automate your Pulumi deployment using GitLab CI/CD, start by creating a .gitlab-ci.yml file at the root of your repository. This file defines your pipeline and deployment process.

Before proceeding, ensure you have added the following environment variables to your GitLab project's CI/CD settings under Settings > CI/CD > Variables:

KOYEB_API_TOKEN- Your Koyeb API token.
PULUMI_ACCESS_TOKEN- Your Pulumi access token.

Next, create .gitlab-ci.yml with the following content:

stages:
  - pulumi_setup_and_deploy_koyeb

.standard-rules:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

setup-and-deploy-pulumi:
  stage: pulumi_setup_and_deploy_koyeb
  image: python:3.11-slim
  before_script:
    - cd pulumi-koyeb
    - apt-get update
    - apt-get install -y curl
    - curl -fsSL https://get.pulumi.com | sh
    - mkdir -p ~/.pulumi/plugins/resource-koyeb-v0.1.11
    - curl -L https://github.com/koyeb/pulumi-koyeb/releases/download/v0.1.11/pulumi-resource-koyeb-v0.1.11-linux-amd64.tar.gz | tar -xz -C ~/.pulumi/plugins/resource-koyeb-v0.1.11
    - export PATH=$HOME/.pulumi/bin:$PATH
  script:
    - pulumi login
    - pulumi stack select glowberry-dev-gha-stack || pulumi stack init glowberry-dev-gha-stack
    - pulumi preview
    - pulumi up -y
  variables:
    KOYEB_TOKEN: $KOYEB_API_TOKEN
    PULUMI_ACCESS_TOKEN: $PULUMI_ACCESS_TOKEN

Explanation of the Pipeline

Stages: Defines a single stage called pulumi_setup_and_deploy_koyeb which runs the deployment job.
Rules: The .standard-rules ensure the pipeline runs only on the default branch (usually main).
Job setup-and-deploy-pulumi:
- image: Uses the official slim Python 3.11 Docker image as the runtime environment.
- before_script:
- Changes directory to pulumi-koyeb where your Pulumi project files are located.
- Updates the package lists and installs curl.
- Installs the Pulumi CLI by downloading and running its installation script.
- Downloads and installs the Koyeb Pulumi provider plugin into Pulumiâ€™s plugin directory.
- Adds Pulumi CLI to the PATH environment variable to make the pulumi command available.
- script:
- Runs pulumi login to authenticate with the Pulumi service.
- Selects the Pulumi stack named glowberry-dev-gha-stack or creates it if it doesn't exist.
- Runs pulumi preview to display the planned infrastructure changes.
- Executes pulumi up -y to apply the changes automatically, deploying your app to Koyeb.
- variables:
- Passes the necessary KOYEB_TOKEN and PULUMI_ACCESS_TOKEN as environment variables inside the job, linked to your GitLab project's stored secrets.

This pipeline fully automates the Pulumi infrastructure deployment process to Koyeb each time you push changes to the default branch, making your CI/CD flow smooth and reliable.

Pushing Code and Monitoring Your Pulumi Deployment

After setting up your .gitlab-ci.yml pipeline file, push your changes to the default branch (usually main) to trigger the GitLab CI/CD pipeline and start your Pulumi deployment on Koyeb.

Run the following commands in your terminal:

git add .
git commit -m "Add Pulumi deployment pipeline for Koyeb on GitLab"
git push origin main

This push will automatically start the CI/CD pipeline you configured, which will execute the Pulumi deployment job.

Monitoring Deployment Progress in GitLab

Navigate to Your GitLab Project: Open your GitLab repository in a browser.
Go to the CI/CD > Pipelines Section: In the left sidebar, click on CI/CD and then Pipelines to see the list of pipeline runs.
Find the Latest Pipeline Run: The pipeline triggered by your recent push should be at the top of the list. Click on the pipeline status (e.g., running, passed, failed) to view its details.
View Job Logs: Click on the job named setup-and-deploy-pulumi to see real-time logs of the deployment process. You can monitor:
- The Pulumi CLI installation steps.
- Authentication with Pulumi and Koyeb.
- Pulumi preview output showing the planned infrastructure changes.
- The progress and completion of pulumi up which applies your infrastructure updates.
Confirm Successful Deployment: A successful pipeline run indicates your application and infrastructure have been deployed or updated on Koyeb via Pulumi.

This monitoring approach helps you verify your deployment's status and troubleshoot any issues early by examining the job logs in detail.

Conclusion

In this guide, you successfully set up a GitLab CI/CD pipeline to deploy your Glowberry Tax Simulator infrastructure on Koyeb using Pulumi. By leveraging Pulumi's powerful infrastructure-as-code capabilities combined with GitLab's robust CI/CD platform, you can automate deployments efficiently and reliably.

For a deeper understanding of the Pulumi infrastructure code and setup, refer to our foundational Pulumi article. With this pipeline in place, you are well-equipped to manage and scale your cloud infrastructure as your application grows.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Full Stack Application to Koyeb Using Docker Compose, Terraform, and GitLab CI/CD

EphraimX — Fri, 13 Jun 2025 13:20:22 +0000

Deploying a full stack application efficiently requires combining reliable infrastructure management with automated deployment pipelines. In this article, we’ll walk through how to deploy a Docker Compose-based full stack app to Koyeb using Terraform for infrastructure as code and GitLab CI/CD for continuous integration and deployment.

If you haven’t already, please refer to our detailed guide on deploying with Docker Compose, Terraform, and GitHub Actions, which covers the Terraform configuration and Docker setup (in a linked article) in depth. This article focuses specifically on adapting that setup to GitLab’s CI/CD platform, so you can leverage GitLab pipelines to automate your deployments seamlessly.

By the end, you’ll have a clear understanding of configuring GitLab CI/CD to work hand-in-hand with Terraform to deploy and manage your application on Koyeb, enabling a modern, scalable, and maintainable deployment workflow.

Prerequisites
Setting Up Koyeb API Token in GitLab CI/CD
GitLab CI/CD Pipeline Configuration
Deploying to Koyeb with GitLab CI/CD
Conclusion

Prerequisites

Before proceeding, ensure you have completed the following:

Familiarity with the Docker Compose setup: This article builds upon the Docker Compose configuration explained in How to Deploy a Full Stack App to Koyeb Using Docker Compose and GitHub Actions. Please review that article if you haven’t already.
A Koyeb account: Sign up at https://www.koyeb.com and verify your account.
GitLab repository access: Ensure your full stack application code is hosted on a GitLab repository.
Basic knowledge of GitLab CI/CD: If you’re new to GitLab CI/CD, familiarize yourself with GitLab’s pipeline concepts and how to add CI/CD variables. The official documentation is a good starting point: GitLab CI/CD Basics
Koyeb API Token: You will need your Koyeb API token to allow GitLab to deploy your app. Instructions on obtaining and adding this token to your GitLab project’s CI/CD variables will be covered in the next section.

Obtaining Your Koyeb API Token and Adding It to GitLab

Get your Koyeb API token:
- Log in to your Koyeb dashboard.
- Navigate to Account Settings > API Tokens.
- Generate a new token if you don’t have one already.
- Copy the token securely.
Add the token to GitLab:
- Go to your GitLab project.
- Navigate to Settings > CI/CD > Variables.
- Click Add Variable and enter:
  - Key: KOYEB_API_TOKEN
  - Value: (paste your Koyeb API token)
  - Enable Masked and Protected options for security.
- Save the variable.

GitLab CI/CD Configuration Explained

To get started with GitLab CI/CD, create a file named .gitlab-ci.yml in the root directory of your repository. This file will define the pipeline stages and the necessary steps to build and deploy your application using Terraform on Koyeb. Paste the following configuration into the file to automate the deployment process.

stages:
  - terraform_build_and_deploy

.standard-rules:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

koyeb-deploy:
  stage: terraform_build_and_deploy
  image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest
  script:
    - cd terraform
    - gitlab-terraform init
    - gitlab-terraform fmt
    - gitlab-terraform validate
    - gitlab-terraform plan
    - gitlab-terraform apply
  variables:
    KOYEB_TOKEN: $KOYEB_API_TOKEN

Breakdown

stages: Defines the pipeline stages; here, there’s a single stage called terraform_build_and_deploy.
.standard-rules: This reusable rule ensures the pipeline runs only when the commit is pushed to the default branch (usually main).
koyeb-deploy job:
- stage: Specifies which stage this job belongs to.
- image: Uses an official Terraform Docker image optimized for GitLab CI to run the Terraform commands.
- script: Runs the following Terraform commands inside the terraform directory:
- gitlab-terraform init: Initializes Terraform.
- gitlab-terraform fmt: Checks Terraform code formatting.
- gitlab-terraform validate: Validates Terraform configuration files.
- gitlab-terraform plan: Creates and shows an execution plan.
- gitlab-terraform apply: Applies the Terraform plan automatically, deploying your infrastructure on Koyeb.
- variables:
- KOYEB_TOKEN: Passes the Koyeb API token securely from the GitLab CI variable KOYEB_API_TOKEN to authenticate Terraform with Koyeb.

Pushing Code and Monitoring Your Terraform Deployment

Triggering Deployment by Pushing Code

After committing and pushing your updated Terraform configuration along with the .gitlab-ci.yml pipeline file to the default branch (usually main), GitLab CI/CD will automatically trigger the deployment pipeline.

To push your changes, use:

git add .
git commit -m "Add Terraform deployment pipeline for Koyeb"
git push origin main

This push initiates the CI/CD pipeline, which runs the configured jobs to deploy or update your application on Koyeb.

Monitoring the Deployment Process

Navigate to Your GitLab Project: Open your project on the GitLab web interface.
Go to the Build Pipelines Section: Click on Build in the left sidebar, then select Pipelines. This shows all pipeline runs for your project.
Find the Latest Pipeline Run: Locate the most recent pipeline triggered by your push to the main branch, and click on its status to view details.
Review Job Logs: Inside the pipeline view, click on the koyeb-deploy job to see step-by-step logs. Look for:
- Successful checkout of your code repository.
- Terraform initialization and formatting checks completing without errors.
- Validation of Terraform configuration.
- Execution of the plan showing infrastructure changes.
- Application of the changes with terraform apply completing successfully, confirming your app is deployed or updated on Koyeb.

Conclusion

Deploying your full-stack application to Koyeb using Terraform and GitLab CI/CD streamlines infrastructure management and deployment into an automated, reliable process. By leveraging Terraform’s declarative configuration and GitLab’s powerful pipeline capabilities, you can confidently provision, update, and maintain your deployment environment with minimal manual intervention.

With the setup covered in this guide, every push to your main branch will trigger an automated workflow that handles infrastructure provisioning and application deployment seamlessly. This approach not only boosts development productivity but also ensures consistent, repeatable deployments.

If you’re new to Terraform or GitLab pipelines, consider exploring additional resources to deepen your understanding. Once comfortable, you can extend this setup with advanced Terraform modules, multi-environment deployments, or integration with other tools.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Full Stack App to Koyeb Using Docker Compose and GitLab CI/CD

EphraimX — Mon, 09 Jun 2025 13:42:10 +0000

Deploying applications consistently across different CI/CD tools is a skill every engineer should master. This article focuses on how to use GitLab CI/CD to deploy a full stack Docker Compose application to Koyeb, a developer-friendly PaaS.

If you’ve already followed our GitHub Actions deployment guide for the same project, you’re in the right place. We won’t rehash the application structure, Dockerfiles, or Docker Compose file — those details remain unchanged. Instead, this guide zeroes in on the CI/CD configuration differences specific to GitLab.

You’ll learn how to:

Set up your .gitlab-ci.yml file for Docker Compose deployments.
Use the Koyeb CLI within GitLab runners.
Trigger deployments by pushing to the main branch.
Monitor and debug deployment from GitLab’s UI.

Prerequisites
GitLab CI/CD Configuration
- Getting the Koyeb API Token
- .gitlab-ci.yml Explained
Deploying to Koyeb
Conclusion

Prerequisites

Before you begin, ensure the following:

Review the supporting article: If you haven’t already, please read How to Deploy a Full Stack App to Koyeb Using Docker Compose and GitHub Actions as it covers the core setup including Dockerfiles and Docker Compose configuration. This article will focus mainly on the GitLab CI/CD pipeline.
A Koyeb account: Create a free account at https://www.koyeb.com if you don’t have one already.
Koyeb API token: You will need this token to authenticate deployments from GitLab. Instructions on how to generate this token will be covered later in this article.
GitLab repository: Your application code should be pushed to a GitLab repository with the necessary Dockerfiles and Docker Compose configuration included.
Basic knowledge of GitLab CI/CD: Familiarity with GitLab pipelines and .gitlab-ci.yml syntax will help you follow along smoothly.
Docker installed locally (optional): To build and test your Docker containers before pushing changes to GitLab.

GitLab CI/CD Configuration and Koyeb API Token Setup

Getting Your Koyeb API Token

To deploy your app to Koyeb via GitLab CI/CD, you’ll need an API token to authenticate deployment commands.

Log in to your Koyeb account.
Navigate to Account Settings → API Tokens.
Click Create New Token, give it a meaningful name (e.g., GitLab CI/CD Deployment), and generate the token.
Copy the token immediately — you won’t be able to see it again.

Adding the API Token to GitLab CI/CD Variables

Open your GitLab project.
Go to Settings → CI/CD → Variables.
Click Add Variable.
Enter the following:
- Key: KOYEB_API_TOKEN
- Value: Paste your copied Koyeb API token here
- Enable Masked and Protected to keep the token secure.
Save the variable.

Understanding the `.gitlab-ci.yml` File

stages:
  - build-and-deploy

.standard-rules:
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

koyeb-setup-and-deploy-job:
  stage: build-and-deploy
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
    - curl -fsSL https://raw.githubusercontent.com/koyeb/koyeb-cli/master/install.sh | sh
    - export PATH=$HOME/.koyeb/bin:$PATH
    - export KOYEB_TOKEN=$KOYEB_API_TOKEN
  script:
    - koyeb app create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb
    - koyeb service create glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb --app glowberry-tax-structure-simulator-glabcicd-docker-compose-koyeb --git github.com/EphraimX/glowberry-global-tax-structure-simulator-gha-docker-compose-koyeb --instance-type free --git-builder docker --git-docker-dockerfile Dockerfile.koyeb --port 3000:http --route /:3000 --privileged

stages: Defines the pipeline stages — here build and deploy. In this case, deployment happens in the build stage since it's a simple flow.
.standard-rules: This reusable rule block ensures the job only runs on the default branch (usually main or master). It prevents unnecessary pipeline runs on other branches.
koyeb-setup-and-deploy-job: The core job that builds and deploys the app to Koyeb.
- stage: build — tells GitLab this job belongs to the build stage.
- image: alpine:latest — uses a minimal Alpine Linux image as the runner environment.
- before_script: Runs before the main script:
- Installs curl to download the Koyeb CLI.
- Downloads and installs the Koyeb CLI tool from the official repo.
- Updates the system PATH to include the Koyeb CLI executable.
- Sets the KOYEB_TOKEN environment variable from the secure GitLab variable KOYEB_API_TOKEN.
- script: Runs the actual deployment commands:
- koyeb app create <app-name> Creates a new app on Koyeb with the specified name.
- koyeb service create <service-name> ... Creates a service under the app with the following parameters:
  - --app <app-name>: Associates the service with the app created above.
  - --git <repo-url>: Points to the GitHub repository where your code lives.
  - --instance-type free: Specifies the free tier instance for cost-effectiveness.
  - --git-builder docker: Instructs Koyeb to use Docker as the build method.
  - --git-docker-dockerfile Dockerfile.koyeb: Uses the provided Dockerfile for the build.
  - --port 3000:http: Exposes port 3000 as HTTP.
  - --route /:3000: Sets the routing path.
  - --privileged: Grants elevated permissions if required by your app.

Deploying to Koyeb

To deploy your full stack application to Koyeb using GitLab CI/CD, all you need to do is push your code changes to the main branch of your GitLab repository. The configured pipeline in .gitlab-ci.yml will handle the build and deployment automatically.

Here’s how you can do it from your local machine:

# Stage all your changes
git add .

# Commit your changes with a descriptive message
git commit -m "Deploy full stack app to Koyeb via GitLab CI/CD"

# Push changes to the main branch
git push origin main

Once the push is complete:

GitLab will detect the commit and trigger the pipeline defined in .gitlab-ci.yml.
Navigate to your project in GitLab, then go to Build > Pipelines to monitor the deployment progress.
Click on the active pipeline to view detailed logs for each job step, helping you troubleshoot if any errors occur.

The pipeline executes the following automatically:

Installs the Koyeb CLI.
Authenticates using your KOYEB_API_TOKEN stored securely in GitLab CI/CD variables.
Creates or updates your Koyeb app and service by referencing your repository and Docker Compose configuration.
Deploys your app, making it accessible via the specified routes.

If you want to redeploy your application without new code changes, simply trigger the pipeline manually from the Pipelines page in GitLab.

Conclusion

In this guide, you learned how to deploy a full stack application to Koyeb using GitLab CI/CD with Docker Compose. By securely managing your Koyeb API token in GitLab’s CI/CD variables and configuring the .gitlab-ci.yml pipeline, you can automate your app’s build and deployment seamlessly.

This setup not only streamlines your deployment process but also ensures consistency and repeatability with every push to your repository. With this foundation, you can confidently iterate on your application while letting GitLab handle the deployment details.

Feel free to revisit the previous article for a deeper understanding of the Docker Compose setup and application structure. With this knowledge, you’re well equipped to adapt this workflow to other CI/CD platforms or customize it further for your project needs.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

How to Deploy a Full Stack Application to Koyeb Using Pulumi, Docker Compose and GitHub Actions

EphraimX — Fri, 06 Jun 2025 12:58:46 +0000

In modern DevOps workflows, Infrastructure as Code (IaC) tools like Pulumi offer powerful, flexible, and developer-friendly ways to define and manage infrastructure using general-purpose programming languages. When paired with CI/CD tools like GitHub Actions, you can fully automate the provisioning and deployment of cloud infrastructure — from code to production — using code you already understand.

In this guide, you'll learn how to deploy a full stack Docker Compose application to Koyeb using Pulumi for infrastructure configuration and GitHub Actions for continuous deployment.

We’ll assume you’ve already containerized your app and understand basic Docker Compose and CI/CD concepts. If you’re new to those, check out our earlier guide: Deploying a Full Stack App to Koyeb with Docker Compose and GitHub Actions

Prerequisites
Setting Up Your Pulumi Project
Configuring Pulumi to Deploy on Koyeb
Storing Your Koyeb API Token in GitHub Secrets
Creating the GitHub Actions Workflow
Triggering Deployment: Push and Monitor
Conclusion

Prerequisites

Before proceeding, ensure you have the following ready:

Read the Docker Compose Deployment Article: This article builds on deploying the Glowberry full-stack app with Docker Compose. Please review that article if you haven’t already.
GitHub Repository: Your Glowberry project should be pushed to a GitHub repository with the Docker Compose setup completed.
Pulumi CLI Installed: Install Pulumi on your local machine. See Pulumi Installation for detailed steps.
GitHub Account: Access to your repository with permissions to create GitHub Actions workflows and add secrets.
Koyeb Account and API Token: Create a Koyeb account if you don’t have one, and generate an API token. You’ll add this token as a secret in GitHub.
Basic familiarity with Infrastructure as Code (IaC): Prior experience with Pulumi or similar IaC tools will be helpful but not required.

Setting Up Pulumi Infrastructure for Glowberry on Koyeb

To begin managing your Glowberry deployment with Pulumi, create a dedicated directory for your Pulumi infrastructure files:

mkdir pulumi-koyeb
cd pulumi-koyeb

Inside this folder, you’ll create the following essential files:

1. `Pulumi.yaml`

This file configures the Pulumi project, specifying its name, runtime, and Python virtual environment settings:

name: pulumi-koyeb
description: Infrastructure for Glowberry Tax Simulator
runtime:
  name: python
  options:
    toolchain: pip
    virtualenv: venv
config:
  pulumi:tags:
    value:
      pulumi:template: python

2. `main.py`

This Python script defines the infrastructure as code, using the Pulumi Koyeb provider to create the app and service resources:

"""Pulumi Infrastructure for Glowberry Tax Simulator"""

import pulumi
import pulumi_koyeb as koyeb


glowberry_application = koyeb.App("gtaxsim-gha-dkr-plkb", name="gtaxsim-gha-dkr-plkb")

glowberry_application_service = koyeb.Service("gtaxsim-gha-dkr-plkb-service",

    app_name=glowberry_application.name,
    definition={
        "name": "gtaxsim-gha-dkr-plkb",
        "instance_types": [{
            "type": "nano",
        }],
        "ports": [{
            "port": 3000,
            "protocol": "http",
        }],
        "scalings": [{
            "min": 1,
            "max": 1,
        }],
        "envs": [
            {
                "key": "PORT",
                "value": "3000",
            },
        ],
        "routes": [{
            "path": "/",
            "port": 3000,
        }],
        "regions": ["fra"],
        "git": {
            "branch": "main",
            "repository": "github.com/EphraimX/glowberry-global-tax-structure-simulator-gha-docker-compose-pulumi-koyeb",
            "dockerfile": {
                "dockerfile": "Dockerfile.koyeb",
                "privileged": True,
            },
        },
    },
  opts = pulumi.ResourceOptions(depends_on=[glowberry_application])
)

3. `requirements.txt`

This file lists all the Python dependencies required for Pulumi and the Koyeb provider:

Arpeggio==2.0.2
attrs==25.3.0
debugpy==1.8.14
dill==0.4.0
grpcio==1.66.2
parver==0.5
protobuf==4.25.7
pulumi==3.167.0
pulumi_koyeb==0.1.11
PyYAML==6.0.2
semver==3.0.4
setuptools==80.1.0
wheel==0.45.1

4. `.gitignore`

To keep your repo clean, add the following file to exclude compiled Python files and the virtual environment folder:

*.pyc
venv/

This setup forms the foundation of your Pulumi infrastructure project to deploy the Glowberry application on Koyeb. Next, you’ll learn how to initialize and configure Pulumi locally before integrating it with GitHub Actions for automated deployment.

Initializing and Running Pulumi Locally

Before integrating Pulumi with GitHub Actions, you should first test and deploy your infrastructure locally to ensure everything works as expected.

1. Set Up a Python Virtual Environment

In your pulumi-koyeb directory, create and activate a virtual environment to isolate your Python dependencies:

python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

2. Install Dependencies

With the virtual environment activated, install the required Python packages from requirements.txt:

pip install -r requirements.txt

3. Install the Pulumi CLI

Make sure you have the Pulumi CLI installed on your system. You can download it from the official site: https://www.pulumi.com/docs/get-started/install/

Confirm installation by running:

pulumi version

4. Login to Pulumi

Pulumi requires you to be logged in to manage your stacks. For local testing, use the local backend:

pulumi login --local

5. Configure Koyeb API Token

Pulumi needs your Koyeb API token to authenticate and deploy resources. Set it as a Pulumi config secret:

pulumi config set koyeb:token <YOUR_KOYEB_API_TOKEN> --secret

Replace <YOUR_KOYEB_API_TOKEN> with your actual Koyeb API token.

6. Preview the Infrastructure

Run the following command to see what Pulumi will create or update without making changes:

pulumi preview

This gives you a detailed plan of the infrastructure changes.

7. Deploy the Infrastructure

If the preview looks good, apply the changes to deploy your Glowberry app on Koyeb:

pulumi up --yes

Pulumi will provision the app and service defined in your __main__.py.

With these steps, you can confidently manage and test your infrastructure locally before automating deployments. Next, we’ll integrate this setup with GitHub Actions for continuous deployment.

Setting Up GitHub Actions for Pulumi Deployment

Before creating the GitHub Actions workflow, you need to securely store the required credentials in your GitHub repository settings.

Add Required Secrets to GitHub

Get Your Koyeb API Token: Log in to your Koyeb account and navigate to your API tokens page to create or copy your token.
Get Your Pulumi Access Token: Sign in to Pulumi (https://app.pulumi.com/), go to Settings > Access Tokens, and create a new access token.
Add Secrets in GitHub: In your GitHub repository, go to Settings > Secrets and variables > Actions > New repository secret, then add:
- KOYEB_TOKEN — your Koyeb API token
- PULUMI_ACCESS_TOKEN — your Pulumi access token

Create the Workflow File

In your repository, create a folder called .github/workflows (if it doesn’t exist), and inside that folder create a file named glowberry-github-actions-docker-compose-pulumi-koyeb.yml. Paste the following workflow code into that file:

name: glowberry-github-actions-docker-compose-pulumi-koyeb

on:
  push:
    branches:
      - main

jobs:
  pulumi-deploy:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout Repository Code
        uses: actions/checkout@v4

      - name: Koyeb Setup and Deploy
        working-directory: ./pulumi-koyeb
        run: |
          sudo apt-get update
          sudo apt-get install -y curl
          export KOYEB_TOKEN=${{secrets.KOYEB_TOKEN}}
          export PULUMI_ACCESS_TOKEN=${{secrets.PULUMI_ACCESS_TOKEN}}
          curl -fsSL https://get.pulumi.com | sh
          mkdir -p ~/.pulumi/plugins/resource-koyeb-v0.1.11
          curl -L https://github.com/koyeb/pulumi-koyeb/releases/download/v0.1.11/pulumi-resource-koyeb-v0.1.11-linux-amd64.tar.gz | tar -xz -C ~/.pulumi/plugins/resource-koyeb-v0.1.11
          export PATH=$HOME/.pulumi/bin:$PATH
          pulumi login
          pulumi stack select glowberry-dev-gha-stack || pulumi stack init glowberry-dev-gha-stack
          pulumi preview
          pulumi up -y

Workflow Explanation

Trigger: This workflow runs on every push to the main branch.
Checkout Step: Pulls your repository code to the runner.
Setup and Deploy Step:
- Updates the system and installs curl.
- Exports your Koyeb and Pulumi tokens from GitHub Secrets as environment variables.
- Installs the Pulumi CLI.
- Downloads and installs the Koyeb Pulumi resource plugin (version 0.1.11).
- Adds Pulumi to the PATH.
- Logs into Pulumi using the CLI.
- Selects or creates a Pulumi stack named glowberry-dev-gha-stack.
- Runs pulumi preview to show what will be deployed.
- Runs pulumi up -y to deploy the infrastructure automatically. This CI/CD pipeline automates deploying your Glowberry full-stack app on Koyeb using Pulumi, triggered by code pushes.

Pushing Code and Monitoring Your Pulumi Deployment

Pushing Code to Trigger Deployment

After creating the Pulumi setup and the GitHub Actions workflow file, committing and pushing your changes to the main branch will automatically trigger the deployment pipeline.

To push your changes, run:

git add .
git commit -m "Add Pulumi deployment workflow for Koyeb"
git push origin main

This push activates the GitHub Actions workflow glowberry-github-actions-docker-compose-pulumi-koyeb.yml, initiating the deployment of your infrastructure via Pulumi.

Monitoring Deployment Progress

Open Your GitHub Repository: Navigate to your repository on GitHub.
Access the Actions Tab: Click on the Actions tab in the repository menu to view recent workflow runs.
Locate the Pulumi Deployment Workflow: Find the latest run titled glowberry-github-actions-docker-compose-pulumi-koyeb triggered by your push.
Inspect the Workflow Run: Click on the workflow run to see the detailed job and step logs.
Review the Logs for Deployment Status:
- Confirm the checkout step succeeded.
- Verify installation of Pulumi CLI and Koyeb Pulumi plugin without errors.
- Check the output of pulumi preview to understand the infrastructure changes planned.
- Monitor the pulumi up step to ensure successful deployment or updates to your application on Koyeb.

If any step fails, the logs provide detailed error messages to help you troubleshoot.

Conclusion

In this guide, you’ve learned how to deploy a full-stack application on Koyeb using Pulumi for infrastructure as code and GitHub Actions to automate your CI/CD pipeline. This approach offers powerful infrastructure management with Pulumi’s flexible Python SDK, combined with the automation and integration capabilities of GitHub Actions.

By setting up Pulumi locally and configuring the GitHub workflow to run your deployment, you gain a streamlined, repeatable process to provision and update your cloud resources efficiently. You can easily extend this foundation to support more complex infrastructure needs or integrate with other tools in your development workflow.

Found this guide helpful? Follow EphraimX for more hands-on DevOps walkthroughs. You can also connect with me on LinkedIn or explore more of my work on my portfolio.

DEV Community: EphraimX

How to Track and Manage Your Cloud Infrastructure Costs With Infracost

What is Infracost

Why Use Infracost

Getting Started With Infracost

Installing Infracost

Creating an Account on Infracost

How Infracost Works in Pull Requests

How Infracost Works with Terraform

Conclusion

Dev, Test, Stage, Prod: How Applications Are Deployed in the Real World

Dev, Test, Stage, Prod: What Do They Mean?

Dev, Test, Stage, Prod: Who Uses What and Why?

Dev, Test, Stage, Prod: Features And Differences

Development Environment

Testing Environment

Pre-prod or Staging Environment

Production Environment

Conclusion

End-to-End DevOps: Deploying and Monitoring a Full-Stack App with Docker, Terraform, and AWS

Prerequisites and Setup

Understanding the Application

Application Containerization

Monitoring Setup With Prometheus and Grafana

Understanding the Architecture

Architectural Diagram

Implementing the Architecture with Terraform

Verifying the Deployment

Conclusion

How to Deploy a Full Stack App to Koyeb Using Docker Compose and CircleCI

Table of Contents

Prerequisites

Obtaining Your Koyeb API Token

Adding the Koyeb API Token to CircleCI

Setting Up CircleCI Configuration

Explanation of the CircleCI Configuration

Pushing to GitHub and Monitoring Your Deployment

Steps to Trigger Deployment

Monitoring the Deployment on CircleCI

Conclusion

How to Deploy a Frontend App to Vercel Using Jenkins

Table of Contents

Prerequisites

Setting Up Jenkins for CI/CD

Option 1: Installing Jenkins via Docker (Recommended)

Option 2: Installing Jenkins via Script (Linux)

First Time Setup

Creating a Jenkins Pipeline with Git Polling

Creating the Jenkinsfile for Vercel Frontend Deployment

Explaining the Jenkinsfile

Setting Up the Vercel Token in Jenkins

Step 1: Access Jenkins Credentials

Step 2: Add the Vercel Token

Deploying and Testing the Frontend Application

Step 1: Execute the Build

Step 2: Confirm Deployment

Conclusion

How to Deploy a Python Backend to Fly.io Using Jenkins

Table of Contents

Prerequisites

Setting Up Jenkins for CI/CD

Option 1: Installing Jenkins via Docker (Recommended)

Option 2: Installing Jenkins via Script (Linux)

First Time Setup

Creating a Jenkins Pipeline with Git Polling

Creating the Jenkinsfile for Fly.io Deployment

Explaining the Jenkinsfile

Setting Up the Fly.io API Token in Jenkins

Step 1: Access the Credentials Manager

Step 2: Add the Fly.io Token

Deploying and Testing the Backend Application

Step 1: Run the Pipeline

Step 2: Verify Deployment

Conclusion

How to Deploy a Full Stack Application to Koyeb Using Docker Compose, Terraform, and Jenkins

Table of Contents

Prerequisites

Obtaining Your Koyeb API Token and Configuring Jenkins Credentials

How to Get Your Koyeb API Token

Adding the API Token to Jenkins Credentials

Creating the `Jenkinsfile` for Your Pipeline

1. Create the `Jenkinsfile`

2. Paste the Following Code into the `Jenkinsfile`

Understanding the `.gitlab-ci.yml` File

1. `Pulumi.yaml`

2. `main.py`

3. `requirements.txt`

4. `.gitignore`