Arun SD

Posted on Aug 13

A Developer’s Journey to the Cloud 3: Building a CI/CD Pipeline

#cloud #devops #cicd #git

My Deployments Were a Ritual, Not a Process

We've come so far. Our application is neatly containerized in Docker, and our data is safe and sound in managed cloud services. I had eliminated the "works on my machine" curse and outsourced my 3 AM data-loss fears. I should have been happy.

But a new kind of dread was creeping in—a dread that arrived every time I had to ship a new feature: the deployment ritual.

The Manual Dance

It was a clunky, manual dance that I had perfected:

git push my changes.
Open a terminal and SSH into my server.
Navigate to the project folder.
Run docker-compose down, git pull, and finally docker-compose up --build.

Every. Single. Time.

It worked, but it felt wrong. It was slow. It was nerve-wracking—what if I accidentally typed the wrong command on my production server?

Most importantly, it was a bottleneck. I was the only one who could do it. I had become the deployment guy, a title I never wanted.

The Day I Locked Myself Out

The breaking point came in a coffee shop on shaky Wi-Fi. I SSH'd into my server to deploy a critical hotfix, but my connection dropped midway through.

The application stopped.

The server never got the command to bring it back up.

The site was down.

It took me ten frantic minutes to get a stable connection and fix it, but the damage was done.

I realized my manual process wasn’t just inefficient—it was fragile. It relied on me, my laptop, and a stable internet connection.

What If Deployment Just… Happened?

That night, I asked myself:

What if deploying wasn't a ritual I had to perform? What if it was just… something that happened automatically when the code was ready?

That question led me to CI/CD (Continuous Integration / Continuous Deployment)—an assembly line for code.

Tools like GitHub Actions or GitLab CI act as robots on this assembly line: you give them a recipe, and they execute it perfectly, every time.

Giving the Robot My Ritual

I built a recipe so that GitHub itself would:

Build and push my Docker image.
Tell my server to pull and run the new version.

After a day of tinkering, I had my workflow:

# .github/workflows/deploy.yml
name: Deploy to Production

on:
  push:
    branches: [ main ] # Run on every push to main branch

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Code
        uses: actions/checkout@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Build and Push Docker Image
        uses: docker/build-push-action@v4
        with:
          context: .
          push: true
          tags: myusername/my-awesome-app:${{ github.sha }}

      - name: Deploy to Server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.SERVER_HOST }}
          username: ${{ secrets.SERVER_USER }}
          key: ${{ secrets.SSH_PRIVATE_KEY }}
          script: |
            cd /home/user/app
            export IMAGE_TAG=${{ github.sha }}
            docker-compose up -d --no-build

The first time the pipeline ran successfully, it felt like magic.

I pushed my code, and a few minutes later—changes were live.

I hadn’t even opened my terminal. My server was no longer a sacred place I had to log into. It was just a machine that ran containers. The “keys” now lived securely in GitHub’s secrets, not in my pocket.

From Ritual to Repeatable Code

My deployment process was no longer trapped in my head—it was now code in my repository. Version-controlled. Repeatable. Secure.

Deployments went from being a 10-minute, high-anxiety event to a complete non-event. They just… happened.

A New Problem Emerges

For the first time, I felt real peace. I could focus entirely on the application itself.

But then I noticed:

A slight lag when the dashboard loaded.
A user emailing to say their profile page "felt sticky".

Nothing was crashing. Nothing was broken. But the system was straining.

The irony? My new, efficient deployments made it easier for more users to sign up—creating the very load that was slowing things down.

Diving into the logs, I saw the truth:

My database was getting hammered with the same queries over and over.

The app wasn’t broken. It was tired.

It was time to give it a break.

Stay tuned for the next post: A Developer’s Journey to the Cloud 4: Caching with Redis

DEV Community