DEV Community: Tanay Jain

From Manual Deployments to Production-Style CI/CD: Building an Automated Flask Pipeline with GitHub Actions, Amazon ECR, and AWS EC2

Tanay Jain — Wed, 15 Jul 2026 05:56:03 +0000

Most developers eventually reach the point where deploying an application manually starts to feel repetitive. Mine looked like this:

SSH into EC2 → pull the latest code → build or pull the Docker image → restart the containers → open the browser → hope everything still works.

It wasn't difficult. It also wasn't reliable. Every deployment depended on me remembering the correct sequence of commands, in the correct order, with no automatic verification, no quality gate, and no guarantee that what worked on my machine would work in production.

The application wasn't the problem. The deployment process was.

That realization changed how I approached this project. Instead of learning GitHub Actions, Docker, and AWS as separate topics, I treated them as parts of one system, with one goal:

A single git push should be enough to take verified code from my laptop to a running application on AWS.

Today, every push to main automatically lints the code with Flake8, runs the test suite with Pytest, builds a Docker image with Buildx (reusing cached layers and dependencies), pushes the image to Amazon ECR, connects to an EC2 instance over SSH, deploys the new container, verifies it with a health check, and posts the result to Slack. No manual steps, no guessing whether it actually started.

Why Build a CI/CD Pipeline at All?

Most beginner AWS tutorials stop once the application is running: launch an EC2 instance, install Docker, run the container, visit the public IP. That's technically a deployment, but it leaves the real questions unanswered — how does new code reach the server, what stops broken code from being deployed, how does the server know which image to run, and how do you actually know a deployment succeeded?

Those questions are what CI/CD is for. The pipeline is the process that answers all of them, every time, without relying on memory.

The Project

The application itself is intentionally simple: a Flask app connected to PostgreSQL, packaged with Docker. Keeping it simple let me focus on the deployment pipeline instead of business logic — the interesting part was never the Flask code, it was everything built around it:

Python (Flask) + PostgreSQL, Docker & Docker Compose
GitHub Actions, with matrix builds, reusable workflows, and composite actions
Amazon ECR for image storage, AWS EC2 for compute
Pytest for testing, Flake8 for code quality
Docker Buildx with layer and dependency caching
Post-deploy health checks and Slack notifications

tanayjdev / flask-docker-app

Production-style Flask application with an end-to-end CI/CD pipeline using GitHub Actions, Docker, Amazon ECR and AWS infrastructure (EC2, ALB, RDS, VPC, CloudWatch).

Flask + PostgreSQL - Production-style CI/CD Pipeline on AWS

Key Highlights

Production-style Flask application
Dockerized deployment
GitHub Actions CI/CD Pipeline
Amazon Elastic Container Registry (ECR)
Automated deployment to Amazon EC2
AWS Infrastructure (EC2, ALB, RDS, VPC)
Matrix builds for multiple Python versions
Reusable GitHub Actions workflows
Composite GitHub Actions
Docker layer & pip dependency caching
Slack deployment notifications
Project evolution from local Docker → AWS Infrastructure → Full CI/CD

Project Goal

Build, containerize, and deploy a production-style Flask web application on AWS while implementing an end-to-end CI/CD pipeline using GitHub Actions and modern DevOps practices.

The project demonstrates modern DevOps practices including automated code quality checks, testing, container image builds, continuous deployment, AWS infrastructure, monitoring, deployment automation, and infrastructure best practices.

Current Architecture (v3.0)

What This Project Is

A production-style Flask web application deployed on AWS using Docker and automated through a complete CI/CD pipeline.

The application tracks page visits…

View on GitHub

The Pipeline, End to End

Push
  │
  ▼
Lint (Flake8)
  │
  ▼
Unit Tests (Pytest)
  │
  ▼
Docker Build (Buildx + cache)
  │
  ▼
Push Image to Amazon ECR
  │
  ▼
Deploy to EC2 over SSH
  │
  ▼
Health Check (/health)
  │
  ▼
Slack Notification

Each stage has exactly one responsibility, and each stage protects the next: if a stage fails, the pipeline stops immediately and broken code never reaches production. That single rule shaped every other decision in this project.

Stage 1 — Code Quality First

Linting runs before anything else, on purpose:

lint:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
        python-version: "3.10"
        cache: pip
    - run: pip install flake8
    - run: flake8 . --count --statistics

Not because style matters more than correctness — because it's the fastest gate. Spinning up a full Docker build only to fail on a stray import is wasted time and wasted compute. Fail fast, fix early, then move on.

Stage 2 — Tests Gate the Pipeline

Passing lint doesn't mean the app works, so the next job runs the test suite — and only starts if lint succeeded:

test:
  needs: lint

That one needs keyword turns a set of independent jobs into an explicit dependency graph. Inside the job:

pip install -r requirements.txt
pytest test_app.py -v

If a test fails, nothing downstream runs. No image is built, nothing is deployed, and production never sees the broken code. Separating verification from deployment this way is one of the most useful ideas in the whole pipeline.

Making It Fast: Caching

Early on, every run re-downloaded every dependency from scratch, even when nothing had changed. setup-python's built-in cache: pip handles most of it, and a dedicated actions/cache step — keyed off a hash of requirements.txt — restores previously downloaded packages instantly when the file hasn't changed. Less waiting, faster feedback after every commit.

Docker layers get the same treatment. Docker only rebuilds what changed, so the Dockerfile's instruction order matters:

COPY requirements.txt .
RUN pip install -r requirements.txt
COPY app.py .

Dependencies rarely change; application code changes constantly. Installing dependencies before copying the source means Docker can reuse that dependency layer across nearly every build, and only rebuild the thin application layer on top. It's a small reordering that noticeably speeds up almost every run — a reminder that performance work isn't always about adding new tools, sometimes it's about arranging the ones you already have.

Building and Storing the Image

The image itself is built with Docker Buildx rather than the legacy builder, since it has better support for modern caching:

- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v5
  with:
    context: .
    push: true
    cache-from: type=gha
    cache-to: type=gha,mode=max

Once built, the image needs somewhere to live before deployment. I used Amazon ECR: GitHub Actions authenticates with AWS credentials stored as repository secrets, then pushes the image automatically, tagged both latest and with the commit SHA so every deployment is traceable back to an exact commit.

Splitting image creation from image storage from image deployment keeps each piece single-purpose — GitHub Actions builds the artifact, ECR stores it, EC2 just pulls whatever's latest.

Deploying to EC2

The deploy job runs only after the image has been built and pushed successfully, and connects to EC2 over SSH using a key stored in GitHub Secrets. Because ECR is a private registry, the server has to authenticate before it can pull anything — this part is easy to gloss over, but skip it and every pull fails silently:

aws ecr get-login-password --region ap-south-1 \
  | docker login --username AWS --password-stdin <ecr-registry-url>

docker compose pull
docker compose up -d --force-recreate

No manual login, no copying files, no rebuilding images on the server. EC2 authenticates, pulls whatever ECR says is latest, and recreates the containers — which means every environment always runs the exact same image, byte for byte.

A Deployment Isn't Successful Until It's Healthy

Containers starting doesn't mean the application is working. After the restart, the workflow waits briefly and then checks the app's own health endpoint:

curl -s -o /dev/null -w "%{http_code}" http://localhost:5000/health

HTTP 200 means the deployment succeeded; anything else fails the workflow immediately, logs are dumped, and the run is marked as failed. Without this, "the container started" and "the application works" get treated as the same thing — and they aren't. With it, success actually means what it says: the app is responding, not just running.

Beyond a Single Workflow: Matrix Builds, Reusable Workflows, and Composite Actions

Once the core pipeline was solid, three GitHub Actions features made it easier to maintain and extend:

Matrix builds run the test job across multiple Python versions in parallel (3.9, 3.10, 3.11), so a change that only breaks one version gets caught before it reaches main, instead of after.
Reusable workflows let the testing logic live in one workflow_call-triggered file that other workflows invoke by reference, instead of copy-pasting the same lint-and-test steps into every pipeline.
Composite actions package a repeated sequence — checkout, Python setup, dependency install — into a single custom action (setup-python-env), so each job calls it in one line instead of repeating four steps every time.

None of these change what the pipeline does. They change how much duplication exists in how it's defined — which matters the moment there's more than one workflow file to maintain.

Mistakes I Made Along the Way

The project wasn't smooth. A few of the failures taught me more than any of the successful runs did.

The biggest one: I initially configured the Docker health check to hit an HTTPS endpoint, even though the Flask app only served plain HTTP. The container reported unhealthy indefinitely — not because the app was broken, but because the health check itself was wrong. It's an easy trap: a red status doesn't always mean the thing you think it means, and a deployment is only as trustworthy as the check verifying it.

I also spent real time debugging GitHub Actions IAM permissions, ECR authentication, and SSH connectivity — each one forced me to actually understand how the pieces fit together instead of treating the pipeline as a black box I copy-pasted into existence.

What I'd Improve Next

The pipeline is fully automated, but there's a clear list of what production systems usually add next:

Rolling or blue-green deployments to eliminate the brief downtime during --force-recreate
Automatic rollback if a deployment's health check fails
Infrastructure as Code with Terraform, instead of hand-provisioned AWS resources
Kubernetes for orchestration once a single EC2 instance stops being enough
Observability with Prometheus and Grafana, beyond a single /health check

Key Takeaways

Cheap checks should run first — lint before test, test before build, build before deploy.
A needs dependency graph is what turns a pile of jobs into an actual pipeline.
A private registry means the deploy target has to authenticate — don't skip that step in your own writeup or your own script.
A health check only means something if it's checking the right thing.
Reusable workflows and composite actions aren't about new capability — they're about not repeating yourself once you have more than one pipeline.
CI/CD isn't one tool. It's a sequence of quality gates that all have to agree before production changes.

Conclusion

This project started as a way to learn GitHub Actions. It ended up teaching me far more about how a deployment pipeline actually fits together — from code quality and testing, to container builds and image registries, to cloud deployment and the health check that proves it worked.

There's still plenty left to learn — Kubernetes, Infrastructure as Code, rollback strategies — but building this gave me a real, working mental model of how modern software delivery happens, instead of a list of tool names I could recognize but not explain.

If you're learning DevOps or cloud engineering, build a pipeline yourself instead of only reading about one. Debugging the failures taught me more than watching the successful runs ever did.

Check out the full source and workflow files on GitHub

I'm curious — what was the moment that convinced you to stop deploying manually? Or if you're still doing it by hand, what's the biggest thing stopping you from automating it?

How I Deployed a Flask App on AWS with RDS, ALB, and VPC as a Student

Tanay Jain — Mon, 15 Jun 2026 08:37:17 +0000

Most Flask deployment tutorials end after launching an EC2 instance.

Mine did too.

The application worked, but the architecture had serious problems:

Database exposed on the same machine as the application
No load balancing
No monitoring
No network isolation
Application accessible directly through EC2 public IP

So I rebuilt the entire stack using AWS VPC, RDS, ALB, Security Groups, IAM, and CloudWatch — to understand how production cloud systems are actually designed.

This article documents that journey.

Why I Built This
Initial Architecture (v1.0)
Architecture Evolution
Target Architecture (v2.0)
Tech Stack
AWS Services Used
VPC Design
Security Model
RDS Migration
Load Balancer
Monitoring
Challenges
Security Improvements
Limitations
What This Project Taught Me
Project Outcomes
Roadmap
Source Code

Why I Built This

I wanted to move beyond tutorial deployments and understand:

How databases are isolated in production
Why load balancers exist and what they actually do
How AWS networking works at a real level
How monitoring is implemented across multiple services
How security is enforced between application layers

Instead of deploying another demo app, I focused on building a production-style architecture around a simple application — a Flask visit counter backed by PostgreSQL.

Initial Architecture (v1.0)

Flask App
    │
PostgreSQL Container
    │
Docker Compose
    │
Single EC2 Instance

Everything ran together on one machine. It worked for learning but had real limitations:

PostgreSQL inside a container — no managed backups, no isolation
No health checks — broken app still received traffic
No monitoring — no visibility into what was happening
Direct EC2 access — no traffic management layer

Architecture Evolution

Before (v1.0)	After (v2.0)
Flask + PostgreSQL on one machine	Flask on EC2 + RDS PostgreSQL
Direct EC2 public IP access	ALB as single entry point
No monitoring	CloudWatch Dashboard
No network isolation	Public + Private Subnets
PostgreSQL in a container	Managed Amazon RDS
Single point of failure	Production-style 3-tier architecture

Target Architecture (v2.0)

Figure 1: Final 3-tier AWS architecture used in this project.

Internet
    │
    ▼
Application Load Balancer
    │
    ▼
EC2 Instance (Docker + Flask)
    │
    ▼
Amazon RDS PostgreSQL (Private Subnet)

This follows a standard 3-tier architecture pattern:

Tier	Components
Presentation	Application Load Balancer
Application	EC2 + Docker + Flask
Data	Amazon RDS PostgreSQL

Tech Stack

Application

Python, Flask
PostgreSQL

Containerization

Docker
Docker Compose

AWS Services

VPC, EC2, RDS PostgreSQL
Application Load Balancer
IAM, CloudWatch

Networking and Security

Security Groups
Public and Private Subnets
Internet Gateway, Route Tables

AWS Services Used

Service	Purpose
Amazon VPC	Network isolation and segmentation
Amazon EC2	Application hosting
Docker	Application containerization
Amazon RDS PostgreSQL	Managed database
Application Load Balancer	Traffic distribution and health checks
IAM	Access control and least-privilege
Amazon CloudWatch	Monitoring and metrics
Security Groups	Traffic filtering between layers
Internet Gateway	Internet access for public subnets
Route Tables	Subnet traffic routing

Building a Custom VPC

The first step was replacing the default VPC with a custom one designed around isolation.

VPC: 10.0.0.0/16
│
├── Public Subnets
│   ├── 10.0.1.0/24   (AZ: ap-south-1b)
│   └── 10.0.4.0/24   (AZ: ap-south-1a)
│
└── Private Subnets
    ├── 10.0.2.0/24   (AZ: ap-south-1b)
    └── 10.0.3.0/24   (AZ: ap-south-1a)

Public subnets host the EC2 instance and ALB — internet-facing resources.

Private subnets host RDS — no route to the internet gateway, no direct access from anywhere outside the VPC.

The database is unreachable from the internet not just because of a firewall rule, but because it has no path there at all.

Security Group Layering

Each layer only accepts traffic from the layer directly above it:

Internet
   │
   ▼  HTTP 80, HTTPS 443
ALB Security Group
   │
   ▼  HTTP 80 — from ALB Security Group only
EC2 Security Group
   │
   ▼  PostgreSQL 5432 — from EC2 Security Group only
RDS Security Group

Nobody can reach EC2 directly from the internet.
Nobody can reach RDS — not from the internet, not even from a developer's machine.
Every request must pass through the ALB first.

IAM and Access Control

Before deploying anything, I set up proper access control:

IAM Admin User with MFA — root account never used for daily work
IAM Roles for AWS service-to-service access
Least-privilege policy — no account has more permissions than it needs

Root account credentials are the most dangerous thing in an AWS account. Building this habit early matters.

Migrating Database to Amazon RDS

Moving PostgreSQL from a Docker container to Amazon RDS was one of the biggest architectural improvements.

Containerized PostgreSQL	Amazon RDS PostgreSQL
Manual backups	Automated backups
Manual patching	Automatic patching
Container restart = potential data loss	Managed persistent storage
No monitoring built in	CloudWatch integration

The Flask application connects to RDS using SSL:

psycopg2.connect(
    host=DB_HOST,
    sslmode="require",
    connect_timeout=5
)

RDS is accessible only from the EC2 Security Group. Nothing else can reach it.

Application Load Balancer

The ALB became the single entry point for all traffic.

Health Check Configuration:

Path:               /health
Interval:           30 seconds
Healthy Threshold:  2
Success Code:       200

The /health endpoint verifies actual database connectivity before reporting healthy:

{
  "status": "healthy",
  "database": "connected"
}

If RDS goes down, the health check fails, and the ALB stops routing traffic to that instance.

Figure 2: ALB Target Group showing healthy EC2 target.

CloudWatch Monitoring

I configured a unified CloudWatch dashboard covering all three tiers.

Figure 3: CloudWatch dashboard monitoring EC2, RDS, and ALB metrics.

Load Balancer

Request Count
Target Response Time
Healthy Host Count

EC2

CPU Utilization
Network In / Out

RDS

CPU Utilization
Database Connections
Free Storage Space

Having visibility across all layers meant that when something broke during testing, I could immediately see which tier was affected.

Challenges I Encountered

RDS Connectivity Issues

When I first deployed, the Flask application could not connect to PostgreSQL at all.

The issue was Security Group configuration. The RDS Security Group was only allowing inbound traffic from a specific IP — which changed every time the EC2 instance restarted.

Fixing this by referencing the EC2 Security Group ID instead of an IP address made me understand AWS network security in a way no tutorial had managed before.

Health Check Design

My first health check endpoint simply returned 200 OK with no logic.

The problem was that the application could lose database connectivity and still appear healthy to the ALB. Traffic kept routing to a broken instance.

I redesigned the endpoint to verify actual database access before returning healthy status. If the database query fails, the endpoint returns 503. The ALB catches this and stops routing traffic until connectivity is restored.

Security Improvements

The migration introduced multiple security improvements over the original setup:

Database moved into private subnets — no public IP, no internet route
Security Group based access control between every layer
IAM Admin User with MFA — root account locked away
Least-privilege access model throughout
ALB as the only public entry point — EC2 not directly reachable
SSL connection between Flask and RDS PostgreSQL

These controls significantly reduced the attack surface compared to the original single-machine deployment.

Approximate Cost

This project was built within AWS Free Tier limits where possible:

Service	Cost
EC2 t2.micro	Free Tier (750 hrs/month)
RDS db.t3.micro	Free Tier (750 hrs/month)
Application Load Balancer	Minimal cost during testing
CloudWatch	Free Tier metrics

The goal was to experience production-style architecture without significant cost — and to understand which services carry real costs at scale.

Application Endpoints

Home

GET /
Response: Page visited X times!

Health Check

GET /health
Response: { "status": "healthy", "database": "connected" }

Current Limitations

While this architecture follows production patterns, it still has limitations:

Single EC2 instance — no redundancy at the application layer
No Auto Scaling Group — cannot handle traffic spikes
No CI/CD pipeline — deployments are manual
No Infrastructure as Code — resources provisioned via console
No HTTPS — SSL termination via ACM not yet configured

These are planned improvements for the next iteration.

What This Project Taught Me

Before this project, I knew how to deploy applications.

After this project:

I understood why VPC design matters before writing a single line of application code
I understood why databases belong in private networks, not just from a tutorial but from seeing what happens when they are not
I understood how health checks directly affect availability — a bad health check is worse than no health check
I understood how AWS services interact in a real architecture — not in isolation

The shift from deployment to architecture thinking was the biggest takeaway.

Project Outcomes

By the end of this project:

Designed a custom AWS VPC with public and private subnets across two Availability Zones
Deployed a Dockerized Flask application on Amazon EC2
Migrated PostgreSQL from a container to Amazon RDS in private subnets
Configured an Application Load Balancer with real health check logic
Implemented Security Group based network isolation across all three tiers
Built CloudWatch dashboards monitoring EC2, RDS, and ALB metrics
Documented the full architecture and deployment workflow

Roadmap

The current architecture uses manually provisioned AWS resources. Next steps:

Infrastructure as Code using Terraform — provision the entire stack from code
Automated CI/CD using GitHub Actions — code push triggers build, test, and deploy
HTTPS with AWS Certificate Manager
Container orchestration with Kubernetes on Amazon EKS
Auto Scaling Groups for high availability
Centralized logging with CloudWatch Logs

Source Code

Full source code, architecture diagrams, Dockerfile, Docker Compose, and documentation:

👉 GitHub Repository

Conclusion

This project taught me that cloud architecture is not about deploying applications — it is about designing systems that are secure, observable, and resilient.

Building the VPC, isolating the database, configuring Security Groups, implementing real health checks, and monitoring the full stack with CloudWatch gave me a much deeper understanding of AWS than any tutorial had.

The application itself is simple.

The infrastructure behind it is where the real learning happened.

Thanks for reading.

Feedback, suggestions, and architecture improvements are always welcome.

How I Built 5 Linux Automation Scripts on AWS EC2

Tanay Jain — Sat, 23 May 2026 08:21:06 +0000

I wanted to find out what working on a real Linux server actually feels like — not a local VM, not a simulator.

So in May 2026, I spun up an Ubuntu 22.04 server on AWS EC2, connected via SSH, and spent the entire month doing real work on it.

Here's what I built.

🖥️ Environment

Tool	Details
Cloud	AWS EC2 t2.micro
OS	Ubuntu 22.04 LTS
Editor	VS Code Codespaces
Auth	SSH key-based authentication
Automation	Bash scripting + cron jobs

📚 Topics Covered

Linux Fundamentals

User and group management
File permissions (chmod, chown)
Process management (ps, top, kill, systemctl)
Networking basics (ss, curl, UFW, DNS)
Package management with apt

Automation & Scripting

Bash scripting — functions and validation
Log management
Cron job scheduling
SSH workflows (scp, rsync)
Log analysis using grep, awk, and sed

🔧 The 5 Automation Scripts

By the end of the month, I had built and automated 5 production-style Bash scripts.

1. Server Health Check

A monitoring script that checks:

CPU usage
RAM usage
Disk usage
Service status
Internet connectivity

Scheduled every 15 minutes using cron.

./server_health.sh

Example output:

================================================
        SERVER HEALTH CHECK REPORT
================================================

Date: 2026-05-12 10:00:00
Hostname: ip-172-xx-xx-xx

--- CPU Usage ---
✅ CPU is OK (2.3%)

--- Memory Usage ---
✅ RAM is OK (45%)

--- Services Status ---
✅ ssh: RUNNING
✅ nginx: RUNNING
✅ docker: RUNNING

--- Network ---
✅ Internet: CONNECTED

================================================

2. Disk Usage Alerter

A script that scans partitions and generates alerts when disk usage exceeds a threshold.

Features:

Threshold-based alerts
Partition monitoring
Log generation
Color-coded terminal output

Runs every hour through cron.

3. Log Cleaner

A maintenance script that:

Compresses older logs
Removes outdated logs
Reduces disk usage automatically

Built using find, gzip, and mtime filters for log retention management.

Runs every Sunday.

4. User Creation Script

A provisioning script for creating users with a consistent setup.

Features:

Username validation
Group assignment
Home directory creation
Temporary password generation
Batch user creation using CSV files

sudo ./user_creation.sh --file users.csv

5. Backup Script

Creates compressed backups using tar.gz archives.

Features:

Backup verification
Retention policy
Automatic cleanup of old backups
Logging and integrity checks

Scheduled daily at 2 AM.

⏱️ Cron Job Automation

All scripts were automated using cron jobs.

# Health check — every 15 minutes
*/15 * * * * /home/ubuntu/scripts/server_health.sh >> /home/ubuntu/logs/health_cron.log 2>&1

# Disk alerter — every hour
0 * * * * /home/ubuntu/scripts/disk_alerter.sh >> /home/ubuntu/logs/disk_cron.log 2>&1

# Backup — daily at 2 AM
0 2 * * * /home/ubuntu/scripts/backup.sh >> /home/ubuntu/logs/backup_cron.log 2>&1

# Log cleaner — every Sunday at 11 PM
0 23 * * 0 /home/ubuntu/scripts/log_cleaner.sh >> /home/ubuntu/logs/cleaner_cron.log 2>&1

Once configured, the server handled routine maintenance automatically.

💡 Biggest Learnings

1. Linux becomes comfortable through repetition

At the beginning, basic terminal commands felt unfamiliar.

After working daily on a remote server, navigating Linux from the command line became much more natural. There's no shortcut — you just have to do it daily.

2. Automation changes how you think

One of the biggest mindset shifts was noticing repetitive work and immediately thinking:

"Can this be automated?"

That shift alone made scripting feel much more practical — and honestly, more fun.

3. Real infrastructure teaches different lessons

Working on an actual EC2 instance exposed me to problems that are difficult to fully understand in local environments:

SSH authentication issues
File permission problems
Cron debugging
Disk usage management
Log analysis workflows

Solving those problems on a live server taught me far more than just reading commands from documentation.

🚀 What's Next

Next, I'm moving into AWS Core Infrastructure — VPC, IAM, RDS, and Terraform.

That work starts in June 2026. Follow along if you're on a similar path.

📁 GitHub Repository

All scripts and documentation are open source:

👉 github.com/tanayjdev/linux-bash-scripts

BCA Student • Aspiring Cloud & DevOps Engineer

My CI Pipeline Failed on the First Push. Here's What I Learned.

Tanay Jain — Sun, 03 May 2026 12:59:58 +0000

My CI pipeline failed on the first push.

Not because of a big mistake.
Because of a layer conflict in my Dockerfile that
only showed up in a clean environment.

Locally — everything worked.
CI — red on the first run.

That one failure taught me more about Docker
than a week of tutorials.

Here's exactly what I built and what happened.

The Project

A Flask + PostgreSQL app running in Docker Compose.

It tracks page visits and stores the count
in a real PostgreSQL database.
Browser → Flask Container → PostgreSQL → pgdata Volume

Multi-container setup. Persistent storage.
Environment variables via .env.
Health checks and restart policies.

Every decision is production-style —
even though it's a learning project.

Why I Added CI

Before CI, my workflow looked like this:
Make a change
↓
docker compose down
↓
docker compose build
↓
docker compose up -d
↓
Open browser, check manually
↓
Repeat

This works for one or two changes.

But when you're pushing regularly —
manually checking every time is not sustainable.

I wanted the system to tell me if something broke.
Not discover it later.

What I Built

.github/workflows/docker-build.yml

name: Docker Build CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v3

    - name: Setup Docker Buildx
      uses: docker/setup-buildx-action@v2

    - name: Build Docker image
      run: docker build -t flask-app:test .

    - name: Verify image exists
      run: docker images | grep flask-app

    - name: Run container test
      run: |
        docker run --rm flask-app:test python -c "
        import flask
        import psycopg2
        print('Flask:', flask.__version__)
        print('All imports OK')
        "

    - name: Check image size
      run: |
        SIZE=$(docker image inspect flask-app:test \
        --format='{{.Size}}')
        echo "Image size: $SIZE bytes"
        echo "Build verified"

What Each Step Does

Checkout — GitHub's server downloads my code

Docker Buildx setup — Prepares build tools on the runner

Build image — Runs my Dockerfile from scratch,
clean environment every time

Verify image — Confirms image was actually created

Container test — Runs the container,
checks Flask and psycopg2 import correctly

Size check — Tracks image size on every build

The First Run Failed

Pushed the code. Opened Actions tab. Red.

The error was in my Dockerfile.

I had this:

RUN apt-get update && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

The layer order was causing a conflict
in the clean CI environment that
didn't show up on my machine.

Reordered the layers. Pushed again. Green.

This is the whole point of CI.

Your local machine has cached layers,
existing images, leftover containers.
CI starts completely fresh every time.

It catches what you miss.

The Health Endpoint

I also added a /health route to Flask:

@app.route("/health")
def health():
    try:
        conn = get_connection()
        conn.close()
        return {"status": "healthy", "database": "connected"}, 200
    except Exception as e:
        return {"status": "unhealthy", "error": str(e)}, 500

A running container doesn't mean a working app.

This endpoint checks actual database connectivity.
200 = Flask is up and DB is reachable.
500 = Something broke inside.

Real monitoring tools and load balancers
use exactly this kind of endpoint.

Architecture

GitHub Actions CI:
Code Push → Build → Test → Validate
↓
Docker Hub (image stored)
↓
AWS EC2 (deployed)
↓
Flask Container ←→ PostgreSQL Container
↓
pgdata Volume

Project Versioning

v1.0 — Flask + PostgreSQL + Docker + EC2 deploy
v1.1 — CI pipeline + /health endpoint added

Tagging versions made me treat this
like a real product — not just practice code.

What Actually Changed

Before:
Push code → manually test → hope for the best.

After:
Push code → pipeline runs in ~2 minutes → green or red.

No guessing. No "I think it should work."
The pipeline knows.

Three Things CI Taught Me Practically

1. Clean environment matters

Your machine lies to you.
Cached layers, existing images, leftover state —
none of that exists in CI.

If it works locally but fails in CI,
the CI is right.

2. The pipeline is documentation

Anyone reading the workflow file
understands exactly how the project builds.
No README needed for that part.

3. Start CI early

Adding it to an existing project is harder
than starting with it.

Next project — pipeline goes in on day one.

Links

GitHub: https://github.com/tj2905

Docker Hub: docker pull tanayjain29/flask-devops-app:v1.0

Learning by building real things.

Turning ideas into working projects.
Sharing everything on GitHub.

From Zero to Deployment: Dockerizing a Flask + PostgreSQL App on AWS

Tanay Jain — Sun, 29 Mar 2026 18:56:16 +0000

When I started learning Docker, I had no idea I’d end up deploying a real application on a cloud server within the same month.

It ended up changing how I think about building software.

The Idea

I didn’t want to build just another static project.

I wanted:

A real backend
A real database
A real deployment

So I built a simple web app that tracks page visits and stores them in PostgreSQL.

Tech Stack

Python Flask (backend)
PostgreSQL (database)
Docker & Docker Compose
AWS EC2 (deployment)

Architecture

User (Internet)
   ↓
AWS EC2 Server
   ↓
Docker Engine
   ↓
Flask Container (Port 80)
   ↓ psycopg2
PostgreSQL Container
   ↓
Persistent Volume (pgdata)

Key Concepts I Implemented

🔹 Multi-container system

Instead of running containers manually, I used Docker Compose.

docker compose up -d

This starts the entire system in one command.

🔹 Environment variables (.env)

No secrets inside the code.

Cleaner
Safer
Production-ready

🔹 Health checks

Docker monitors the application continuously.

If Flask stops responding, the container is marked as unhealthy.

🔹 Auto-restart

Using:

restart: always

If a container crashes, Docker automatically restarts it.

🔹 Optimized Dockerfile

Used python:3.10-slim
Cleared apt cache after install
Used --no-cache-dir for pip
Optimized layer ordering

Result: Image size reduced significantly (~300MB → ~160MB)

Biggest Challenge

Connecting Flask to PostgreSQL inside Docker.

The key realization:

Containers communicate using service names, not IP addresses.

DB_HOST=db

Docker internally resolves db to the PostgreSQL container.

Once this clicked, the setup became much clearer.

Deployment

I deployed the app on AWS EC2.

Steps:

Launch EC2 instance
Install Docker
Clone project
Run Docker Compose

The application was then accessible via a public IP.

What I Learned

Docker is not just about containers.

It’s about building systems that are:

Reproducible
Isolated
Self-healing
Easy to deploy

Final Thoughts

This project was my first real experience combining backend, database, and deployment into one system.

If you're learning Docker, building and deploying even a small project like this gives much more clarity than just following tutorials.

DEV Community: Tanay Jain

From Manual Deployments to Production-Style CI/CD: Building an Automated Flask Pipeline with GitHub Actions, Amazon ECR, and AWS EC2

Why Build a CI/CD Pipeline at All?

The Project

tanayjdev / flask-docker-app

Production-style Flask application with an end-to-end CI/CD pipeline using GitHub Actions, Docker, Amazon ECR and AWS infrastructure (EC2, ALB, RDS, VPC, CloudWatch).

Flask + PostgreSQL - Production-style CI/CD Pipeline on AWS

Key Highlights

Project Goal

Current Architecture (v3.0)

What This Project Is

The Pipeline, End to End

Stage 1 — Code Quality First

Stage 2 — Tests Gate the Pipeline

Making It Fast: Caching

Building and Storing the Image

Deploying to EC2

A Deployment Isn't Successful Until It's Healthy

Beyond a Single Workflow: Matrix Builds, Reusable Workflows, and Composite Actions

Mistakes I Made Along the Way

What I'd Improve Next

Key Takeaways

Conclusion

How I Deployed a Flask App on AWS with RDS, ALB, and VPC as a Student

Table of Contents

Why I Built This

Initial Architecture (v1.0)

Architecture Evolution

Target Architecture (v2.0)

Tech Stack

AWS Services Used

Building a Custom VPC

Security Group Layering

IAM and Access Control

Migrating Database to Amazon RDS

Application Load Balancer

CloudWatch Monitoring

Challenges I Encountered

RDS Connectivity Issues

Health Check Design

Security Improvements

Approximate Cost

Application Endpoints

Current Limitations

What This Project Taught Me

Project Outcomes

Roadmap

Source Code

Conclusion

How I Built 5 Linux Automation Scripts on AWS EC2

🖥️ Environment

📚 Topics Covered

Linux Fundamentals

Automation & Scripting

🔧 The 5 Automation Scripts

1. Server Health Check

2. Disk Usage Alerter

3. Log Cleaner

4. User Creation Script

5. Backup Script

⏱️ Cron Job Automation

💡 Biggest Learnings

1. Linux becomes comfortable through repetition

2. Automation changes how you think

3. Real infrastructure teaches different lessons

🚀 What's Next

📁 GitHub Repository

My CI Pipeline Failed on the First Push. Here's What I Learned.

The Project

Why I Added CI

What I Built

What Each Step Does

The First Run Failed

The Health Endpoint

Architecture

Project Versioning

What Actually Changed

Three Things CI Taught Me Practically

Links

From Zero to Deployment: Dockerizing a Flask + PostgreSQL App on AWS