DEV Community: Bharath Aaleti

Building a Real DevOps Project from Scratch: A Beginner-Friendly Walkthrough

Bharath Aaleti — Mon, 24 Nov 2025 11:35:47 +0000

How a Simple 2-Tier Flask App Became a Full CI/CD + Docker + AWS Pipeline

When I started working on this project, I wasn’t trying to build something “big.”
I simply wanted a clean, practical way to help beginners understand DevOps, not through theory, but through a real, working cloud project.

And that’s how this 2-Tier Flask + MySQL + Docker + Jenkins + AWS project began.

What it ended up becoming is much more than a demo app, it’s a complete end-to-end DevOps learning path.

🧠 First, What Are We Actually Building?
🌩️ Understanding the DevOps Concepts (Beginner-Friendly Explanations)
🏗️ Architecture & CI/CD Pipeline
🚀 What a Beginner Learns
🔍 Real Challenges You’ll Face
🚀 Quick Start
🎨 Customization
📚 What’s Next?
🤝 Contributing
🙏 Acknowledgments

First, What Are We Actually Building?

This project is a 2-Tier Web Application:

✔️ Tier 1 → Flask Web App

Handles UI, routing, and basic logic.

✔️ Tier 2 → MySQL Database

Stores messages, data, and user inputs.

✔️ Both running as Docker containers

(So there are no “it works on my machine” issues.)

And the entire thing is deployed automatically using:

Jenkins for CI/CD
AWS EC2 as the host
GitHub for version control

Understanding the DevOps Concepts (Beginner-Friendly Explanations)

🐳 Docker (Containerization)

What it is: A way to package your app and its environment into a portable unit.

Why it matters: If it works inside a container, it will work anywhere.

🔄 CI/CD with Jenkins

CI: Auto-build on each code push

CD: Auto-deploy to server

In this project: Jenkins pulls code → builds Docker images → deploys on AWS.

☁️ AWS EC2

Your cloud server where everything runs.

📦 Git + GitHub

Version control + triggers Jenkins pipeline.

Architecture (Simplified)

Developer → GitHub → Jenkins → Docker → AWS EC2 → Live App


+-----------------+      +----------------------+      +-----------------------------+
|   Developer     |----->|     GitHub Repo      |----->|        Jenkins Server       |
| (pushes code)   |      | (Source Code Mgmt)   |      |         (on AWS EC2)        |
+-----------------+      +----------------------+      |                             |
                                                       | 1. Clones Repo              |
                                                       | 2. Builds Docker Image      |
                                                       | 3. Runs Docker Compose      |
                                                       +--------------+--------------+
                                                                      |
                                                                      | Deploys
                                                                      v
                                                       +-----------------------------+
                                                       |      Application Server     |
                                                       |        (Same AWS EC2)       |
                                                       |                             |
                                                       | +-------------------------+ |
                                                       | | Docker Container: Flask | |
                                                       | +-------------------------+ |
                                                       |              |              |
                                                       |              v              |
                                                       | +-------------------------+ |
                                                       | | Docker Container: MySQL | |
                                                       | +-------------------------+ |
                                                       +-----------------------------+

Architecture diagram generated with AI assistance

What a Beginner Learns

Containerization
Docker Compose networking
CI/CD automation
Cloud deployment
Debugging pipelines
Security groups + networking
Real project structure

Real Challenges You’ll Face

Jenkins not having Docker permissions
Database containers failing
Networking issues inside Docker
Wrong GitHub credentials
Security group misconfigurations

These experiences teach faster than tutorials.

🛠️ Full Project Stack

Python/Flask
MySQL
HTML/CSS/JS
Docker + Compose
Jenkins
AWS EC2
GitHub Webhooks

Quick Start

1️⃣ Server Setup

Launch a EC2 Instance
Configure Security Group
- Allow ports 5000 (App), 8080 (Jenkins) in Inbound-Rules.
Connect to EC2 Instance ssh -i /path/to/key.pem ubuntu@<ec2-public-ip>

2️⃣ Environment Setup

Install Dependencies on EC2
Update System Packages:
sudo apt update && sudo apt upgrade -y
Install Git, Docker, and Docker Compose:
sudo apt install git docker.io docker-compose-v2 -y
Start and Enable Docker:

   sudo systemctl enable docker
   sudo systemctl start docker

Add User to Docker Group (to run docker without sudo):

   sudo usermod -aG docker $USER
   newgrp docker

3️⃣ Jenkins Installation and Setup

Installation docs - Official jenkins link

Update the Debian apt repositories, install OpenJDK 21, and check the installation with the commands:

   sudo apt update
   sudo apt install fontconfig openjdk-21-jre
   java -version

Adding Jenkins Repository and Install:

   sudo wget -O /etc/apt/keyrings/jenkins-keyring.asc \
   https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
   echo "deb [signed-by=/etc/apt/keyrings/jenkins-keyring.asc]" \
   https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
   /etc/apt/sources.list.d/jenkins.list > /dev/null
   sudo apt update
   sudo apt install jenkins

Enable and Start Jenkins:

   sudo systemctl enable jenkins
   sudo systemctl start jenkins
   sudo systemctl status jenkins

Access Jenkins:
- Open your browser and navigate to http://<ec2-public-ip>:8080
- Follow the on-screen instructions to unlock Jenkins using the initial admin password:

   sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Install Suggested Plugins and Create Admin User.
Install Pipeline: Stage View plugin.

4️⃣ Jenkins Pipeline.

Create a New Pipeline Job in Jenkins:
- From the Jenkins dashboard, select New Item.
- Name the project, choose Pipeline, and click OK.
Configure the Pipeline:
- In the configuration, provide description,
- Triggers select - GitHub hook trigger for GITScm polling
- Setup a webhook on github
- Pipeline: Pipeline script from SCM.
- Choose Git as the SCM.
- Enter your GitHub repository URL, add Credentials
- In Branches to build, select your
- Verify the Script Path is Jenkinsfile.
- Save the configuration.
Testing:
- Initally click on Build Now from the left panel
- You may encounter Your pipeline failed because Jenkins does NOT have permission to access Docker
  - This is the most common Jenkins + Docker error.
  - Run these commands on your Jenkins server:
```
  sudo usermod -aG docker jenkins
  sudo systemctl restart jenkins
  sudo systemctl restart docker
```

Now try again.

Open your browser and navigate to -> http://your-ec2-ip:5000

Confirm the containers are running on the EC2 instance with docker ps.

Demo

Customization

Personalizing the Application

Update Profile Information

   <!-- In templates/index.html -->
   <div class="name">Your Name</div>
   <div class="title">Your Title</div>
   <div class="bio">Your bio here...</div>

Modify Tech Stack

   <!-- Add/remove tech tags -->
   <span class="tech-tag">Your Tech</span>

Customize Styling

   /* In templates/index.html */
   /* Modify colors, fonts, animations */

What’s Next?

Add monitoring
Add load balancer
Move deployment to Kubernetes
Add Terraform for IaC
Add automated testing inside pipeline

Contributing

Contributions are welcome! Here's how you can help:

Fork the repository
Create a feature branch: git checkout -b feature/amazing-feature
Commit your changes: git commit -m 'Add amazing feature'
Push to the branch: git push origin feature/amazing-feature
Open a Pull Request

Development Guidelines

Follow PEP 8 for Python code
Use meaningful commit messages
Add comments for complex logic
Test your changes locally
Update documentation as needed

🙌 Final Note

DevOps is not about memorizing tools, it’s about understanding how systems connect.

This project helps beginners experience that connection in a real, hands-on way.

🔗 GitHub Repository

👉 https://github.com/bHARATHaALETI018/2Tier_flask_app-Python-MySql-Jenkins-AWS-Docker.git

Acknowledgments

Flask Community for the amazing web framework
Docker for containerization technology
AWS for reliable cloud infrastructure
Jenkins for CI/CD automation
Open Source Community for inspiration and tools

📞 Contact & Support

Bharath Aaleti

💼 DevOps Engineer
🌐 LinkedIn: Connect with me
📧 Email: aaletibharath@gmail.com
🐙 GitHub: @bHARATHaALETI018

⭐ If you found this project helpful, share your thoughts, experience, I would really appreciate a LIKE

Built with ❤️ by Bharath Aaleti

The 3 Commands That Turn Chaos into Clarity in DevOps

Bharath Aaleti — Sun, 05 Oct 2025 17:45:07 +0000

“You don’t need fancy monitoring tools to troubleshoot production issues.

Sometimes, all you need is a terminal., and three magical commands: grep, awk, and sed.”

These three commands can turn gigabytes of messy logs into clear insights within seconds.

Once you master them, you’ll handle on-call like a pro.

Note: The examples below are just a few to get you started.

You can explore more scenario-based examples by asking ChatGPT or searching online.

🧠 Why Text Processing Matters in DevOps

In DevOps and SRE, every log line tells a story. But when you’re knee-deep in 2GB of logs, you need quick, powerful tools to extract meaning.

This is where grep, awk, and sed shine.

Imagine this: you’re SSH’d into a production server. The application is down. Logs are huge. Instead of downloading them, you filter and extract what you need right there.

🔍 grep - The Search Engine of Your Terminal

Purpose: Quickly find patterns in massive text files.

Syntax:

grep [OPTIONS] PATTERN [FILE...]

🔹 Common Examples

# Find all error lines in logs
grep "ERROR" /var/log/app.log

# Search recursively in directory
grep -R "timeout" /etc/nginx/

# Case-insensitive + show line numbers
grep -in "failed" auth.log

# Highlight matches
grep --color=auto "warning" server.log

💡 How it works:

grep scans text line by line and prints only those containing "ERROR". It’s the fastest way to find specific keywords in huge log files.

💡 Pro Tip:

Combine tail -f and grep to monitor logs live:

tail -f app.log | grep "ERROR"

🧰 Real-World Use Cases

Extracting failed pod names from Kubernetes logs:

  kubectl get pods | grep -i "error"

Searching for error traces in CI/CD pipelines:

  cat build.log | grep "Exception"

Monitoring real-time logs for failures:

  tail -f app.log | grep "500"

Alternatives:

ripgrep (rg) - faster alternative, written in Rust. 👉 But grep remains the universal standard.

⚙️ awk - The Swiss Army Knife for Structured Data

Purpose: Process and analyze structured text line by line.

Syntax:

awk 'pattern {action}' file

🔹 Practical Examples

# Print specific columns
awk '{print $1, $4}' access.log

# Filter based on condition
awk '$3 == "ERROR" {print $0}' app.log

# Calculate average CPU usage
awk '{sum += $2} END {print "Avg:", sum/NR}' cpu.txt

# Show top 5 largest files
ls -lh | awk '{print $5, $9}' | sort -rh | head -5

💡 How it works:

Each line in access.log represents one request, and the 5th column ($5) stores its response time (in ms, for example).

sum+=$5 → adds up all response times
NR → built-in variable for the total number of lines processed
END {print sum/NR} → prints the average at the end

🧰 Real-World Use Cases

Extracting pod memory usage:

  kubectl top pods | awk '{print $1, $3}'

Summarizing response times:

  awk '{sum += $5} END {print "Avg Response:", sum/NR}' access.log

Counting unique IPs from access logs:

  awk '{print $1}' access.log | sort | uniq | wc -l

Alternatives:

cut (simpler but limited)
jq (great for JSON)
csvkit (for CSVs) 👉 awk remains the most versatile for text streams.

✂️ sed - The Stream Editor

Purpose: Edit, replace, and transform text on the fly.

Syntax:

sed [OPTIONS] 'command' file

🔹 Common Examples

# Replace all “foo” with “bar”
sed 's/foo/bar/g' input.txt

# Delete blank lines
sed '/^$/d' file.txt

# Replace in place (useful for config changes)
sed -i 's/localhost/127.0.0.1/g' config.yml

# Prints lines starting from 5-10
sed -n '5,10p' file.txt

💡 How it works:

sed (stream editor) searches and replaces text directly in files.
's/dev/prod/g' → replace all instances of “dev” with “prod”
-i → edit the file in place (no need to open a text editor)

🧰 Real-World Use Cases

Update IPs or URLs in config files:

  sed -i 's/dev.example.com/prod.example.com/g' nginx.conf

Remove comments before deployment:

  sed '/^#/d' .env

Bulk change environment variables in CI/CD:

  sed -i 's/DEBUG=True/DEBUG=False/' settings.py

Alternatives:

perl -pe (more powerful but heavier)
tr (for simple character replacement) 👉 sed is lightweight and available by default everywhere.

🧩 Combining Them for Power One-Liners

Extract failed requests with response time > 500ms:

grep "FAILED" app.log | awk '$5 > 500' | sed 's/FAILED/ERROR/g'

Monitor errors in real time:

tail -f app.log | grep "ERROR"

Find all 404s and count them:

grep "404" access.log | wc -l

Extract usernames from access log and sort by frequency:

awk '{print $3}' access.log | sort | uniq -c | sort -nr | head

Replace all IPs with placeholders (for redaction):

sed -E 's/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/[REDACTED]/g' access.log

💬 Final Thoughts

Once you start combining these three, you’ll realize you can transform any text stream into usable insights without ever leaving the terminal.

Whether you're debugging a failed deployment, analyzing performance logs, or cleaning config files, these commands will make your life easier.

Learn them once. Use them forever.

👋 I'm Bharath Aaleti, DevOps Engineer - learning DevOps and sharing everything I know/learn.

If you found this helpful, connect or drop your favorite one-liner below 👇

🔐Raise your hand if you use SSH every day without actually knowing what it does. Yeah, me too😁 you’re definitely not alone.

Bharath Aaleti — Sun, 05 Oct 2025 16:19:21 +0000

The spy-movie handshake of cryptography and trust that makes remote login possible.

🟢 In simple terms

An SSH connection is a secure tunnel between your computer and another computer (usually a server) over the internet.

It’s like a private, encrypted walkie-talkie:

You type commands → they get encrypted → sent to the server.
The server runs them → sends results back → also encrypted. So even if someone is watching your network, all they see is gibberish.

🚨 The Problem SSH Solved (And Why We Almost Lost the Internet)

Back then, system administrators were sending ROOT PASSWORDS across the internet in PLAIN TEXT.

Yes, you read that right.

Every login, every command, every sensitive operation. Completely visible to anyone sniffing network traffic.

Protocols like Telnet, rsh, and FTP were basically handing over your servers to attackers on a silver platter.

🛡️ How SSH Actually Works (The Magic Behind The Scenes)

💻🕵️ What Actually Happens When You Hit ssh user@server

SSH isn’t just remote login. It’s a secure negotiation, identity verification, and a puzzle-solving protocol, all happening in milliseconds. When you type ssh user@server.com, here’s what happens in milliseconds:

The beautiful part? Even if someone intercepts your traffic, they see gibberish.

✅ 1. Client says: “Yo, can we talk?”

ssh user@server.com

Your SSH client tries to open a TCP connection on port 22 to server.com.

It’s like your laptop walking up to the server and knocking:

🧑‍💻💬 “Hey, you open for secure chat?”

🔐 2. The server replies: “Let’s talk, but securely.”

The server responds with: A list of encryption algorithms it supports

Its public host key (used to verify its identity)

This is the server’s way of saying:

🖥️💬 “Cool! but first, here’s my ID and my preferred encryption dance moves.”

Check server fingerprint manually (first time only):

ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub

🕵️‍♂️ 3. Client checks: “Do I know you?”

SSH compares the server’s fingerprint with your ~/.ssh/known_hosts.

✅ If it’s already known → we’re chill

❌ If not → SSH warns you:

Are you sure you want to continue connecting (yes/no)?

You type yes, and it gets saved for next time.

Basically:

“Hey, this guy’s new. You sure this isn’t a trap server?”

🔄 4. Encryption Negotiation (the handshake)

Now the client and server agree on a common encryption algorithm (like AES, ChaCha20, etc.) and key exchange method (like Diffie-Hellman or ECDH).

They perform a key exchange to generate a shared session key (used to encrypt the session).

Both solve the same crypto puzzle, ending with the same session key.

⚠️ The cool part?

They both end up with the same session key, but no one else on the network can figure it out.

It’s like solving the same puzzle separately and both ending up with the same secret phrase.

This is what makes SSH secure even over open networks.

No command here — it’s all under the hood.

👤 5. User Authentication (Prove you’re you)

Once the tunnel is encrypted, the client says:

🧑💬 “Okay server, now let me in.”

The server asks for user authentication, which could be:

🛑 Password (not recommended)

ssh user@server.com

# Then enter password

✅ Public Key (much better)

ssh-copy-id user@server.com # one-time setup

ssh user@server.com # passwordless login

Your client proves it has the private key that matches a public key on the server (usually in ~/.ssh/authorized_keys).

It signs a challenge, and the server verifies it with your public key.

No passwords fly over the wire. 🛡️

🎉 6. Session Established: Let the magic begin

Now the secure session is live:

Every keystroke → encrypted
Every response from the server → encrypted

You’re inside the Matrix 🧠🔒

This tunnel can now be used for:

🧑‍💻 Interactive shell

📂 File transfer (SCP, SFTP)

🔄 Port forwarding

🧰 Tunneling other protocols (Git, DB access, etc.)

> Here a video by @ByteByteGo, really helpful.

How SSH Really Works

Bonus:

🔧 SSH Config —> Stop Memorizing Commands!

Instead of typing:

ssh -i ~/.ssh/prod-key.pem -p 2222 deployer@203.0.113.10

Create ~/.ssh/config:

Host prod
  HostName 203.0.113.10
  User rhel
  IdentityFile “~/.ssh/prod-key.pem”
  Port 2222

Now just type: `ssh prod` 🎯

But SSH wins because:

✅ Universal (works everywhere)

✅ Lightweight (minimal resources)

✅ Scriptable (perfect for automation)

✅ Battle-tested (years of security hardening)

✅ Port forwarding (tunnel ANY protocol)

🔄 TL;DR -> for my scroll-happy friends:

SSH is still around because it’s 🔒 simple + powerful + everywhere

It’s not magic, It’s a full-on encrypted handshake, identity check, cryptographic puzzle, and channel manager, all rolled into one elegant protocol.

Alternatives exist, but SSH is the OG and still undefeated.

Next time you type ssh, remember:

You’re starting a secret spy mission over port 22. And your terminal?

It’s the encrypted walkie-talkie.

Let’s hear the horror stories. Bonus points if it involves rm -rf or scp to the wrong server. 😂👇👇

This is my first post in here, this piece is my small step toward learning in public and giving back to the community that’s taught me so much.

Found this helpful? Hit that Like and share your story.🚀

DEV Community: Bharath Aaleti

Building a Real DevOps Project from Scratch: A Beginner-Friendly Walkthrough

Table of Contents

First, What Are We Actually Building?

✔️ Tier 1 → Flask Web App

✔️ Tier 2 → MySQL Database

✔️ Both running as Docker containers

Understanding the DevOps Concepts (Beginner-Friendly Explanations)

🐳 Docker (Containerization)

🔄 CI/CD with Jenkins

☁️ AWS EC2

📦 Git + GitHub

Architecture (Simplified)

What a Beginner Learns

Real Challenges You’ll Face

🛠️ Full Project Stack

Quick Start

1️⃣ Server Setup

2️⃣ Environment Setup

3️⃣ Jenkins Installation and Setup

4️⃣ Jenkins Pipeline.

Demo

Customization

Personalizing the Application

What’s Next?

Contributing

Development Guidelines

🙌 Final Note

🔗 GitHub Repository

Acknowledgments

📞 Contact & Support

The 3 Commands That Turn Chaos into Clarity in DevOps

Note: The examples below are just a few to get you started.

You can explore more scenario-based examples by asking ChatGPT or searching online.

🧠 Why Text Processing Matters in DevOps

🔍 grep - The Search Engine of Your Terminal

🔹 Common Examples

💡 How it works:

💡 Pro Tip:

🧰 Real-World Use Cases

⚙️ awk - The Swiss Army Knife for Structured Data

🔹 Practical Examples

💡 How it works:

🧰 Real-World Use Cases

✂️ sed - The Stream Editor

🔹 Common Examples

💡 How it works:

🧰 Real-World Use Cases

🧩 Combining Them for Power One-Liners

💬 Final Thoughts

🔐Raise your hand if you use SSH every day without actually knowing what it does. Yeah, me too😁 you’re definitely not alone.

🟢 In simple terms

🚨 The Problem SSH Solved (And Why We Almost Lost the Internet)

🛡️ How SSH Actually Works (The Magic Behind The Scenes)

💻🕵️ What Actually Happens When You Hit ssh user@server

✅ 1. Client says: “Yo, can we talk?”

🔐 2. The server replies: “Let’s talk, but securely.”

🕵️‍♂️ 3. Client checks: “Do I know you?”

🔄 4. Encryption Negotiation (the handshake)

👤 5. User Authentication (Prove you’re you)

🎉 6. Session Established: Let the magic begin

> Here a video by @ByteByteGo, really helpful.

Bonus:

Now just type: ssh prod 🎯

But SSH wins because:

🔄 TL;DR -> for my scroll-happy friends:

Next time you type ssh, remember:

Now just type: `ssh prod` 🎯