When we use apps like Instagram, Uber, or Netflix, we don’t think about what’s happening behind the scenes.
But behind every app, there’s:
A server handling requests
A database storing information
A system that checks if everything is working
Monitoring tools watching performance
In this project, I built a small version of that real-world system using:
Flask (a Python web framework)
PostgreSQL (a database)
Redis (a cache)
Docker (to run everything in containers)
Prometheus (to collect metrics)
Grafana (to visualize those metrics)
Let me explain it in simple words.
💡 What This Project Does (In Simple Terms)
Imagine this scenario:
You open an app and request some data.
Here’s what happens in my project:
You send a request to the Flask API.
The API checks Redis first (fast memory).
If data isn’t there, it asks PostgreSQL (database).
The API sends the response back to you.
Meanwhile, Prometheus tracks how many requests were made.
Grafana displays those numbers in charts.
So this project is basically:
👉 A small backend system that works like a real production server.
🏗️ Architecture Overview (Simple Illustration)
4
Let’s imagine it like a restaurant:
👤 Customer → Sends request
👨🍳 Flask API → Chef preparing the food
🧠 Redis → Quick memory of popular orders
📚 PostgreSQL → Storage room with all ingredients
📊 Prometheus → Manager counting how many customers came
📈 Grafana → Dashboard showing daily performance
Each service has its own “room” (container), but they all work together.
🐍 The Core Flask App (The Heart of Everything)
Here’s a simplified version of the main app:
from flask import Flask
from prometheus_client import Counter, generate_latest
app = Flask(name)
Create a counter to track requests
REQUEST_COUNT = Counter('app_requests_total', 'Total number of requests')
@app.route('/')
def home():
REQUEST_COUNT.inc() # Increase counter every time someone visits
return "Hello, World!"
@app.route('/metrics')
def metrics():
return generate_latest()
What’s happening here?
Flask() creates our web server.
Counter() creates a metric that counts visits.
Every time someone visits /, we increase the counter.
/metrics lets Prometheus read those numbers.
Think of it like putting a click counter on a website.
Every visit = +1.
🐳 What is Docker and Why Did I Use It?
Before Docker, setting up apps could be messy:
“It works on my computer but not yours.”
Different software versions.
Dependency conflicts.
Docker solves that by putting each service into its own container.
Think of a container like:
📦 A lunchbox with everything the app needs inside.
No matter where you run it — it behaves the same.
*📄 Dockerfile Explained *
Here’s the Dockerfile:
FROM python:3.10
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "app.py"]
This is like a recipe:
FROM python:3.10 → Start with a kitchen that already has Python.
WORKDIR /app → Work inside the “app” folder.
COPY requirements.txt . → Bring the list of ingredients.
RUN pip install → Install dependencies.
CMD → Start the app.
Without this file, Docker wouldn’t know how to build the app.
🧩 Docker Compose Explained (The Big Organizer)
If Dockerfile builds one container,
Docker Compose runs many containers together.
Here’s a simplified version:
version: "3.9"
services:
web:
build: .
ports:
- "5000:5000"
postgres:
image: postgres:15
volumes:
- pgdata:/var/lib/postgresql/data
redis:
image: redis:7
prometheus:
image: prom/prometheus
ports:
- "9090:9090"
grafana:
image: grafana/grafana
ports:
- "3000:3000"
volumes:
pgdata:
This file says:
Build the Flask app
Run PostgreSQL
Run Redis
Run Prometheus
Run Grafana
Save database data permanently
And start everything with:
docker compose up -d
One command.
Five services.
Fully running system.
💾 Volumes (Why They Matter)
Without volumes:
If the container stops, your database data disappears.
With volumes:
Data stays safe.
In the compose file:
volumes:
- pgdata:/var/lib/postgresql/data
This means:
“Store database files outside the container.”
Volumes are like external hard drives attached to containers.
🌐 Networking (How Containers Talk)
At first, Grafana couldn’t connect to Prometheus.
It kept timing out.
The mistake?
I used:
Inside Docker, localhost means:
“The container itself.”
The correct way was:
Because Docker automatically creates a private network.
Containers talk using service names.
It’s like calling someone using their office extension number.
📊 Monitoring (Why This Is Important)
Most beginner projects stop at:
“It works!”
But companies need to know:
How many users are hitting the API?
Is the server slow?
Is it down?
That’s where monitoring comes in.
📈 Prometheus
Prometheus collects numbers (metrics).
Example metrics:
Total requests
Requests per second
Service status
Prometheus configuration:
scrape_configs:
- job_name: 'flask_app'
static_configs:
- targets: ['web:5000']
This tells Prometheus:
“Go check the Flask app for metrics.”
📊 Grafana
Grafana turns those numbers into beautiful graphs.
When I added this query:
app_requests_total
I could literally see the graph go up as I refreshed the page.
That moment changed everything.
I wasn’t just coding.
I was observing a live system.
⚠️ Challenges I Faced
1️⃣ Grafana Timeout Errors
Problem:
It wouldn’t connect to Prometheus.
Cause:
Wrong URL.
Lesson:
Containers talk using service names like:
Not localhost.
2️⃣ “Unhealthy” Flask Container
Problem:
Docker marked my app as unhealthy.
Cause:
Health check didn’t match real route.
Example health check:
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:5000/"]
Lesson:
Health checks must point to real working endpoints.
Monitoring is about reliability, not just graphs.
3️⃣ Data Disappearing
Without volumes, data resets when containers restart.
That was a big lesson:
Containers are temporary.
Data must be persistent.
This is the link to the project on github: https://github.com/ManYuki/Project.git
Top comments (0)