A learning-in-public story about Flask, Jenkins, AWS EC2, systemd, and finally shipping a live demo on Vercel.
🎯 TL;DR
I built PyPulse, a tiny Flask app, and wired it up to a full CI/CD pipeline: push to GitHub → Jenkins builds → tests → deploys to AWS EC2 → auto-triggered via webhook → managed by systemd. Along the way I broke almost every piece of it at least once, and fixed each one. I also deployed a permanent live demo on Vercel, since my EC2 instance is running on the AWS free trial and won't live forever.
🔗 Live demo: pypulse-pi.vercel.app 🔗 Live demo (health check): pypulse-pi.vercel.app/health
If you're learning DevOps and want to see what the real, messy version of "just set up a CI/CD pipeline" looks like — not the polished tutorial version — this is that.
🧰 The Stack
Piece Tool Job
App Flask + pytest + gunicorn The actual web app and its tests
CI/CD Jenkins (on EC2, Ubuntu 22.04) Build → Test → Deploy automation
Source control GitHub Single source of truth
Trigger GitHub Webhook Auto-runs the pipeline on every push
Process management systemd Keeps the app alive on reboot/crash
Permanent demo Vercel Live URL that survives EC2 termination
🏗️ The App: PyPulse
Nothing fancy on purpose — the whole point of this project was the pipeline, not the app.
python
app.py
from flask import Flask, jsonify
from datetime import datetime, timezone
app = Flask(name)
@app.route("/")
def home():
return jsonify({
"message": "Hello from PyPulse",
"time": datetime.now(timezone.utc).isoformat()
})
@app.route("/health")
def health():
return jsonify({"status": "ok"}), 200
if name == "main":
app.run(host="0.0.0.0", port=5000)
Two routes. Two tests. That's it. Small enough that when something broke, I knew it wasn't the app — it was the plumbing around it. That turned out to be the right call, because the plumbing broke a lot. 😅
⚙️ The Pipeline: Build → Test → Deploy
Here's the mental model I ended up with for a Jenkinsfile:
Each stage is a gate. If Build fails, Test never runs. If Test fails, Deploy never runs. That ordering is the entire value of CI/CD — automation with checkpoints, not just automation.
🔨 Build
groovy
stage('Build') {
steps {
sh '''
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
'''
}
}
Runs on the Jenkins machine itself. Fresh virtual environment, install dependencies. If a dependency is broken or missing, this fails immediately — before wasting time testing or deploying broken code.
✅ Test
groovy
stage('Test') {
steps {
sh '''
. venv/bin/activate
pytest
'''
}
}
Runs the two tests against / and /health. This is the safety net: if I break a route later, this stage fails and the pipeline stops before broken code ever reaches the live server.
🚀 Deploy
groovy
stage('Deploy') {
steps {
sshagent(credentials: ['pypulse-ec2-ssh']) {
sh '''
set -e
timeout 60 ssh -o StrictHostKeyChecking=no ${EC2_USER}@${EC2_IP} "
set -e
if [ ! -d ${APP_DIR} ]; then
git clone https://github.com/IrfanPasha05/pypulse.git ${APP_DIR}
fi
cd ${APP_DIR}
git pull origin main
source venv/bin/activate
pip install -r requirements.txt
sudo systemctl restart pypulse
"
'''
}
}
}
Only runs if Build and Test both passed. SSHes into EC2, pulls the latest code, reinstalls dependencies, and restarts a systemd service (more on why, below).
🐛 The Debugging Journey (the actually useful part)
Anyone can copy a working Jenkinsfile. Here's what happened when mine didn't work — because this is the part that actually teaches you something.
Bug #1: "ensurepip is not available"
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package
What happened: Jenkins runs on the same machine as the OS, and that machine didn't have the python3-venv package installed for its Python version.
Fix:
bash
sudo apt install python3.14-venv -y
Lesson: the Jenkins host machine needs every tool your build script assumes exists. It's easy to forget Jenkins isn't magic — it's just running your shell commands on a real box.
Bug #2: "No such DSL method 'sshagent' found"
java.lang.NoSuchMethodError: No such DSL method 'sshagent' found among steps [...]
What happened: my Jenkinsfile called sshagent(...), but the SSH Agent plugin wasn't actually installed on this Jenkins instance. Jenkins helpfully prints every step it does recognize — a big wall of text that's actually a clue: if your step isn't in that list, it's a missing plugin, not a typo.
Fix: Manage Jenkins → Plugins → install "SSH Agent" → restart.
Bug #3: "Could not find specified credentials: pypulse-ec2-ssh"
Plugin installed, but now:
ERROR: [ssh-agent] Could not find specified credentials: pypulse-ec2-ssh
What happened: I referenced a credential ID in the Jenkinsfile that I'd never actually created in Jenkins. Checked Manage Jenkins → Credentials → Global, and — empty. 🙃
Fix: Added an "SSH Username with private key" credential, ID matching exactly, pasted in the EC2 .pem key contents.
Lesson: credential IDs are just strings you're trusting to exist. Jenkins won't tell you "hey, go create this" — it just fails at the exact moment it needs it.
Bug #4: The silent deploy that wasn't deploying anything
The pipeline reported SUCCESS. The app was not reachable. curl to the EC2 IP: ERR_CONNECTION_REFUSED.
bash
$ ls -la ~/pypulse
ls: cannot access '/home/ubuntu/pypulse': No such file or directory
What happened: my Deploy script used git pull, assuming the repo already existed on the server — but I'd never actually cloned it there. The cd into a nonexistent directory silently failed, and because the shell script didn't have set -e, everything after it just... didn't run. Jenkins still reported success, because no command technically errored out loud.
Fix: made the deploy script self-healing (clone if missing, pull if present) and added set -e so any real failure stops the pipeline and reports it honestly:
bash
if [ ! -d ${APP_DIR} ]; then
git clone https://github.com/IrfanPasha05/pypulse.git ${APP_DIR}
fi
Lesson: a green checkmark only means what your script actually checks. set -e is not optional if you care whether "success" is real.
Bug #5: The SSH hang (this one took a few rounds)
Even after fixing the clone issue, deploys would just... hang. Forever. No error, no timeout, just Still waiting.
What happened: I was starting gunicorn in the background with nohup ... & inside an SSH command chained with &&. Turns out, chaining a backgrounded process into a && sequence backgrounds the whole chain, not just gunicorn — and the SSH session keeps waiting for that background job before it'll actually close. Classic "it works until you actually test what 'done' means."
Fix, in stages:
Split commands onto separate lines instead of one long && chain
Added setsid to fully detach gunicorn from the SSH session
Added timeout 60 around the whole SSH call, so if it ever hung again, the build would fail loudly instead of hanging forever
That got it working — but it was still fragile. Which led to the real fix:
The Real Fix: systemd
Instead of fighting nohup/setsid/disown forever, I made systemd own the process:
ini
/etc/systemd/system/pypulse.service
[Unit]
Description=PyPulse Flask app (gunicorn)
After=network.target
[Service]
User=ubuntu
WorkingDirectory=/home/ubuntu/pypulse
ExecStart=/home/ubuntu/pypulse/venv/bin/gunicorn --bind 0.0.0.0:5000 app:app
Restart=always
RestartSec=3
[Install]
WantedBy=multi-user.target
bash
sudo systemctl daemon-reload
sudo systemctl enable pypulse
sudo systemctl start pypulse
Now the Deploy stage is just:
bash
sudo systemctl restart pypulse
No backgrounding, no SSH hangs, no manual process management. Ubuntu handles restarts, crash recovery, and boot survival on its own.
Lesson: if you're fighting shell process semantics, you're probably solving the wrong problem. Let the OS's actual process manager do its job.
(One gotcha: sudo systemctl restart needs passwordless sudo for that specific command, via visudo, or SSH hangs waiting on a password prompt that never comes.)
🔔 Auto-Triggering With a GitHub Webhook
Manually clicking "Build Now" isn't really continuous anything. So — GitHub webhook time.
The catch: GitHub's servers need to reach Jenkins over the public internet. Since my Jenkins was already running directly on the EC2 instance (which has a public IP), this was simpler than expected — no tunnel needed, just:
Opened port 8080 in the EC2 security group
Added a webhook in GitHub → http://:8080/github-webhook/
Enabled "GitHub hook trigger for GITScm polling" in the Jenkins job config
Pushed a test commit, watched Jenkins dashboard — a new build kicked off on its own, no click required. That's the moment this stopped being "a pipeline I run" and became "a pipeline that runs itself."
🌐 Bonus: A Permanent Demo on Vercel
Here's the thing about learning on a free-tier EC2 instance: eventually you terminate it, and everything built on top of it goes dark. So I added a second, independent deployment target — Vercel — purely to have a permanent, always-live URL to show off the app itself (not the pipeline).
Two small files, no changes to app.py:
python
api/index.py
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(file), '..'))
from app import app
json
// vercel.json
{
"version": 2,
"builds": [
{ "src": "api/index.py", "use": "@vercel/python" }
],
"routes": [
{ "src": "/(.*)", "dest": "api/index.py" }
]
}
Connected the GitHub repo to Vercel, clicked deploy, and it worked on the first try — a nice contrast after everything Jenkins put me through. 😄
Important distinction: Vercel doesn't run my Jenkinsfile at all — it has its own build system. So I now have two independent things: the Jenkins CI/CD pipeline (the thing I actually built and learned from) and a Vercel-hosted demo (a permanent link that outlives EC2). Both matter, for different reasons.
🧭 What I'd Do Differently
Add set -e from day one. A pipeline that can lie about success is worse than one that fails loudly.
Use systemd from the start, not as a bug-fix. Backgrounding processes over SSH is a trap.
Check "does this plugin/credential actually exist" before writing Jenkinsfile steps that assume it does.
🔭 What's Next
Containerize the app with Docker and deploy via Jenkins to ECR/ECS instead of a raw EC2 box
Add a staging environment before production deploy
Basic monitoring/alerting on the systemd service
🏁 Final Thoughts
None of this went smoothly, and that's kind of the point. Every error message here was a real thing I hit, read carefully, and fixed — not something I knew going in. If you're early in learning DevOps: the tutorials that work on the first try teach you syntax. The ones that break teach you how the pieces actually fit together.
🔗 Try the live app: pypulse-pi.vercel.app 💻 GitHub: github.com/IrfanPasha05/pypulse
If you've hit any of these same errors, I'd love to hear about it in the comments — misery loves company, and so does debugging. 👋
Top comments (0)