From SQLite to Kubernetes: Building a Test-Gated CI/CD Pipeline for a Task Relay Service
I recently completed an engineering exercise that started as a small FastAPI + SQLite application and gradually evolved into a containerized, Kubernetes-deployed service with a full CI/CD pipeline.
The project is Agent Relay — a task relay service for registering agents, delivering tasks, tracking claims, and recording results.
The evolution
SQLite → PostgreSQL → Docker → Kubernetes (Kind) → GitHub Actions CI/CD
What made this interesting wasn't just "getting it running" — it was watching each layer expose a new class of problem that the previous layer didn't have to deal with.
What I implemented and verified along the way
• Agent registration and authentication
• At-least-once task delivery
• Lease-based task claims
• Worker heartbeats
• Task recovery after worker failure
• Idempotent terminal requests
• PostgreSQL persistence
• Docker containerization
• Kubernetes Deployment, Service, Secret, and PVC
• Automated pytest verification
• Git SHA-based Docker image tagging
• Test-gated CI/CD
• End-to-end deployment verification
The lesson that mattered most
A successful Kubernetes rollout does not mean an application is actually working. kubectl rollout status can go green while the app underneath is silently broken — a bad config, a missing secret, a dependency that never connected.
So the pipeline doesn't stop at "the pods are Running." After deployment, it port-forwards the service and runs an application-level check against the dashboard — the same kind of check a human would do by opening the page and looking at it.
The pipeline, accurately
My first pass at diagramming this pipeline drew pytest, Docker build, and Kubernetes deployment as three parallel branches. That's not what actually happens. The real pipeline is sequential and test-gated: nothing gets built or deployed unless the tests pass first.
Developer
↓
Git Repository (push)
↓
GitHub Actions
↓
job: test
├── setup Python 3.11 + uv
├── Postgres 16-alpine (service container)
└── uv run pytest -q
↓ (needs: test → only proceeds if tests pass)
job: build-and-deploy
├── Docker build (image tagged with Git SHA)
└── deploy to Kind cluster
├── kubectl apply -f k8s/
├── kubectl set image
└── rollout status
↓
Agent Relay (FastAPI service, incl. dashboard)
↓
Verification: port-forward + curl → dashboard check
Why this matters beyond one project
The instinct to treat "deployed" as "done" is exactly the instinct that breaks pipelines at scale. Whether it's a Kubernetes rollout or a data/ML pipeline, the pattern is the same: infrastructure-level success (pod is Running, DAG task is green) is not application-level success (the thing actually does what it's supposed to do). Closing that gap with a real, purpose-built check — not just a health probe — is what turns "it deployed" into "it works."
I'm currently exploring how these same engineering principles — test gating, staged evolution, verifying at the application level rather than the infrastructure level — apply to larger Data Engineering and AI/LLM systems, where the failure modes are often even less visible than a crashed pod.
The repository contains the implementation, Kubernetes manifests, Docker configuration, automated tests, and the CI/CD workflow.
GitHub repo: https://github.com/ketut-garjita/agent-relay
Top comments (0)