DevPilot: I Built the IDP I Wish Existed as a DevOps Engineer

#devchallenge #githubchallenge #devops #github

GitHub “Finish-Up-A-Thon” Challenge Submission

DevPilot: How I Finally Finished My IDP After Abandoning It for a Year

What I Built

DevPilot — a self-hosted Internal Developer Platform (IDP) that does in one UI what used to take a full platform team: connects your GitHub/Bitbucket repos, auto-generates Jenkinsfiles and Dockerfiles using AI, runs CI/CD pipelines, self-heals failed builds automatically, deploys to any VM via SSH, and provisions AWS infrastructure (EC2, RDS, S3) on demand.

No Backstage. No YAML sprawl. No plugin ecosystem to maintain. Just connect your repo and go.

GitHub repo:

pavan-3000 / DevopsPilot

The Problem I Was Solving

I was working as a DevOps engineer in India. Most of our developers were based in the US — which means their working hours are our midnight.

The pattern repeated itself every week:

A US developer hits a blocker at their 10am (our 10pm) — they need a new Jenkins pipeline, or an EC2 instance, or a Dockerfile fix
Our DevOps team is asleep
The developer waits. Half a day of productivity gone.
Or we get a call. Someone on the India team works through the night to unblock one developer on the other side of the world.

This wasn't a people problem. Our team was good. It was a process problem — developers shouldn't need to wait for a DevOps engineer to create a Jenkins job. That's not skilled work. It's a form.

I started asking: what if a developer could just click a button, pick their repo and branch, and have a working pipeline in 60 seconds? No Slack message. No ticket. No waiting for someone in a different timezone to wake up.

That's when I found Platform Engineering and the concept of an IDP — a self-service layer that puts pipeline creation, infrastructure provisioning, and build management directly in the developer's hands. My first attempt was Backstage.

The "Before" — The Abandoned Backstage Project

About a year ago I started building an IDP on top of Backstage (Spotify's open-source platform). I got excited. I read the docs. I cloned the repo.

Then reality hit:

Backstage is a framework, not a product. You still build everything yourself.
The plugin ecosystem is huge but poorly documented.
Kubernetes is basically required for production hosting.
Every config change meant rebuilding the entire catalog.
My day job kept interrupting — on-call rotations, release freezes, incident reviews.

After 3 months I had: a running Backstage instance that showed a software catalog. That's it. No pipelines. No automation. No healing.

I closed the laptop and didn't open that folder again for eight months.

The "After" — DevPilot

This challenge made me pick it up again. Except I didn't pick up Backstage — I threw it out and built what I actually needed.

DevPilot is the IDP I wish existed when I started as a DevOps engineer:

Connect in minutes

Link GitHub or Bitbucket via OAuth. Connect your Jenkins server with a URL + API token. That's the setup.

AI-powered pipeline creation

No Jenkinsfile? No Dockerfile? DevPilot generates both using Claude AI — a production-ready Dockerfile based on your language, and a Jenkinsfile with Docker build, Trivy security scan, registry push, and VM deploy stages pre-wired.

Self-healing CI

When a build fails, DevPilot reads the build log, asks the AI what's wrong, patches the Jenkinsfile or Dockerfile, pushes the fix to your repo, and retriggers the build — automatically, up to 5 times. Engineers wake up to a green build, not a Slack mention.

Infrastructure provisioning

EC2, RDS, S3 — provision from the UI. DevPilot generates the SSH keypair, spins the instance, bootstraps Docker, and registers it as a deploy target in under 5 minutes. Or sync your existing AWS resources with one click.

Command Center

A split-panel AI chat UI. Ask "what's failing?" or "give me a pipeline report" in plain English. Deploy, stop, or debug pipelines directly from the chat. Manage Docker containers running on your EC2 instances without leaving the browser.

How GitHub Copilot Helped

When I restarted this project, I used AI heavily throughout development — including GitHub Copilot for autocompletion in VS Code. The parts where it made the biggest difference:

AWS SDK v3 modular imports — the new @aws-sdk/client-* pattern was unfamiliar to me. Copilot correctly autocompleted CreateBucketCommand, PutPublicAccessBlockCommand, and the polling loop for DescribeInstanceStatus without me having to read the full SDK docs every time.
Jenkins XML config generation — the config.xml format for pipeline jobs is tedious. Copilot completed the XML template once I typed the opening tags, saving me from reading the Jenkins API docs for each credential type.
SSE (Server-Sent Events) streaming — I hadn't implemented SSE before. Once I typed res.setHeader('Content-Type', 'text/event-stream'), Copilot suggested the correct keep-alive, flush pattern, and cleanup on req.on('close') which I would have missed.
node-forge RSA keypair generation — unfamiliar library. Copilot autocompleted the async keygen call and the PEM export format on the first try.

The self-healing loop logic I wrote myself — it was the core innovation and I wanted full ownership of how it worked. But Copilot saved me hours on the boilerplate that surrounds the interesting parts.

Key Features (Before vs After)

Capability	Backstage attempt (Before)	DevPilot (After)
CI pipeline creation	Manual plugin setup, never finished	AI-generated Jenkinsfile, one wizard
Dockerfile	Not implemented	AI-generated on first push
Build failure recovery	Not implemented	Self-healing loop (5 retries, AI-patched)
Infrastructure	Not implemented	EC2 + RDS + S3 provisioning
AI integration	Not implemented	Claude/GPT throughout — generate, fix, analyze, chat
Deploy to VM	Not implemented	SSH deploy stage, docker pull + run
Status	Abandoned	Shipped

Technical Stack

Frontend: React 18, React Router v6, inline CSS (dark GitHub-style theme)
Backend: Node.js, Express 4, PostgreSQL 16
AI: Anthropic Claude (claude-haiku-4-5) or OpenAI GPT-4o-mini
AWS SDK: @aws-sdk/client-ec2, @aws-sdk/client-rds, @aws-sdk/client-s3 (v3)
SSH/Crypto: node-forge (RSA keypair gen), ssh2 (EC2 bootstrap)
Deploy: Docker + docker-compose, Nginx reverse proxy

What I Learned

Building this in the "finish-up" sprint taught me something I should have known earlier: the value of a product comes from finishing, not from architecting. Backstage is architecturally beautiful. DevPilot is a 4,000-line Express app. DevPilot is the one that actually solves the problem.

The self-healing loop is the feature I'm most proud of — not because it's technically impressive (it's a polling loop with an AI call inside), but because it's genuinely useful. The first time I watched it fix a broken Dockerfile at midnight without me touching anything, I understood why I built this.