Building a DevSecOps Pipeline with Jenkins, SonarCloud, Trivy & Docker Hub, What I Learned the Hard Way

If you have ever followed a CI/CD tutorial and had it work perfectly on the first try, this article probably is not for you. This is the story of building a real Jenkins DevSecOps pipeline from a bare EC2 instance including every wrong turn, cryptic error, and "wait, why is this failing now" moment along the way.

I'm sharing the full journey (not just the polished end result) because the errors were honestly the most useful part of building this.

What I built

A multi stage Jenkins pipeline that takes a small Node.js/Express app from source code to a published Docker image, with real quality and security checks gating every step:

1. Checkout — pull the latest code from GitHub
2. Install & Test — install dependencies, run automated tests (Jest + Supertest)
3. Code Quality — static analysis via SonarCloud
4. Quality Gate — block the pipeline if code doesn't meet quality standards
5. Security Scan — Trivy scans dependencies for known vulnerabilities
6. Build Docker Image
7. Scan Docker Image — Trivy scans the built image, catching OS-level vulnerabilities too
8. Push to Docker Hub — only if every check above passed

The idea is "shift-left" security: catch problems before you build and ship, not after.

The setup

Everything runs on a single EC2 instance (t3.small, Ubuntu 24.04):

• Jenkins (the orchestrator)
• Docker (to build/scan/push images)
• Trivy (vulnerability scanner)
• Node.js (to run the app's own install/test steps)

SonarCloud handles code quality analysis as a hosted service, so I did not need to self host SonarQube.

Real problems I hit (and how I fixed them)

1. Jenkins wouldn't even start

Running with Java 17 from /usr/lib/jvm/java-17-openjdk-amd64, which is older than the minimum required version (Java 21).

Modern Jenkins requires Java 21, not 17. Installed openjdk-21-jre, switched the default with update-alternatives --config java, and Jenkins started cleanly.

Lesson: always check your CI tool's actual current requirements before installing dependencies from memory or an old tutorial — these things move faster than you'd expect.

2. apt-key is dead

Installing Trivy's repo the "classic" way failed:

sudo: 'apt-key' command not found

apt-key has been removed from modern Ubuntu because it was too permissive (a key added via apt-key was trusted system-wide, not scoped to one repo). The fix is the modern signed-by approach:

wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | \
  gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null

echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | \
  sudo tee /etc/apt/sources.list.d/trivy.list

3. Builds stuck at "Waiting for next available executor"

Turned out to be a disk space issue that Jenkins' built-in node monitors will silently refuse to schedule work if free disk space drops below a threshold (default 1GB). My root volume was undersized (6.7GB), so I resized the EBS volume to 20GB — but AWS resizing the volume isn't enough on its own. You also have to extend the partition and filesystem to actually use the new space:

sudo growpart /dev/nvme0n1 1
sudo resize2fs /dev/nvme0n1p1

A separate but related gotcha: on a small instance, /tmp is a tmpfs sized off available RAM — meaning it can be smaller than Jenkins' default 1GB threshold even when completely empty. That one just needed the threshold itself lowered in Jenkins' node monitor config, since no amount of cleanup would ever satisfy a 1GB requirement on a sub-1GB partition.

4. waitForQualityGate() timed out — but SonarCloud had already passed

This one took the longest to figure out. The standard approach for waiting on a SonarQube/SonarCloud quality gate result relies on a webhook — SonarCloud calling back into Jenkins once analysis finishes. I configured everything correctly, but it kept timing out at 10 minutes, even though checking SonarCloud directly showed the analysis had completed successfully.

Turned out: webhooks are a paid-plan-only feature on SonarCloud. The free tier can't call back into Jenkins at all — my analysis was finishing fine, Jenkins just had no way of being told.

The fix: replace the webhook-dependent wait with active polling instead. After the scan runs, SonarCloud writes a report-task.txt file containing a task ID. Instead of waiting to be notified, Jenkins repeatedly asks SonarCloud's API "is this task done yet?" using that ID, then separately checks the quality gate status once the task completes:

def taskFile = readFile('.scannerwork/report-task.txt')
def ceTaskId = (taskFile =~ /ceTaskId=(.+)/)[0][1]

waitUntil {
    def response = sh(
        script: "curl -s -u \${SONAR_TOKEN}: https://sonarcloud.io/api/ce/task?id=${ceTaskId}",
        returnStdout: true
    ).trim()
    def status = (response =~ /"status":"(\w+)"/)[0][1]
    return status == 'SUCCESS' || status == 'FAILED'
}

Same end result as the webhook approach, just Jenkins asking instead of SonarCloud telling.

5. Real code quality findings

Once the pipeline could actually reach the quality gate, SonarCloud flagged genuine issues worth fixing:

• Missing --ignore-scripts on npm install — without it, any dependency can run arbitrary code during install via lifecycle scripts. A known supply-chain attack vector.
• No lockfile — package.json alone (with ^ version ranges) doesn't guarantee reproducible installs. Added package-lock.json and switched to npm ci.
• Container running as root — the node base image defaults to root. Added USER node in the final Docker stage so the running container has minimal privileges.
• Express's X-Powered-By header — quietly discloses framework info to anyone inspecting response headers. One line to disable it: app.disable('x-powered-by').
• 0% test coverage — SonarCloud's default gate requires 80% coverage on new code. Added a couple of Jest/Supertest tests, and used /* istanbul ignore next */ to exclude the untestable server-bootstrap block from the coverage calculation (since it only runs in node index.js, not when the app is imported as a module for testing).

6. SonarCloud reported 52 issues on a ~30-line app

This one was almost funny — turned out SonarCloud was scanning node_modules/ by default, analyzing every dependency's source code as if it were mine. Adding an exclusions flag fixed it immediately:

-Dsonar.exclusions=node_modules/**,coverage/**

7. Case sensitivity bit me — twice

macOS's filesystem is case-insensitive by default, but Git, Docker, and Jest all are case-sensitive. I ended up with files like Jenkinsfile (correct) sitting next to jenkinsfile (what Jenkins was actually looking for and couldn't find), and Index.JS instead of index.js. The fix each time was git mv oldname.js newname-temp.js followed by git mv newname-temp.js newname.js — a two-step rename that forces Git to actually register the case change instead of assuming nothing happened.

8. Docker Hub push failed — twice, for two different reasons

First: ERROR: Could not find credentials entry with ID 'dockerhub-creds' — a simple mismatch between the credential ID in my Jenkinsfile and what I'd actually named it in Jenkins (Dockerhub-credentials vs dockerhub-creds). IDs are exact-match, case-sensitive.

Second, after fixing that: authentication required - access token has insufficient scopes. My Docker Hub access token had been generated with Read-only permissions instead of Read & Write. Regenerated it with the correct scope, updated the Jenkins credential, and the push finally succeeded.

What I'd tell someone starting this from scratch

• Check your tool's current system requirements before installing anything — Java, Node, Docker versions all move faster than tutorials get updated.
• If you're on a small/free-tier VM, budget real time for disk and memory tuning — it will come up.
• Read error messages literally before assuming you know what's wrong. Several of these ("Free Temp Space" warning, the webhook timeout) looked like one kind of problem but were actually something else entirely.
• Free-tier SaaS tools (SonarCloud, in this case) sometimes gate features you'd assume are standard. Always check what your specific plan actually supports before building your pipeline around an assumption.

The final result

A working pipeline that checks out code, runs tests, gates on code quality, scans for known vulnerabilities (twice — before and after the build), and only then ships an image to Docker Hub. Every stage failing for a real reason, at some point, along the way.

Repo: github.com/AnitaAliCloud/devsecops-pipeline
Image: hub.docker.com/r/anitaalicloud/devsecops-pipeline

---

If you're building something similar and hit one of these exact errors, hopefully this saves you the hour (or several) it took me to figure out.