DEV Community: huangchengsir

Should your pipeline live on a canvas, or in your repo? Two paradigms for self-hosted CI/CD

huangchengsir — Wed, 08 Jul 2026 14:54:48 +0000

Self-host a CI/CD tool long enough and you'll hit a philosophical fork: where should the pipeline definition actually live?

One camp is the canvas crowd: open a web page, drag a few nodes, draw a few edges — build → test → deploy, laid out visually, editable with a few clicks. Old-school Jenkins config and a lot of homegrown DevOps platforms work this way.

The other camp is the code crowd: the pipeline is just a YAML file in your repo — .github/workflows/*.yml, .gitlab-ci.yml, .pipewright.yml — committed, reviewed, and rolled back alongside your code. GitHub Actions made this camp mainstream.

Both have real upsides and real fatal flaws. When I built my own deploy tool I started with just the canvas, used it for a while, and hit a wall the canvas can't get around. Then I added YAML too — and promptly fell into the "how do two definitions coexist" pit. This post lays out the tradeoffs of each paradigm, and the one mechanism that's easiest to get wrong when you support both.

1. The canvas: the good and the painful

The visual canvas is genuinely nice:

Zero learning curve. No YAML fields to memorize. Drag and drop. New people are productive fast.
What you see is what you get. Stage dependencies (what runs after what, what runs in parallel) drawn as a graph are far more legible than a wall of indented YAML.
Change and run immediately. Tweak a parameter in the browser, hit run. No commit, no push.

But the canvas has two problems it can't escape:

The config isn't in version control. You changed the pipeline in the web UI — who changed it, what, and why? No record. Something breaks and you want "last week's version that worked"? Sorry, the canvas has no git log.
Config and code live apart. Code is in the repo; the pipeline is in a database. They live separate lives. Want a feature branch to temporarily tweak a build step? The canvas can't — it's a project-level config that doesn't follow branches. Change the canvas on main and your feature branches inherit it too.

Point 2 is the killer. Half the reason GitHub Actions is so pleasant is exactly this: the workflow file follows the branch. Change CI on a feature branch and it only affects that branch; main is untouched until you merge. The canvas can't give you that.

2. The code camp's answer: put the pipeline in `.pipewright.yml`

The code camp's fix is direct — the pipeline is a YAML file at the repo root. In my tool it looks like this:

version: 1
stages:
  - id: stg_src
    name: fetch-source
    kind: source
    jobs:
      - { id: job_src, name: source, type: git_source }

  - id: stg_build
    name: build
    kind: build
    needs: [stg_src]          # DAG dependency: build after source
    jobs:
      - id: job_compile
        name: compile
        type: script
        script:
          image: golang:1.24
          commands:
            - go build ./...

  - id: stg_deploy
    name: deploy
    kind: deploy_ssh
    needs: [stg_build]
    gate: true                # manual approval gate before deploy
    when:
      branches: [main]        # only deploy on main

One thing to draw a hard line around — a lot of people hear "YAML + git" and immediately think docker-compose. But these are different layers:

docker-compose.yml describes what the deployed artifact looks like: which services, ports, volumes, networks.
.pipewright.yml describes how the pipeline is orchestrated: fetch source, then build, then deploy — which stage depends on which, which needs manual approval, which only runs on main.

The former is the end state, the latter is the process. A deploy_ssh step inside .pipewright.yml can run docker-compose up, but that's one of its actions, not the same layer of thing. Don't conflate them.

Once the pipeline is in YAML, both canvas pains are cured: config is in version control (has git log, reviewable, rollback-able), and config follows the branch (a feature branch editing the YAML only affects itself).

3. The real problem: how do two paradigms coexist

Here, the lazy move is "pick one" — force users to choose all-canvas or all-YAML. But reality is: old projects already drew their pipeline on the canvas, and a new requirement wants YAML — you can't make people start over.

So the real question is: when a single project has both a canvas config and a repo YAML, which does one run actually obey?

My approach is a per-project switch (default off):

Switch off (default): 100% canvas config. Even if a .pipewright.yml exists in the repo, it's ignored — never read. Old projects behave identically.
Switch on: prefer the repo YAML. At run time, it reads .pipewright.yml from the repo root on the branch this run is for, parses it, and drives this run with it.

Three points, each worth spelling out:

1. This is "pick one," not "merge." A run's pipeline definition comes from exactly one source — either the repo YAML (repo) or the canvas config (stored). There's no blending the two. Blending is a disaster: you'd never be able to say what actually ran. Pick one, cleanly.

2. It reads from the run's branch — the YAML camp's biggest win. With the switch on, edit .pipewright.yml on a feature branch, push it, and that branch's runs use the new YAML, main unaffected. That's exactly the "config follows the branch" property the canvas couldn't give you.

3. On failure it silently falls back — the run never breaks. YAML file missing? Malformed? A binary blob? — it automatically falls back to the canvas config and the run proceeds. It won't wedge your pipeline just because the YAML has a typo. This is deliberate: pipeline-as-code is an enhancement, it shouldn't become a new single point of failure.

4. The fallback is a safety net — and a trap

"Silent fallback" sounds considerate, but it has a flip side: silent — it falls back without saying a word.

Picture this: you think the run is using the repo YAML, but the YAML has an indentation error, and the system quietly fell back to the old canvas config. The run goes green, and you believe the new YAML took effect — when it never even ran.

So I added a preview endpoint: before actually running, you can have the system pull .pipewright.yml from the repo, parse it, and tell you "found it or not (found) / is it valid (valid) / which stages it would generate." Edit the YAML, preview it, confirm it actually parses, then rely on it. Don't let "silent fallback" hide your config errors along with everything else.

The run detail view also carries a source badge: whether this run used "repo main · .pipewright.yml" or "web config," visible at a glance. If it says "web config" when you expected YAML, a fallback happened — go check your YAML.

5. Being honest: the boundaries

When you roll your own, you owe people the boundaries, so they don't use it on wrong assumptions:

No bidirectional sync. Editing the canvas doesn't generate YAML; editing YAML doesn't write back to the canvas. It's strictly one-way: switch on → repo YAML drives the run. Want to "export" the canvas as YAML? No such feature — hand-write it (or have AI draft it).
YAML changes require a push to take effect. It's "push code, run time reads it," not "save the YAML and hot-reload." To make new YAML take effect, trigger a new run.
YAML expresses a subset of the canvas's fields. The advanced "variables & cache" / "trigger config" tabs on the canvas aren't all covered by YAML yet. Core stages/jobs/dependencies/approval-gates/matrix/conditions are there, but don't expect 100% parity with the canvas.
Fixed file location: repo root, filename .pipewright.yml. No subdirectories or renames.
Source is a "record," not a "force." The badge tells you which source a run used, but you can't say "force this one run to use the canvas" — switching is per-project only, no run-level override.

These aren't bugs, they're "good-enough-first" tradeoffs. Spelling them out is how you know when to reach for the canvas and when for YAML.

6. So which should you pick?

My experience:

Canvas fits — just starting out, the pipeline is still churning, someone on the team doesn't write YAML, you want to click around and see results fast.
YAML fits — the pipeline has stabilized, you want it in version control and code review, you need "config follows the branch," you want changing the pipeline itself to go through PR review.

And the point of "support both, one switch per project" is: you don't have to pick a side up front. Get the flow working on the canvas first, then once it stabilizes, freeze it into .pipewright.yml, commit it, flip the switch, and migrate over smoothly — keeping the canvas config around as a fallback safety net. That's probably the least painful path for self-hosting.

7. I built all of this into a tool

Everything above — visual canvas orchestration, declarative .pipewright.yml, a per-project switch, reading YAML from the run's branch, silent fallback on error, a preview endpoint to validate, and the source badge in run detail — I packed it, along with the CI builds, multi-host deploy, automatic Caddy HTTPS, per-PR preview environments, and zero-downtime deploy + rollback from earlier posts, into my Go single-binary deploy tool Pipewright:

Both paradigms: web canvas drag-and-drop, or declarative repo .pipewright.yml, switchable per project;
Reads YAML from the run's branch, so a feature branch editing the pipeline only affects itself — config truly follows the branch;
Missing/invalid YAML falls back to canvas config automatically, so runs never break; a preview endpoint to validate before you rely on it;
A source badge in run detail so whether a run used repo YAML or web config is obvious at a glance.

Open source, MIT. Single binary, no runtime dependencies (frontend baked in via embed.FS, SQLite by default). Aimed at individual developers and small teams self-hosting.

Repo: https://github.com/huangchengsir/pipewright

But even if you never touch it, the core takeaway holds: canvas vs YAML isn't "which is more advanced," it's "which for which phase." And if you truly want both, the key is deciding that "one run obeys exactly one source," then using a single switch to migrate smoothly between them.

Issues and teardowns welcome.

Zero-downtime deploys and one-click rollback for self-hosted apps — no Kubernetes

huangchengsir — Wed, 01 Jul 2026 14:53:55 +0000

The most awkward moment in self-hosted deployment is the couple of seconds after you run docker stop old && docker run new — to the outside world, your service is a string of 502s. A user who clicks in during that window sees a big red error page.

Rollback is even messier. Something breaks in prod, and now you're frantically digging for "which image tag was the last good one?" or squinting at a list of SHAs in git log trying to guess which one was fine.

Small teams, homelabs, internal services — most don't have Kubernetes' rolling updates and one-click rollback. Are you just stuck tolerating that window? I used to think so. Then I actually took it apart, and the conclusion is: zero-downtime deployment isn't about K8s-grade scheduling at all. It's three small things — stage the new version first, flip to it with an atomic operation, and flip straight back if it's wrong. Here's how the pieces work.

1. The essence: compress "the switch" into one atomic action

First, see clearly why stop old → run new returns 502s: between those two commands there's a vacuum where the old one is gone and the new one isn't up yet. No matter how short the window, any request that lands in it fails.

The key mental shift is to split a deploy into two phases: "stage" and "switch".

Stage phase: put the complete new version next to the old one (a new directory / a new image). During this, the old version keeps serving, untouched — zero impact. It can be slow; nobody's affected.
Switch phase: point from old to new. This step must be as atomic as possible — ideally compressed into a single kernel-guaranteed operation, so the "vacuum" shrinks to nonexistent or a single instant.

Once you internalize that, the rest is just finding the "atomic switch action" for each artifact type.

2. File artifacts (jar / dist / static site): atomic symlink is the standard answer

If you deploy a jar, a frontend dist, or a static site, the standard solution is an atomic symlink switch:

releases/
  20260701-a1b2c3/   <- this deploy's new version
  20260630-f9e8d7/   <- previous version
current -> releases/20260701-a1b2c3   <- one symlink; the app only knows "current"

On deploy, you first upload the new version into releases/<new>/ while leaving the current symlink completely untouched. Once the new version is fully staged, you flip the symlink. There's an easy-to-miss detail in how you flip it:

# ❌ Has a window: ln -sfn is unlink + symlink, two steps; for one instant current doesn't exist
ln -sfn releases/new current

# ✅ Atomic: create a temp symlink, then mv -T does a single rename(2) overwrite
ln -sfn releases/new current.tmp && mv -T current.tmp current

mv -T is a single rename(2) syscall underneath, which the kernel guarantees is atomic — there is never an instant where current points at nothing. At any moment, current points either at the complete old version or the complete new one. No in-between state.

An honest but important caveat: an atomic symlink doesn't mean zero process restart. If your app is a long-running process (say a jar that loads into memory and stays there), after flipping the symlink you still have to send it a reload signal so it re-reads current. The symlink guarantees "current always points at a complete version" — it does not guarantee "the process never restarts." Don't conflate the two.

3. Containers: pull first, then swap — shrink the window to seconds

Containers don't have the symlink layer, but the idea is identical — stage first, then switch:

docker pull <new image>: pull the new image locally. During this, the old container keeps running — zero impact. While you're at it, docker inspect the old container's current image and record it as your rollback anchor.
docker rm -f <old> && docker run -d <new>: remove old, start new, two commands back-to-back to keep the window minimal.

The honest trade-off: between rm and the new container being fully ready, there's a brief "no container" window — usually seconds. For the vast majority of self-hosted scenarios, a few seconds is entirely acceptable. But don't pretend it's zero-window. To truly get zero-window you need blue-green: start the new container, health-check it, flip the reverse-proxy upstream from old to new, then remove the old one. The cost is running two copies during the switch, plus reverse-proxy coordination. Rolling (seconds-long window) vs. blue-green (zero window, double the resources) — pick by how critical the service is. Don't reflexively reach for the most complex option.

4. Where the health check goes: after the switch, roll back on failure

The new version is live — how do you know it didn't crash? Probe a health URI immediately after the switch, and roll back to the previous version the moment it's unhealthy.

One directional point to be clear about: this is "verify after switching," not "canary before switching." The new version is already serving the instant you switch; the health check exists to "catch a broken deploy and back it out fast," not to "gate release until it's verified good." If you want "gate until verified good," what you need is blue-green (verify health on the side, flip traffic only once it passes).

For a small team, "seconds-fast rollback after switch" is usually a better deal than "canary before switch" — far simpler to implement, and the only cost is a broken version being exposed for a few seconds. Your reverse proxy (e.g. Caddy) can add a second layer of passive health checking (health_uri + health_interval) that automatically ejects unhealthy upstreams from the load balancer — one more safety net.

5. One-click rollback: don't dig for image tags — have the tool "re-ship the last success"

The easiest way to get rollback wrong is to rely on human memory — "was the last good image tag v1.3.2 or v1.3.1?" People are least reliable exactly when something's on fire.

The right approach: the system already stores the artifact + the target-server list for every deploy. Rollback = find the "last deploy that fully succeeded across all servers" and re-ship its artifact to the same targets. The whole thing reuses the normal deploy path, so rollback itself runs the health check and can roll back again — it's not a special side-channel.

The honest trade-offs you need to accept:

One-click rollback = redeploying a historical version, so its time and cost ≈ a fresh deploy (re-pull the image / re-upload the files). It is not "instantly switch back to the old container." If you truly want a seconds-fast switch-back, you'd keep the old container around undeleted — a different trade-off (it sits there consuming resources).
It depends on history: you can only roll back to the "last success" among the recent N deploys. If you've deleted the old tag from your image registry, pull can't get it back and rollback fails. Don't rush to prune old images.

One real gotcha worth logging: to save display space, deploy records store the 7-char short SHA. But go-git's CommitObject API only accepts the 40-char full SHA — hand it a short SHA to look up a commit and you get "commit unreachable," and the feature silently degrades. The fix is to ResolveRevision first (turning a short SHA / branch name / tag into a full hash), then fetch the commit object. This "stored an abbreviation to save space, have to expand it at use time" trap is exactly the kind of thing you fall into when hand-rolling your own deploy tool. Note it down.

6. A few things that will bite you

Atomic symlink is only atomic for "the switch step"; whether the app needs a restart to pick up the new version is a separate concern — wire up a reload for that.
Container rm+run has a seconds-long window — don't advertise it as zero-downtime. For true zero-window, go blue-green and eat the double-resource cost.
The health check is post-switch verification — a failure means the broken version was already exposed for a moment. Fine? Use rolling. Not fine? Go blue-green.
A first-ever deploy has no "previous version" to roll back to — a health failure is a hard failure that needs manual intervention. Don't expect the first ship to auto-recover.
Rollback cost ≈ a fresh deploy, not a free instant switch. Treat it as "one automated redeploy," not an "undo button."

7. I ended up building this into a tool

None of the pieces are complex on their own, but you have to wire every one yourself: mv -T for atomic symlinks, pull-then-swap for containers, choosing blue-green vs. rolling, post-switch health checks, auto-rollback on failure, one-click re-ship of the last success, ResolveRevision for short SHAs... I packed all of this — along with the CI builds, multi-server deploys, Caddy auto-HTTPS, and Per-PR preview environments from earlier posts — into my single-binary Go deployment tool Pipewright:

File artifacts switch via atomic symlink; containers pull-then-swap with a recorded rollback anchor; blue-green / rolling both available as strategies;
runs a health check automatically after the switch and auto-rolls-back to the previous version if unhealthy;
one-click rollback: click once in the panel, it locates the last all-servers-success deploy and re-ships it — no memorizing image tags;
short SHAs in deploy records and code diffs are auto-resolved via ResolveRevision, so no "commit unreachable."

Open source, MIT, single binary, no runtime dependencies (frontend baked in via embed.FS, SQLite by default). Aimed at individual developers and small teams self-hosting.

Repo: https://github.com/huangchengsir/pipewright

But even if you never use it, the core takeaway holds: zero-downtime deploy = stage first + atomic switch + roll back after; one-click rollback = re-ship the last successful version. No Kubernetes required — self-hosting can have this experience too. It really isn't magic.

Issues and pushback welcome.

Per-PR preview environments for self-hosted apps aren't actually magic

huangchengsir — Fri, 26 Jun 2026 15:08:34 +0000

Anyone who's used Vercel or Netlify has been spoiled by one feature: open a PR, get a unique preview link automatically. Click it and you see exactly what that branch renders. Reviewers don't pull the code or spin up anything locally — they glance at a link and know whether the change is right. Merge or close the PR, and the environment quietly disappears.

Then you go back to the self-hosted world — your own backend, internal services, a small team's homegrown tooling — and that experience is suddenly gone. Want to review a frontend change? Ask for a screenshot, or git fetch it and run it yourself.

I used to think Per-PR preview environments were Vercel-grade platform magic that self-hosting just couldn't reach. Then I actually took one apart. The conclusion: it's really just three things — a dynamic subdomain, a temporary reverse-proxy route, and automatic cleanup. Here's how the pieces fit.

1. Strip the magic: what a preview environment actually is

Peel off the "platform magic" wrapper and a Per-PR preview environment only has to answer three questions:

Where does this PR's app run? Your CI already builds and deploys a container (or a port) for that branch. The preview doesn't need to spin up a second copy — reuse the artifact this deployment already produced.
How does it get its own entrypoint? Give it a dedicated subdomain, e.g. pr-128.preview.example.com, and point a reverse proxy at the container above.
Who cleans up after the PR is gone? Once the PR merges or closes, something has to reclaim that route and subdomain — otherwise after a few months you've accumulated a pile of zombie environments.

Once you see those three points, you'll notice everything from my earlier posts applies directly: the dynamic subdomain rides on a wildcard certificate (one cert for *.preview.example.com covers every PR subdomain — no per-PR issuance), and the route is just a single reverse_proxy. The rest is wiring them into an automatic flow.

2. Dynamic subdomains: the wildcard cert is the prerequisite

Without a wildcard cert, issuing a fresh certificate for pr-128.preview.example.com on every PR will instantly hit Let's Encrypt's rate limit (50 certs per registered domain per week). Open a few dozen PRs and you've burned through it.

The right move is to issue one wildcard cert for *.preview.example.com (via DNS-01 validation — covered in the last post), and every pr-N.preview.example.com reuses it with zero extra issuance. The Caddy config looks like this:

*.preview.example.com {
    tls {
        dns cloudflare {env.CF_API_TOKEN}
    }
    # dynamic routing table: pr-128 -> 10.0.0.5:auto, written by your program
    reverse_proxy {
        to {http.reverse_proxy.upstream}
    }
}

In a real setup that upstream isn't hardcoded — your deploy program writes a route dynamically the moment a PR deployment succeeds: pr-128.preview.example.com → the container IP:port this deploy produced.

3. Wire it to the "deploy succeeded" event

The most natural trigger for a preview environment isn't "PR opened" — it's "this PR's code deployed successfully." The reasoning is practical: when a PR is first opened the code might not even compile, so spinning up an environment is wasted. Once CI actually builds and deploys that branch, the artifact is already running — allocating a subdomain that points at it is nearly free.

So the full chain is:

PR pushed → CI builds → deploy succeeds (container is running)
                              │
                              ▼
              terminal hook: does this run belong to a PR?
                  ├─ parse PR number from branch name (pr-128 / pull/128 / bare digits)
                  ├─ is preview enabled for this project?
                  └─ find the container/port this deploy produced
                              │
                              ▼
              write a route pr-128.preview.example.com → container
                  record a preview-env row (PR number, subdomain, status=active)

There's an honest but important design tradeoff here: a run (one deployment) usually doesn't carry the PR number directly. The cleanest engineering move isn't to rip through the whole deploy pipeline stuffing in a PR field — it's to parse it from the branch name by convention: pr-128, pull/128, even a bare-digit branch 128, all parse best-effort; if it doesn't parse, silently skip, and never disrupt a normal deploy. Likewise, the upstream container reuses the project's existing outbound route on the target host rather than inventing a new one. Reuse existing information instead of fabricating data — that's the key to staying out of trouble.

4. Auto-cleanup: the real cure for zombie environments

Spinning up is easy; the hard part is tearing down. Manual cleanup will be forgotten, and a few weeks later your reverse proxy is full of dead routes from long-merged PRs.

The reliable approach is a periodic sweeper: walk all active preview environments on an interval and re-check the corresponding PR's status —

PR merged / closed → delete the route, mark the env reclaimed;
PR still open → leave it.

Make reclamation a reconciliation-style periodic check rather than relying on a precise "PR closed" event, because webhooks get dropped and arrive out of order, while periodic reconciliation is idempotent: miss it this round, catch it next round, never pile up. This is one of the most robust patterns in self-hosted land — prefer periodic reconciliation over betting on a one-shot event.

5. A few things that'll bite you

Subdomain depth: *.preview.example.com only covers one level. Don't use pr-128.app.preview.example.com (two levels) — the wildcard cert won't cover it. Flatten preview subdomains to one level.
Data isolation: previews run real containers. If one points at your production database, a reviewer clicking around in the preview could mutate prod data. Previews should hit a separate test DB, or be explicitly read-only. This matters more than the technical implementation.
Don't let previews eat all your resources: one container per PR means memory blows up as PRs pile on. Add a cap (e.g. at most N concurrent previews) or a per-preview quota.
Reclamation must be idempotent: confirm a route still exists before deleting it; don't abort the whole sweeper because "it was already deleted."

6. I eventually built this into a tool

None of the pieces is complex on its own, but you have to wire every link yourself: the wildcard cert, the dynamic route writes, PR-number parsing, the terminal hook, the cleanup sweeper — and you have to guarantee that "allocating a preview" can never drag down a normal deploy (any step failing has to degrade gracefully and skip silently). I packed all of this, alongside the CI builds, multi-host deploys, and Caddy auto-HTTPS from earlier posts, into my Go single-binary deploy tool Pipewright:

A per-project preview toggle; once on, a successful PR deploy auto-allocates a pr-<n> subdomain — no hand-written routes;
Fully best-effort: can't parse a PR number, no DNS configured, no container found → silently skip, never block the deploy;
A preview-env panel showing which PRs have live previews, their subdomains, and status at a glance;
A periodic sweeper that reclaims by real PR status, so merged/closed environments need no manual cleanup.

It's essentially the "dynamic subdomain + temporary reverse proxy + auto cleanup" from this post, productized to skip the hand-rolling. MIT-licensed, single binary, no runtime deps (frontend baked in via embed.FS, SQLite by default), aimed at individual devs / small teams self-hosting.

Repo: https://github.com/huangchengsir/pipewright

But even if you never touch it, the takeaway stands: Per-PR preview environments aren't out-of-reach platform magic for self-hosting — they're a wildcard cert + a dynamic reverse-proxy route + a periodic reclaimer. Wire those three together and your self-hosted setup gets the "one link per PR" experience too.

Issues and pushback welcome.

Stop hand-rolling nginx + certbot for your self-hosted HTTPS

huangchengsir — Mon, 22 Jun 2026 14:56:32 +0000

If you've done any self-hosting - a homelab, internal services, a small team's own tooling - you've probably been through this ritual:

A service is running, you want to put a domain and HTTPS in front of it. So you install nginx, write a reverse-proxy config, install certbot, request a Let's Encrypt cert, and wire up a cron job to renew it. Three months later, the day the cert expires, you discover the cron path was wrong, or certbot's webroot mode is fighting your nginx config, and the site is a wall of red.

The third time I went through this, I decided to replace the whole thing. The short version: for self-hosted HTTPS, Caddy + DNS-01 gets you to "set it once, never touch it again." Here's the part that's actually worth knowing.

1. What Caddy's automatic HTTPS actually saves you

Caddy's most underrated feature is that the entire ACME flow is built in. No separate certbot, no renewal cron. A whole Caddyfile can be this short:

app.example.com {
    reverse_proxy localhost:8080
}

Three lines. On startup Caddy will, on its own:

Request a cert for app.example.com from Let's Encrypt;
Store it locally and renew it automatically before expiry;
Redirect port 80 to 443 with sane modern TLS.

Compared to nginx + certbot, what you delete is: the cert-request script, the renewal cron, the redirect rules, the TLS tuning. nginx can do all of this - it just makes you assemble every piece by hand, while Caddy ships "the correct defaults" as factory settings.

2. The real killer: DNS-01 gives certs to internal services too

The above is HTTP-01 validation: Let's Encrypt calls back to your port 80 to confirm you own the domain. The problem - what if your service is on a private network? The public internet can't reach your port 80, so HTTP-01 is simply out. This is the most common wall in self-hosting: a NAS, an internal admin panel, a homelab box - you want HTTPS but you're stuck at validation.

The fix is DNS-01 validation: instead of checking a port, it checks whether you can edit the domain's DNS records. Caddy uses your DNS provider's API (Cloudflare, Alibaba DNS, DNSPod, etc.) to write a TXT record automatically and complete validation. No inbound port ever has to be exposed to the public internet.

Better still, DNS-01 supports wildcard certs (*.example.com). Get one wildcard cert and every internal subdomain - grafana.lab.example.com, nas.lab.example.com, pad.lab.example.com - is covered by a single cert; new services don't need a fresh request:

*.lab.example.com {
    tls {
        dns cloudflare {env.CF_API_TOKEN}
    }
    @grafana host grafana.lab.example.com
    handle @grafana {
        reverse_proxy localhost:3000
    }
    @nas host nas.lab.example.com
    handle @nas {
        reverse_proxy localhost:5000
    }
}

Note the official Caddy binary does not ship DNS plugins - you have to rebuild Caddy with the plugin for your provider (xcaddy build --with github.com/caddy-dns/cloudflare). This is where a lot of people get stuck: the official image throws "provider not found" on DNS-01, and you need to xcaddy your own build or use an image that bundles the plugin.

3. The things that will bite you

Once it's running, a few things you only learn by stepping on them:

1. Wildcard cert scope

*.example.com only covers one level of subdomain, not a.b.example.com (two levels). For two levels you need a separate *.b.example.com. Plan your subdomain hierarchy with this in mind.

2. DNS API token scope

The DNS token you hand Caddy should be scoped to edit TXT records on that one zone only - don't take the lazy path and give it an account-wide token. That token is partial control of your domain; leaked, it can be used to sign certs for you.

3. Let's Encrypt rate limits

50 certs per registered domain per week. Normal use won't hit it, but if you script container rebuilds that re-request a cert every time, you can easily get throttled until next week. Always persist the cert directory (mount a volume) so it isn't re-requested on every restart.

4. Don't forget WebSocket / gRPC

Reverse-proxying plain HTTP is fine, but if you later put a WebSocket service behind it (lots of admin panels stream live logs over WS), make sure the proxy forwards the Upgrade header correctly. Caddy's reverse_proxy does this by default, but it's a classic faceplant if you're hand-writing nginx config.

4. And then I packaged this too

I understood all of it, but rebuilding a plugin-enabled Caddy, writing a Caddyfile, configuring a DNS token, and tracking which subdomain points to which backend - for every new machine and every new service - still got old. So I folded it, along with the CI builds, multi-host deploys, and container management from before, into my single-binary Go deploy tool, Pipewright:

A bundled self-built Caddy image (with the major DNS plugins), so DNS-01 works out of the box;
Bind a domain, set up a wildcard cert, configure path routing - all from a few clicks in the web panel, no hand-written Caddyfile;
A certificate overview panel: every domain's cert status and expiry at a glance, no more openssl s_client one at a time;
And the feature I personally wanted most: per-PR preview environments - every PR spins up its own environment on a dedicated subdomain, torn down automatically when it's merged or closed. The "every PR gets a preview link" Vercel experience, self-hosted.

It's basically the "Caddy auto-HTTPS + DNS-01 wildcard" from this post, productized to skip the hand-rolling. MIT-licensed, single binary, no runtime deps (frontend baked in with embed.FS, SQLite by default), aimed at individual developers and small teams.

Repo: https://github.com/huangchengsir/pipewright

But even if you never touch it, the core of this post stands: stop hand-rolling nginx + certbot for self-hosted HTTPS. Caddy + DNS-01 + a wildcard cert - configure once and forget it exists.

Issues and pushback welcome.

You probably don't need ArgoCD - good-enough GitOps with git and docker compose

huangchengsir — Thu, 18 Jun 2026 13:31:20 +0000

Every time someone starts self-hosting - a homelab, a few internal services for a small team - they tend to fall down the same rabbit hole: should I run K8s? And if I run K8s, do I need ArgoCD or Flux for GitOps? Two weeks later they've read a pile of Helm charts and CRDs and still haven't deployed a single service.

Let me say the quiet part out loud: for a single host, or three-to-five machines, you do not need ArgoCD. The part of GitOps you actually want - "the git repo is the source of truth, changes deploy themselves, I can roll back, and there's a record" - you can get 90% of it with git + docker compose and about twenty lines of script, with no control plane to babysit.

What "good-enough GitOps" actually is

GitOps boils down to two things:

Declarative: the desired state lives in git (for self-hosting, that's your docker-compose.yml).
Pull-based: the machine pulls changes and reconciles itself, instead of you SSHing in to type commands.

On one docker host, the minimal version looks like this: a webhook (or a 60-second git fetch) notices the tracked branch moved, and runs:

git pull --ff-only
docker compose pull
docker compose up -d --remove-orphans

--remove-orphans matters: services you delete from the compose file actually get stopped, instead of lingering forever. That's it - you now have a working GitOps loop: edit compose -> push -> the machine reconciles itself.

The two things that will actually bite you

The good-enough version runs, but there are two traps anyone who's done this knows:

1. Zero-downtime

Plain docker compose up -d recreates the container - old one stops, new one starts, with a few seconds of gap in between. Fine for background jobs, but for anything user-facing those few seconds are a 502.

compose up --wait (v2.17+) at least waits for the healthcheck before calling the deploy a success, which catches the "starts then crashes" case;
for true zero-downtime you end up running two service names (blue/green) behind a reverse proxy, flipping traffic once the new one is healthy, then stopping the old.

Don't expect compose to do graceful rolling updates for you. It doesn't.

2. Rollback

This is the one most people skip and the one that hurts most. If your image tag is :latest, then "rollback" means "edit the file and pray" - you have no idea which version last worked.

The fix: pin image tags to the git commit SHA (myapp:9f8322f, not myapp:latest), and record which SHA is currently live. Now rollback degrades to "re-deploy the previous SHA" - deterministic and repeatable. Without it, your GitOps quietly turns into "git pull and hope".

When you should reach for a heavier tool

The boundary is clear:

One machine: the twenty-line script is genuinely fine. Don't overthink it.
3+ machines, or you need an audit trail of who deployed what, when: this is where hand-rolled scripts start accruing debt - you need concurrent multi-host deploys, unified rollback, a record of who triggered it. That's when a thicker tool starts paying for itself.

But note: that threshold is much later than most people assume. For the vast majority of self-hosting, you're still on the "twenty-line script is fine" side.

I eventually packaged this up

I understood all of the above, and still got tired of re-typing the webhook + pull + compose + SHA-pinning + rollback-record dance every time I set up a new machine. So I bundled it - along with the CI build step before it, agentless multi-host SSH deploys, and container management - into a single Go binary called Pipewright: scp one file to a server, run it, open the browser, and you get a visual pipeline + one-click deploy + a container panel, with no other runtime dependencies (the Vue frontend is compiled into the binary via embed.FS, SQLite by default).

It's basically the "good-enough GitOps" above, productized, with the zero-downtime and SHA-based rollback gaps filled in. MIT licensed, aimed at solo devs and small teams - not trying to replace GitLab for a 200-engineer org.

Repo: https://github.com/huangchengsir/pipewright

But even if you never touch it, the takeaway stands: for self-hosted GitOps, you probably don't need ArgoCD. Start with git + compose, and only add weight when you actually hit the wall.

Issues and pushback welcome.

I built a self-hosted CI/CD + deploy + ops platform that fits in one Go binary

huangchengsir — Tue, 16 Jun 2026 15:51:21 +0000

I run a handful of small VPS boxes for side projects. For a while my "deploy setup" was the usual trio: a CI runner (Jenkins, then Drone), something to push releases to servers (Ansible / Kamal scripts), and Portainer to actually see what was running and poke at containers.

On a 2GB box, wiring and babysitting three separate tools was more overhead than the apps I was shipping. So a few weeks ago I started building the thing I actually wanted: CI/CD + multi-server deployment + basic ops, in a single Go binary with zero runtime dependencies.

It's called Pipewright. You scp one file to a server, run it, open the browser — that's the whole install.

What it actually does

Visual pipeline builder — a two-level (stage → job) DAG canvas. Horizontal links run serially, vertical ones run in true parallel. It round-trips with .pipewright.yml, so you can click or commit your pipeline.
Isolated builds — version-pinned, container-isolated build steps with dependency caching; artifacts (images / JARs / dist bundles) you can push to a private registry.
Agentless deploys over SSH — zero-downtime switchover + rollback on failure, fan-out to multiple servers with partial-failure visibility.
Server & container ops — manage containers / images / stacks / volumes / networks across hosts, live stats, logs, and an in-browser terminal.
It can update itself — checks GitHub for the latest release and does a download → checksum verify → atomic replace → restart, from the UI.

The stack

Go backend, Vue 3 frontend, everything embedded into the one binary. SQLite by default (zero setup), MySQL if you want it. Auth is argon2id + CSRF, credentials live in an encrypted vault (never echoed back in plaintext), and there's an append-only audit log. ~50MB binary, runs comfortably on a tiny box.

What I learned building it

Embedding the whole frontend into the Go binary (embed.FS) is underrated. One artifact, no nginx, no separate static host.
Self-update is a great forcing function. The moment your tool updates itself in production, you stop being sloppy about release artifacts, checksums, and atomic file replacement. I've been dogfooding Pipewright to ship its own releases.
"One binary for all of it" is a real design tension. It's tempting to keep adding surface area. I keep reminding myself the goal is the opposite of Jenkins-at-scale: the lightest thing that still does the full build → deploy → watch loop for a few servers.

What it's NOT

It's not trying to replace GitLab/Jenkins for a 200-engineer org. No multi-tenant RBAC, no massive plugin ecosystem. If you're running a big org, use the big tools. If you're a solo dev or small team who just wants build → deploy → ops without standing up three services, that's exactly who I built it for.

It's early (a few weeks old) and I'm actively building. I'd genuinely love feedback — especially on whether the "all-in-one binary" framing makes sense to you, or whether it's doing too much.

Repo (MIT, screenshots + one-command quickstart): https://github.com/huangchengsir/pipewright

Thanks for reading 🙏

DEV Community: huangchengsir

Should your pipeline live on a canvas, or in your repo? Two paradigms for self-hosted CI/CD

1. The canvas: the good and the painful

2. The code camp's answer: put the pipeline in .pipewright.yml

3. The real problem: how do two paradigms coexist

4. The fallback is a safety net — and a trap

5. Being honest: the boundaries

6. So which should you pick?

7. I built all of this into a tool

Zero-downtime deploys and one-click rollback for self-hosted apps — no Kubernetes

1. The essence: compress "the switch" into one atomic action

2. File artifacts (jar / dist / static site): atomic symlink is the standard answer

3. Containers: pull first, then swap — shrink the window to seconds

4. Where the health check goes: after the switch, roll back on failure

5. One-click rollback: don't dig for image tags — have the tool "re-ship the last success"

6. A few things that will bite you

7. I ended up building this into a tool

Per-PR preview environments for self-hosted apps aren't actually magic

1. Strip the magic: what a preview environment actually is

2. Dynamic subdomains: the wildcard cert is the prerequisite

3. Wire it to the "deploy succeeded" event

4. Auto-cleanup: the real cure for zombie environments

5. A few things that'll bite you

6. I eventually built this into a tool

Stop hand-rolling nginx + certbot for your self-hosted HTTPS

1. What Caddy's automatic HTTPS actually saves you

2. The real killer: DNS-01 gives certs to internal services too

3. The things that will bite you

1. Wildcard cert scope

2. DNS API token scope

3. Let's Encrypt rate limits

4. Don't forget WebSocket / gRPC

4. And then I packaged this too

You probably don't need ArgoCD - good-enough GitOps with git and docker compose

What "good-enough GitOps" actually is

The two things that will actually bite you

1. Zero-downtime

2. Rollback

When you should reach for a heavier tool

I eventually packaged this up

I built a self-hosted CI/CD + deploy + ops platform that fits in one Go binary

What it actually does

The stack

What I learned building it

What it's NOT

2. The code camp's answer: put the pipeline in `.pipewright.yml`