DEV Community

Cover image for Everything reported success: five silent failures self-hosting a social scheduler
Zain Amjed
Zain Amjed

Posted on

Everything reported success: five silent failures self-hosting a social scheduler

The API returned {"register":true}. The user row was in Postgres. The login screen kept coming back.

That was the third thing to fail silently in one afternoon of standing up Postiz on a home-lab VM, and the pattern held every time. The component that broke was also the component reporting it was fine.

A default route that was never installed

Fresh Ubuntu 26.04 VM. SSH in from the LAN, works. Ping the box, works. Run apt-get update, and every mirror times out.

The Subiquity installer had written this into /etc/netplan/00-installer-config.yaml:

routes:
  - to: default
    via: 192.168.21.254
Enter fullscreen mode Exit fullscreen mode

The gateway on that segment is 192.168.18.254. One digit. Because the next-hop was off-link, systemd-networkd could not install the route at all, so the box ended up with no default route rather than a wrong one:

$ ip route
192.168.18.0/24 dev ens192 proto kernel scope link src 192.168.18.96
Enter fullscreen mode Exit fullscreen mode

Nothing logged an error. LAN traffic worked, which is why SSH worked, which is why the machine looked healthy while every package operation failed.

ip route | grep default is now the first thing I run on a new VM, before blaming a mirror or DNS.

Postiz is not one container

The obvious docker run crash-looped:

Error code: P1012
error: Environment variable not found: DATABASE_URL.
Enter fullscreen mode Exit fullscreen mode

Postiz needs PostgreSQL and Redis, and current versions also need a Temporal cluster, which brings its own Postgres and an Elasticsearch. Eight containers, roughly 7 GB of images, serving on 4007 rather than the 3000 I had assumed.

Upstream maintains the compose file. Pull it from there instead of hand-rolling one, because the service list changes between releases.

The cookie that browsers throw away

Back to the signup that succeeded and did nothing. The response header:

Set-Cookie: auth=...; Domain=192.168.18.96; Path=/; Secure; SameSite=None
Enter fullscreen mode Exit fullscreen mode

Two independent problems in one line.

A cookie whose Domain attribute is a bare IP address is invalid under RFC 6265. Browsers drop it. And Secure cookies are only stored over HTTPS, which this was not.

So the account got created, the API returned 200, no cookie was kept, and the app bounced back to login with nothing in the backend log. From the server side it had worked perfectly.

Fixing it meant using a DNS name rather than an address, and putting real TLS in front.

LinkedIn will not let you skip its queue

I had assumed self-hosting meant using the tool's LinkedIn app. It does not. Self-hosted Postiz uses your own LinkedIn app, which means registering one and requesting its products yourself.

Three things I did not expect.

Every LinkedIn app must be associated with a Company Page, even when you only ever post to a personal profile. There is no personal-only app type. The Page is administrative ownership, not the posting target.

Postiz requests seven scopes and validates the grant client-side, rejecting a partial one. Four of those seven only arrive with the Advertising API product, which goes to human review. Until it lands, the connect fails on scope validation rather than on anything you can debug locally.

And LinkedIn exposes no API for publishing articles. w_member_social creates feed posts. The long-form publisher is UI-only, for everyone, so a native article stays a manual paste no matter what tool you run.

Google is strict in a different direction. Redirect URIs must use HTTPS unless the host is localhost, hosts cannot be raw IP addresses, and the TLD must be on the public suffix list. A .local name is rejected outright, which ruled out the internal hostname the box already had.

Cloudflare strips private addresses

With a certificate issued and the app on a real name, the record still would not resolve on the LAN.

The A record points at 192.168.18.96, a private address. Cloudflare's 1.1.1.1 strips RFC1918 answers as rebind protection, and both internal resolvers forward to it. 8.8.8.8 returned the record fine.

A local override on the internal DNS servers fixed it. Worth knowing before you conclude the record never propagated.

Docker said healthy, the backend was not

Last one, and the reason a login failure sent me looking in the wrong place.

The container healthcheck probes the frontend on port 5000. The backend runs separately on 3000. After a restart the backend wedged during startup, bound nothing, logged nothing, and never crashed, so pm2 reported it online and Docker reported the container healthy while every /api/* request returned 502.

$ docker inspect -f '{{.State.Health.Status}}' postiz
healthy
$ curl -s -o /dev/null -w '%{http_code}' http://127.0.0.1:4007/api/
502
Enter fullscreen mode Exit fullscreen mode

A docker compose restart cleared it in under a minute. Extending that healthcheck to probe the backend is on the list.

What I took from it

Four of these five reported success while broken. The netplan config parsed. The signup returned 200. The DNS record existed. The container was healthy.

Health signals that observe part of a system will tell you that part is fine, which is a narrower claim than it reads as. When a symptom and the monitoring disagree, check what the monitoring actually measures before trusting it over the symptom.

If you have hit the same thing self-hosting anything with OAuth in it, I would be interested to hear which one caught you.

AWS #DevOps #SelfHosting #Linux #OAuth

Top comments (0)