DEV Community

Zero Heartbeat
Zero Heartbeat

Posted on Edited on Originally published at delta1labs.com

Back up and recover a self-hosted Keyright instance

A license server earns its keep by being there at the exact moment a customer activates. That makes its data precious: lose the record of who is licensed, which seats are claimed, and how many credits remain, and you are not just down — you have lost the truth your customers' installs depend on. So backup and disaster recovery for self-hosted Keyright deserve the same deliberate design you gave high availability. The good news is that the same architecture that makes HA easy makes DR simple too.

Key takeaway: Self-hosted Keyright is a stateless app whose entire state lives in your PostgreSQL, so DR is really Postgres backup/restore plus safeguarding two secrets you generate. Back up the database for all your state; back up your KEYRIGHT_KEK and KEYRIGHT_SIGNING_KEY separately and securely — because without the KEK, the encrypted signing keys in the database cannot be recovered.

What actually needs backing up

Keyright has exactly two moving parts: a stateless app, and your database. That split is the whole backup story.

The app is the ghcr.io/delta1-labs/keyright image — one ASP.NET Core service on port 8080, with no database inside it. It keeps no state between requests. So there is nothing in the app container to back up: it is reconstructed from the published image plus your configuration and environment variables. If a node, a VM, or a whole cluster disappears, you redeploy the same image and it comes back identical. Treat the app tier as cattle, not pets.

Everything that matters lives in PostgreSQL. Licenses, activations and seats, credit balances and their ledger, the audit log, webhook and notification state, your tenant and product configuration — and the signing keys, stored encrypted at rest. Point the app at it with the KEYRIGHT_DB connection string and it creates and migrates its own schema. Because all durable state concentrates in one database, backing up Keyright is backing up that database. There is no scattered file state, no node-local cache of truth, nothing on a disk somewhere that a Postgres backup would miss.

The two secrets a database backup does not contain

Here is the part that catches people, and the single most important thing in this post. Two secrets are yours — you generate them, Keyright never sees them — and they are not reconstructable from a database backup:

  • KEYRIGHT_KEK — a base64 32-byte key-encryption-key. It encrypts your signing keys at rest, which is exactly why those keys can sit safely in the database. The flip side: without the KEK, the encrypted signing keys in your database cannot be recovered. A perfect database snapshot plus a lost KEK equals an instance that can no longer decrypt its own signing keys — it can restore every row and still be unable to sign a single lease with the original key. Every client that pinned the matching public key would then reject new leases.
  • KEYRIGHT_SIGNING_KEY — the base64 PKCS#8 RSA private key that signs offline leases and air-gapped credit blocks, and whose public half your customers' apps embed. If you set it explicitly (recommended), it is part of your configuration, not something you can regenerate without re-pinning public keys in every shipped client.

So the rule, straight from the configuration reference: back up the KEK and the signing key separately from the database and securely — in a secrets manager with its own backup, an offline vault, wherever your organization keeps root-level secrets. Never in the database you are protecting, never in the image, never in source control. A database backup and the KEK are two halves of one recoverable backup; keep them in different places, but keep them both.

Backing up the database

Everything is in PostgreSQL, so use standard Postgres tooling. For most deployments a logical dump on a schedule is the simplest correct approach:

pg_dump --format=custom --no-owner \
  "host=db.internal.example.com port=5432 dbname=keyright user=keyright_app sslmode=require" \
  --file "keyright-$(date +%F).dump"
Enter fullscreen mode Exit fullscreen mode

The custom format restores cleanly with pg_restore and compresses well. Run it on a timer (cron, a sidecar job, your managed scheduler), copy each dump off-box to object storage, and apply a retention policy. Verify the file is non-empty and restorable — an untested backup is a hope, not a plan.

For a larger instance, or a tighter RPO, add physical backups with point-in-time recovery. Take a base backup and archive WAL so you can roll forward to any moment:

pg_basebackup --pgdata=/backup/base --format=tar --gzip \
  --wal-method=stream --checkpoint=fast
Enter fullscreen mode Exit fullscreen mode

With WAL archiving configured, PITR lets you recover to a chosen timestamp — invaluable if a bad migration or an operator mistake needs to be undone to the second before it happened.

If you run managed Postgres — which, for anything past evaluation, you should, with a primary-plus-standby topology — lean on its automated snapshots and backup retention. A daily (or more frequent) snapshot plus the provider's PITR window typically gives you a stronger RPO than hand-rolled dumps, and restoring is a console or CLI operation. Either way, the principle is the same: the database is where every byte of state lives, so the database is what you invest in protecting. Do not rely on the bundled evaluation Postgres the kits offer — it is for trying Keyright on a laptop, not for production DR.

RPO and RTO for a license server

Think about the two numbers explicitly, because a license server has an unusually forgiving failure profile.

Your RPO (how much data you can afford to lose) is set by your backup cadence. A nightly pg_dump risks losing up to a day of activations, credit spends and audit records; continuous WAL archiving or frequent managed snapshots narrow that to minutes. Pick the cadence against how much activation and ledger history you are willing to replay or lose.

Your RTO (how long recovery takes) is where the architecture pays off. Because clients cache a signed lease and validate offline between activations, an outage does not lock anyone out the instant it starts — their grace windows absorb it. You are racing the longest client grace window, not a stopwatch on every app launch. That turns "restore a database and redeploy a stateless container" from a frantic seconds-level scramble into a measured operation you can get right. (And if you also run multi-node HA, routine failures never become a recovery event at all.)

The restore drill

Practice this before you need it. A restore has four steps:

  1. Restore the database. From a logical dump, into a fresh empty database:
   pg_restore --no-owner --clean --if-exists \
     --dbname="host=db.internal.example.com port=5432 dbname=keyright user=keyright_app sslmode=require" \
     keyright-2026-11-04.dump
Enter fullscreen mode Exit fullscreen mode

From a managed snapshot or PITR, restore to the point you want through your provider, and note the new connection string.

  1. Redeploy the stateless app pointed at the restored database via KEYRIGHT_DB. Supply the same KEYRIGHT_KEK and KEYRIGHT_SIGNING_KEY from your separate secret backup — this is the moment the KEK earns its keep, decrypting the signing keys the restore brought back. Migrations apply automatically on startup, so there is no manual SQL.

  2. Install the self-license key. Keyright is fail-closed: it needs a valid self-license (or trial) key to serve any traffic at all. Set KEYRIGHT_SELF_LICENSE (the same key runs on every node) and start the app. Without it, every request returns 402 no matter how clean the database restore was.

  3. Verify. Two checks confirm a good restore:

    • GET /health/ready returns 200 — the app is up and reached the restored database (it returns 503 while Postgres is unreachable).
    • GET /admin/self-license reports your edition and an active phase — the self-license took and the instance is serving, not gated.

Then smoke-test a real /v1/validate against a known license to confirm the restored signing key still produces leases your clients accept.

If all four pass, you are back on the activation hot path with your state intact.

Fold it into operations

Back up the database on a cadence that matches your RPO, store the KEK and signing key where you will not lose them, and rehearse the restore so the first time you run it is not during an incident. The operations guide has the backup and upgrade detail, and because Keyright is one product in both deployment modes, the same stateless-app-plus-your-database shape holds whether you run one node or three. Self-hosted Keyright is priced per deployment across Standard, Pro and Enterprise — see the pricing — but DR discipline is the same at every tier: protect the database, protect the two secrets, and practice getting both back.

Top comments (0)