My Hosting Platform Runs on Two SQLite Files. Here's How I Back Them Up and Bring Them Back.

When people hear that the thing running my server keeps all its state in SQLite, they usually ask the same question. Isn't that a toy database? Shouldn't a hosting platform sit on Postgres, with a replica and a team that knows how to fail it over?

I went back and forth on this for a while. Where I ended up is that the database itself was never the scary part. The scary part is a question nobody likes to answer about their own setup. If this machine disappeared tonight, how long would it take to get everything back? With a couple of small files, the honest answer can be "a few minutes", and you can prove it by doing it.

This post walks through how agentic-hosting stores its state. It's an open-source tool I built that turns a Linux server into a place where AI agents can deploy apps and databases on their own. By the end you'll know where its data lives, how to take a safe backup while it's running, what else you need to copy, and how to restore everything on a fresh server.

⭐ If this is useful, a star on GitHub helps other people find it.

🤖 Give this to your agent: point it at https://raw.githubusercontent.com/dennisonbertram/agentic-hosting/main/llms.txt and it can sign up and set things up from that one link.

A few words first

agentic-hosting is one program, ah, that runs on your server. Agents talk to it over an HTTP API to deploy apps, add Postgres or Redis, and read logs. ah is the control plane. It doesn't run your apps itself. It keeps track of them and tells Docker what to do.

That record keeping is what this post is about. Every account, API key, app, build, environment variable, and database ah knows about is a row in SQLite, a database that lives in a single file on disk. There's no database server to install or keep running.

Where the data lives

On a standard install, everything sits in /var/lib/ah:

/var/lib/ah/
├── ah.db            # state: tenants, keys,
│                    # services, builds, env
├── ah-metering.db   # usage samples
├── master.key       # encryption key
└── backups/         # output of `ah backup`

There are two database files, not one, and I'll come back to why. You'll also see ah.db-wal and ah.db-shm next to them while ah is running. Those belong to SQLite, and they matter a lot for backups.

You can look inside with the regular sqlite3 tool. This lists the schema changes the database has been through:

sudo sqlite3 /var/lib/ah/ah.db \
  'SELECT name FROM schema_migrations;'

ah ships its schema as numbered SQL files built into the binary. On startup it applies any it hasn't recorded yet, each one in its own transaction along with the row that records it. Right now that's 26 files for the state database and 2 for metering. When you upgrade, the new binary applies its new files the first time it starts.

Why SQLite holds up here

Three settings do most of the work. ah opens both files like this (wrapped to fit):

sql.Open("sqlite3", path+
  "?_journal_mode=WAL"+
  "&_busy_timeout=5000"+
  "&_foreign_keys=on"+
  "&_txlock=immediate")

WAL mode. By default SQLite writes changes straight into the main file, and readers wait while that happens. In WAL (write-ahead log) mode, new writes go to the -wal file first and get folded into the main file later. Readers keep reading while a write is in progress. For a control plane, where an agent polling build status shouldn't stall another agent's deploy, this is the setting that matters most.

A busy timeout. SQLite still allows only one writer at a time. With _busy_timeout=5000, a second writer waits up to 5 seconds for its turn instead of failing at once.

Take the write lock up front. This one took me a while to understand. A normal SQLite transaction starts as a reader and asks for the write lock only when it first writes. If another connection committed in between, SQLite returns an error right away and the busy timeout never kicks in. _txlock=immediate makes every transaction ask for the write lock at the start, so it waits its turn like everyone else. The repo has a test for exactly this case.

ah also caps each file at 4 open connections, which is plenty for one server. And one server is the real limit of this design. Everything assumes one machine, and the docs list multi-node as a known gap. I'm fine with that. One machine with good backups is a setup I can understand at 2am.

Why two files

ah.db holds the things you'd be upset to lose: accounts, keys, apps, builds, encrypted environment variables, and database passwords. ah-metering.db holds usage. Every 60 seconds ah records a CPU and memory sample for each running app, in two tables, usage_events and usage_daily.

The way I think about it, those are different kinds of data. Metering writes on a timer, forever, and it grows with every app you run. State changes when something actually happens. Since SQLite allows one writer per file, putting the steady stream of samples in its own file means it never queues behind, or in front of, a deploy. The split also shows up in backups. If the metering backup fails, ah backup logs a warning and carries on. If the state backup fails, the whole run fails.

The code keeps the two apart on purpose. Each migration file has to start with state_ or metering_, and ah refuses to start if one doesn't, so a schema change can't end up in the wrong database.

Taking a backup while it's running

Here's the trap with WAL mode. Recent writes can sit in ah.db-wal for a while before they reach ah.db. If you cp ah.db somewhere, you can get a file that's missing recent changes, or one that's inconsistent.

SQLite has a command for this, VACUUM INTO. It reads the database through SQLite itself, WAL included, and writes a clean standalone copy to a new file. It works on a read-only connection, so the backup never takes the write lock away from the running server.

That's what ah backup does:

sudo -u ah /usr/local/bin/ah backup \
  /var/lib/ah/ah.db

Step by step, it:

1. creates backups/ next to the database, readable only by its owner
2. stops if that disk is 90% full or more, and warns at 80%
3. opens ah.db read-only and runs VACUUM INTO a temp file, with a 5 minute timeout
4. gzips the copy to ah-YYYYMMDD-HHMMSS.db.gz, writing to a .tmp file and renaming it at the end, so a half-written backup never has a real name
5. does the same for ah-metering.db if it exists
6. keeps the 10 newest state backups and the 10 newest metering backups, and deletes older ones

Afterwards you'll see something like:

/var/lib/ah/backups/
├── ah-20261027-091700.db.gz
└── ah-metering-20261027-091700.db.gz

A quick sanity check that a backup is a real SQLite file:

sudo sh -c \
  'gunzip -c /var/lib/ah/backups/ah-2*.db.gz' \
  | head -c 15
# SQLite format 3

To run it every hour, a cron entry is enough:

# /etc/cron.d/ah-backup
17 * * * * ah /usr/local/bin/ah backup /var/lib/ah/ah.db

What the backup doesn't include

The .db.gz files aren't everything you need:

Item	Why you need it
master.key	decrypts env vars and DB passwords
/etc/default/ah	bootstrap token and settings
Docker volumes	your Postgres and Redis data
local registry	images you built on the box

The master key is the big one. Secrets in ah.db are encrypted with it, and ah backup never writes it into a backup. Store it somewhere other than your database backups, because anyone holding both can read your secrets.

One more detail. If you rotate the key with ah rotate-key, older backups still need the old key. Keep each key for as long as you keep backups made with it.

And your apps' actual data, the rows in the Postgres databases ah created for you, lives in Docker volumes named ah-db-*. That needs its own backup. The SQLite file only knows those databases exist.

Getting copies off the box

A backup that lives on the machine it's protecting doesn't protect much. This is the kind of chore I hand to my agent. A prompt like this works:

Every hour, after ah backup runs, rsync
/var/lib/ah/backups/ to my backup host.
Copy master.key to my secrets manager once,
and again after any key rotation. Alert me
if the newest backup is more than 2 hours
old.

The agent writes the timer, the copy script, and the check. I review them once. The big platforms used to give you redundancy, backups, and monitoring as a feature. Now an agent can build the same pieces on your own box in an afternoon.

If an hour of possible loss is too much, the repo has an optional Litestream setup (deploy/litestream.yml and a runbook in docs/ops/) that streams both databases to S3-compatible storage as they change. ah doesn't install or manage it. You run it as its own service.

Bringing it back

There's no ah restore command. A restore is a few file operations, which I like, because there's nothing extra that has to work on the worst day. On the new server, install ah the normal way and copy over /etc/default/ah. Put the backup files in /var/lib/ah/backups/ and the key in your home folder. Then, as root:

systemctl stop ah
cd /var/lib/ah
mkdir -p pre-restore
mv ah.db* ah-metering.db* pre-restore/

TS=20261027-091700
gunzip -c backups/ah-$TS.db.gz > ah.db
gunzip -c backups/ah-metering-$TS.db.gz \
  > ah-metering.db
install -m 600 -o ah -g ah \
  ~/master.key master.key

chown ah:ah ah.db ah-metering.db
chmod 600 ah.db ah-metering.db
systemctl start ah

Moving the old -wal and -shm files out of the way is the step people skip. If a stale WAL file sits next to the restored database, SQLite can try to replay it into the restored file. If you skip the metering file, ah starts with an empty one and you lose usage history, nothing else.

Then check it's up. The API wants HTTPS, so a local request has to say it came through the proxy:

curl -s -H 'X-Forwarded-Proto: https' \
  http://127.0.0.1:8080/v1/system/health

If you restored with the wrong master.key, you'll know right away. On startup ah decrypts a key it uses to check API keys, and if that fails it refuses to start, with wrong master key? in the error in journalctl -u ah. I like that it fails loudly. A server that started and then quietly rejected every API key would be much worse.

What happens to the apps

The restored database describes apps that may not be running on the new machine. ah doesn't trust those rows. Every 30 seconds a loop called the reconciler compares the database with what Docker actually has. An app marked running with no container gets marked crashed, with the reason container not found by reconciler. Nothing restarts on its own.

To bring an app back, redeploy it with ahc, the project's command-line client:

ahc service redeploy my-api
ahc status

A redeploy starts the app again from the image it already had. If that image was built on the old server and you didn't copy the local registry, it won't be there. Start a fresh build from git instead, the same way the app was deployed the first time.

Or tell your agent "the server was restored, redeploy everything that was running." It can list the services, see which ones are marked crashed, and work through them one at a time.

Then do the test that actually matters. Open an app that uses a secret, like a database password, and make sure it works. That's the proof that the key and the database belong together.

Wrapping up

agentic-hosting keeps its whole control plane in two SQLite files: ah.db for state and ah-metering.db for usage. WAL mode keeps reads and writes out of each other's way, ah backup uses VACUUM INTO to take consistent copies while the server runs, and a restore is just putting the files and the master key back. Have your agent ship the copies off the box, and run the restore once on a scratch server before you need it.

• Star agentic-hosting on GitHub so more people find it.
• Give it to your agent: llms.txt to sign up and set up, or the agent skill for daily use.
• Try the quickstart, or read the backups page on the docs site.
• Open an issue if you'd want a built-in ah restore or volume backups for managed Postgres. I'd like to know which one you'd use first.

Cover photo: seats and fittings from old Amtrak passenger cars piled in a warehouse at Mira Loma, California, waiting to be renovated or replaced. Charles O'Rear, May 1974, for the EPA's Documerica project. U.S. National Archives, public domain.