What to restore when LiteLLM holds team keys, budgets and spend data

YNVAR — Wed, 22 Jul 2026 14:35:22 +0000

LiteLLM forwards requests, but a team deployment is not stateless. PostgreSQL holds virtual-key records, teams, budgets and spend data. Stored credentials also depend on the key used to encrypt them.

That makes the database, configuration and decryption key part of one recovery plan. Restoring only the container or only the database is not enough. This article explains what must be restored, what can be rebuilt, and how the salt key changes master-key rotation.

The state a LiteLLM gateway holds

Behind the OpenAI-compatible endpoint, the gateway keeps its state in a PostgreSQL database. The database stores hashed virtual-key records, their scopes, users and teams, budgets, rate limits and spend data. Provider credentials are a separate case: they may be stored in the database encrypted, or supplied from an external secret store, in which case the database never holds them. The gateway also depends on the configuration that defines the model map, and on the key that decrypts any credentials the database does hold.

If you lose the database, you lose the virtual-key records and the spend history. Provider credentials you keep in an external secret store can be re-injected. So the operational unit is the database plus the configuration plus the decryption key, not the container that serves traffic.

What a consistent restore has to bring back

A consistent restore is three things brought back together: the PostgreSQL database, the configuration that points at it, and the key that decrypts stored credentials — the salt key if you set one, or the master key if you did not. The database on its own will not start a working gateway. Version the database and the configuration together as one restore set. The decryption key has to be available to that restore too, but keep it in a separate secret store rather than inside the backup, so the encrypted database and the key that opens it are not sitting in the same place.

Then rehearse the restore on a throwaway instance. Bring the config and database back, start the gateway on the same pinned version, log into the admin UI, confirm a virtual key still resolves and routes a request, and confirm a model whose credentials live in the database still works.

A backup you have never restored has not been shown to work. Set backup frequency and retention from how much spend history and how many live keys you could accept losing between backups, and treat that restore rehearsal as the launch gate, not an afterthought.

The PostgreSQL database is the system of record

LiteLLM documents PostgreSQL through DATABASE_URL as a requirement for virtual keys, budgets and spend tracking. That is the documented position, and it is the reason the database is the system of record here.

The rest of this section is general database operations practice rather than a documented requirement for running the proxy. Once the database holds your keys, users, teams and usage, treat it as production data: give it persistent storage, size the connection pool against the total connection budget your instances and workers share rather than a per-worker guess, and apply the backup-and-restore discipline above.

Spend logging writes on the request path, so watch that write volume as traffic grows — the production docs give concrete settings for batching those writes.

What LiteLLM stores in PostgreSQL

The reason to run the gateway instead of sharing one provider key is virtual keys, and they are the state you are protecting.

Instead of handing an app your real provider key, you issue a gateway key through /key/generate, scoped to an app or team with a team_id or user_id and the models it may call. The real provider keys stay in the gateway; the app only holds a key you can revoke or rotate without touching the provider account.

A key request is small:

{
  "team_id": "team-web",
  "models": ["gpt-4o", "claude-3-5-sonnet"],
  "max_budget": 200,
  "budget_duration": "30d",
  "rpm_limit": 60
}

The team_id has to reference a team you have already created, and you get back an sk- virtual key the app uses as its API key.

max_budget and budget_duration set a spend cap and its reset window; rpm_limit, tpm_limit and max_parallel_requests cap request rate, token rate and concurrency, so one runaway job cannot spend the month in an afternoon.

These apply only where you configure them and enforce against the gateway's own accounting, not the provider invoice. How a budget resets and what happens at the limit are version-specific — confirm the field names and behaviour against the docs for the version you run, because the key API changes between releases.

All of this is stored in PostgreSQL, which is why the restore above matters.

How the salt key changes master-key rotation

Two secrets protect that state, and they behave differently.

LITELLM_MASTER_KEY is the proxy's admin credential — it must start with sk-, and it creates and manages every virtual key.

LITELLM_SALT_KEY, when you set it, is the key that encrypts stored credentials — models, the config environment_variables, and the credentials table — in the database.

Set the salt key from the start, because the master-key rotation procedure splits on exactly whether it is present, and the official rotation guide is worth reading before you rotate anything.

If the master key leaks, an attacker has proxy-admin control: they can create, revoke and re-scope virtual keys, change the configuration, and use the model access the gateway is configured with. Where no salt key is set, the master key is also the credential-encryption key, so they can decrypt the stored credentials too.

Back up PostgreSQL before either rotation, whichever case you are in.

With a salt key configured, the master key is only an authentication credential. To rotate it you generate a new value, update whichever master-key source your deployment uses — the LITELLM_MASTER_KEY environment variable or general_settings.master_key in the config — so every instance gets the same new value, then restart them all.

You do not call POST /key/regenerate with new_master_key here — the docs warn that doing so would re-encrypt your stored credentials under a key the proxy never uses to decrypt, which strands them.

Without a salt key, the master key also encrypts those stored credentials, so rotating it has to re-encrypt them. That is why the database backup matters especially here: the re-encryption deletes and recreates rows and skips any row it cannot re-encrypt, so a partial failure can strand data.

Call POST /key/regenerate with the current master key as key and the new one as new_master_key. It re-encrypts the stored models, the config environment variables, and the credentials, MCP-server and user tables.

After it returns, update the same master-key source — the LITELLM_MASTER_KEY environment variable or general_settings.master_key — so every instance gets the new value, restart them all, log into the admin UI with the new key, and test a request against a DB-stored model.

Neither of those is the same as rotating a single virtual key, which is a different operation on a different endpoint, POST /key/{key}/regenerate.

Virtual keys are stored hashed rather than encrypted, so they keep working through a master-key rotation. You regenerate one only when that specific key is compromised.

The documented rotation flows assume that you still hold the current master key. Keep it in a real secret store, keep the salt key separate from it, and rehearse a rotation before you need one.

Redis is not part of the restore

You do not need Redis for a single instance, and when you do use it, it is not part of what you restore.

It becomes relevant once you run more than one proxy, where it shares rate-limit counters and router state across instances so limits apply across the fleet rather than per process.

It backs the response cache when you have enabled caching and pointed it at Redis, and it can cache virtual-key lookups when you set enable_redis_auth_cache, which keeps the database off the path of every request.

PostgreSQL is the durable system-of-record state. Redis holds rebuildable cache, counter and router state, so losing it costs cache warmth and cross-instance counters, not your virtual-key records or spend history.

Restore PostgreSQL and allow the Redis cache and distributed counters to rebuild.

Upgrades change the schema you are restoring

The schema of that PostgreSQL database moves with LiteLLM, which matters because it is the thing you back up and restore.

The project ships Prisma migrations gated by USE_PRISMA_MIGRATE and DISABLE_SCHEMA_UPDATE, versioned so older releases keep using their own migrations.

Treat an upgrade as a change to production data: pin the image to a specific tag instead of tracking latest, take a database backup first, and read the release notes.

In a multi-pod deployment, run migrations from one place and set DISABLE_SCHEMA_UPDATE on the rest so pods do not race the same migration.

A restore is only clean if you bring the database back on a version whose schema matches it, which is another reason the pinned version is part of the restore rehearsal above.

Keep the admin UI off the open internet

The admin UI at /ui is an access path to all of that state — it creates keys, adds models and shows spend.

It has its own login through UI_USERNAME and UI_PASSWORD or through SSO, and you can turn it off entirely with DISABLE_ADMIN_UI="True".

The docs do not say it must never be public; that is a call I make anyway.

Keep the admin surface behind its login and, where you can, off the public internet, with the gateway behind a reverse proxy over TLS rather than on a raw published port. It serves traffic fine without the UI being publicly reachable.

Failure modes around the state

The failures worth designing around are all about that state.

A leaked master key exposes proxy-admin control, and the stored credentials too if no salt key is set.

A database you cannot restore costs you the keys and the spend history, which is why the restore rehearsal is the gate.

One failure shows up in the spend the database records rather than in the storage itself: fallback between providers can raise spend sharply if a cheap model fails over to an expensive one, depending on how retries and billing are configured. Cap it with the per-key budgets you already keep in that database and watch the spend when it triggers.

A public, unauthenticated admin UI is a direct path to the key namespace.

When you do not need this

If one script uses one provider key, none of this applies — you would be adding a stateful service, a database and an admin surface to operate for no gain over calling the provider directly.

The restore question only becomes real when several apps, teams or people share model access through per-team keys, budgets and spend attribution.

Reach for LiteLLM when that shared state is what you need, and then treat the database and its restore as the load-bearing part.

Operate it yourself, or have it operated

YNVAR can operate the container, TLS boundary, PostgreSQL, backups and upgrade window on European infrastructure. The customer keeps the provider accounts, provider keys, virtual-key policy and model-access decisions.

The full write-up, including the backup scope and the operate-versus-own split goes further than this post.

European hosting documents the location of that platform layer. It does not determine where an external model provider processes requests, and it is not a GDPR-compliance claim.

YNVAR is not an official partner of the LiteLLM project or of any model provider.

— Adrians Petrovs, Founder and Infrastructure Engineer at YNVAR

DEV Community: YNVAR