Full App Backup for Game Receipts: Node.js Nightly Job, User Uploads, Object Storage

#backups #objectstorage #python

Short answer: treat a nightly game-app backup as one recoverable run containing the database dump, the original receipt files, and a manifest that names both; use private object storage, immutable run prefixes, and a restore drill before calling the job complete.

The important decision is access control versus delivery simplicity. Receipt images and purchase records are related evidence, not two independent exports. A public object URL may be easy for a client to consume, but it is a poor default for audit material. A private bucket with short-lived, server-issued access takes more application work and gives the recovery process a boundary worth defending.

This is the invariant: a run is either a candidate for restore or it is not. A scheduler success message is not evidence. A database dump that finished while the upload archive was still being assembled is not a backup.

How should a gaming app combine nightly backups, database dumps, uploads, and a restore checklist?

Give every run a unique UTC identifier, and put that identifier in every key. For example:

from datetime import datetime, timezone


def backup_keys(run_id: str) -> dict[str, str]:
    root = f"backups/production/{run_id}"
    return {
        "database": f"{root}/database/game.sql.gz",
        "uploads": f"{root}/uploads/receipts.tar.gz",
        "manifest": f"{root}/manifest.json",
    }


run_id = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
keys = backup_keys(run_id)

The prefix is not a folder abstraction that makes consistency appear by magic. It is a naming contract. backups/production/20260811T020000Z/ should contain the database export, the receipt archive, and a manifest only after both payloads have been validated and stored. Keep the environment in the path so an operator cannot confuse staging evidence with production evidence during a stressful recovery.

The manifest should record the run ID, the exact object keys, byte counts, checksums produced by the backup worker, and the database export timestamp. It should also say which upload inventory the archive represents. That last field matters: a receipt row may exist in the database while its original object is absent, or the reverse may be true after a failed deletion workflow.

Short rule: no manifest, no restore candidate.

There is a consistency limit here. A nightly snapshot normally has a point-in-time boundary for the database, but user uploads arrive continuously. Decide what “complete” means before writing code: for example, include every receipt whose database record was committed before the export cutoff, and record any later objects for the next run. The exact policy depends on the application’s transaction model. I’m not sure a single archive can prove that relationship by itself; the manifest and an application-level inventory check are what make the claim testable.

The architecture decision record

The write path should have four states: planned, payloads stored, manifest committed, and verified. Only the third state is eligible for restore, and only the fourth should trigger retention cleanup. This gives operators a clear failure boundary when a worker is killed halfway through a large archive.

Choice	Delivery benefit	Control or recovery cost
One `latest/` prefix	The reader always knows one lookup path	A partial overwrite can masquerade as the newest complete backup, and database and upload generations can diverge
Separate top-level database and upload prefixes	Retention rules are easy to target by data class	The restore operator must join two listings and can accidentally select different runs
One dated run prefix with a manifest	One restore boundary; the manifest can name exact objects	Retention and verification code must understand the run lifecycle
Public receipt objects	Very simple delivery to clients	Audit evidence becomes broadly addressable; access policy is harder to reason about

I would choose the third row for the game receipt workflow and keep the bucket private. The catch is that it is not suitable when the primary requirement is anonymous, permanent delivery of public media; a public distribution store or a separate delivery path is the better boundary there. Do not make the backup copy carry that convenience requirement.

Object storage also does not answer every durability question. The bucket may be durable while the design still has one failure domain, one credential, or one region. If the recovery objective includes a regional disaster, create and test a secondary-copy process. If the audit requirement includes immutability, select storage controls and permissions that prevent the backup identity from deleting its own evidence. The capability has to be verified against the actual policy, not inferred from the word “backup” in a product screen.

The critical path in Python

The storage client is deliberately abstract in this example. The application should bind put_bytes to the chosen object-storage SDK or HTTP implementation, while keeping the ordering and key contract unchanged. This is the part worth testing.

import hashlib
import json
from pathlib import Path
from typing import Protocol


class ObjectStore(Protocol):
    def put_bytes(self, key: str, data: bytes, content_type: str) -> None:
        ...


def read_payload(path: Path) -> tuple[bytes, str]:
    data = path.read_bytes()
    digest = hashlib.sha256(data).hexdigest()
    return data, digest


def write_run(store: ObjectStore, run_id: str, database: Path, uploads: Path) -> None:
    keys = backup_keys(run_id)
    database_data, database_sha256 = read_payload(database)
    uploads_data, uploads_sha256 = read_payload(uploads)

    store.put_bytes(keys["database"], database_data, "application/gzip")
    store.put_bytes(keys["uploads"], uploads_data, "application/gzip")

    manifest = {
        "run_id": run_id,
        "database": {"key": keys["database"], "sha256": database_sha256},
        "uploads": {"key": keys["uploads"], "sha256": uploads_sha256},
    }
    store.put_bytes(
        keys["manifest"],
        json.dumps(manifest, sort_keys=True).encode("utf-8"),
        "application/json",
    )

The manifest write is the publication step. A failed payload write must fail the job and leave no manifest; a failed manifest write must not be converted into a success by a scheduler wrapper. Retry policy belongs in the storage adapter, with bounded retries and logs that include the run ID but never receipt contents or credentials.

Before this function runs, the worker still needs a scheduler lock or queue policy so two nightly workers cannot claim the same run. Object keys are useful identifiers, but they are not a substitute for coordination. The job should also retain its local temporary files until the remote objects have been checked, then remove them according to the host’s disk policy.

Restore checks that expose a false backup

A restore test should run against an isolated database and an isolated object-storage prefix. The sequence is intentionally boring:

Select a manifest whose payload keys share the same run ID.
Download both payloads and verify their recorded SHA-256 values before import or extraction.
Restore the database dump into a non-production database.
Extract the receipt archive into an isolated object path or filesystem location.
Compare the database’s receipt inventory with the archive inventory, including a sample of original filenames and content types.
Start the application against the restored data and open a receipt through the same authorization path used by an auditor.
Measure restore duration, record missing or extra objects, and retain the result with the run ID.

The sixth check is where delivery simplicity gets challenged. A file that can be downloaded by a storage administrator is not necessarily accessible to the audit service, and a file that the audit service can read is not necessarily safe to expose to a game client. Test both authorization decisions. For browser delivery, use an application-controlled, time-limited access mechanism rather than turning the backup prefix into a public asset path.

The rejected design is a single latest/ prefix updated by independent database and upload jobs. It is valid for disposable exports where a new artifact replaces the old one and no relational reference must be reconstructed. It is not suitable for a full app restore: a timeout after the database upload can leave a fresh dump next to yesterday’s receipts, and a successful listing can still tell no one whether the pair is complete.

Operational limits and references

Lifecycle expiration can remove old run prefixes, but lifecycle configuration is retention automation, not a restore test. Keep a retention review that asks which manifests are complete, which copies are in the secondary location, and which restore drills still meet the recovery objective. Storage pricing also varies by storage class, request type, retrieval, and transfer; cost should be measured from the workload rather than used as the primary architectural argument.

If the access policy, inventory semantics, or disaster-recovery target cannot be stated in one page, the backup design is not finished. More automation will not repair an undefined restore boundary.

DEV Community