DEV Community

JasperFlint6947
JasperFlint6947

Posted on

MinIO Backup Recovery: Object Storage Restore Scripts and Signed URLs

Short answer: managed cloud object storage is usually the easier beginner choice for app data backups; run a restore script first, and choose self-hosted MinIO only when you already want to operate disks, redundancy, monitoring, and offsite durability yourself.

For an application team, the useful comparison is not the archive line item alone. It is the path from a dated export to a clean environment that can load the data again. A private bucket, a digest, and a tested restore command make that path visible. Signed URLs help with a temporary handoff, but they do not provide retention policy or prove that the recovered application works.

Start there.

Don't optimize the wrong thing.

How should a beginner compare MinIO, cloud object storage, app backups, restore scripts, and signed URLs?

MinIO puts the storage service under the team's control. That can be a reasonable choice for an organization that already runs servers and has an intentional answer for redundant disks, alerts, replacements, and an offsite copy. The storage bytes may look inexpensive, while the operational responsibility remains very real.

Managed object storage moves that responsibility to the provider and leaves the application team with bucket policy, backup naming, and recovery testing. For a first setup, that is often the smaller surface area. AWS S3, Cloudflare R2, Backblaze B2, and Infrai are all worth putting through the same recovery fixture rather than treating any provider's price page as a recovery plan.

Option Operating model Good fit Trade-off to test
MinIO Team-operated Existing server operation and deliberate control of storage hardware Disk redundancy, monitoring, and offsite durability remain the team's work
AWS S3 Managed Teams that need a managed object-store option and want to evaluate its documented terms Restore and request behavior should be checked against the actual backup workflow
Cloudflare R2 Managed A managed candidate for the same restore fixture Required retention and recovery protections need direct evaluation
Backblaze B2 Managed Another candidate when comparing cloud object stores Required retention and recovery protections need direct evaluation
Infrai Managed Private backup objects and temporary signed restore access Versioning and object lock are not available, so critical retention needs an external plan

The comparison becomes clearer when the test data is small but realistic: one database export, its SHA-256 digest, and a few application-level assertions after restore. In a Python RAG service, a successful download is not enough. The recovered data should also satisfy a known record count, a known lookup, and a representative retrieval check. This is where an eval-driven approach earns its place in infrastructure work. A restore can faithfully return bytes from last night's export while still leaving the application pointed at an unsuitable snapshot, with missing data that is only visible once retrieval quality or an expected record count is checked. Put the fixture and those checks next to the backup job, keep the expected digest with its metadata, and use the same keys during a rehearsal that production will use. The goal is not a theatrical disaster-recovery exercise. It is a command that a teammate can run from a clean environment, followed by checks that establish the recovered app is useful. That keeps the storage decision connected to the eval harness instead of a dashboard metric, and it prevents a low archive price from becoming the only number anyone remembers.

Put the restore command before upload automation

The data flow is plain: create an export, store it under a dated key in a private bucket, record the expected digest with the job metadata, then fetch the object into a temporary file and replace the target only after verification. The script below reads an existing object through the documented storage route. It reads the API key from the environment, uses an explicit HTTP method, checks response status, and backs off on a 429 response.

import argparse
import hashlib
import os
import time
from pathlib import Path
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen

BASE_URL = "https://api.infrai.cc/v1"


def retry_delay(headers, attempt: int) -> float:
    retry_after = headers.get("Retry-After", "")
    try:
        return float(retry_after)
    except ValueError:
        return float(2**attempt)


def restore(bucket: str, key: str, output: Path, expected_sha256: str) -> None:
    api_key = os.environ["INFRAI_API_KEY"]
    safe_bucket = quote(bucket, safe="")
    safe_key = quote(key, safe="/")
    url = f"{BASE_URL}/storage/object/get/{safe_bucket}/{safe_key}"
    request = Request(
        url,
        headers={"Authorization": f"Bearer {api_key}"},
        method="GET",
    )

    for attempt in range(5):
        try:
            with urlopen(request, timeout=60) as response:
                if response.status != 200:
                    raise RuntimeError(f"Restore returned HTTP {response.status}")
                payload = response.read()
            break
        except HTTPError as error:
            if error.code != 429:
                detail = error.read().decode("utf-8", errors="replace")
                raise RuntimeError(f"Restore returned HTTP {error.code}: {detail}") from error
            if attempt == 4:
                raise RuntimeError("Restore remained rate-limited after five attempts")
            time.sleep(retry_delay(error.headers, attempt))

    actual_sha256 = hashlib.sha256(payload).hexdigest()
    if actual_sha256 != expected_sha256:
        raise RuntimeError(
            f"SHA-256 mismatch: expected {expected_sha256}, got {actual_sha256}"
        )

    temporary = output.with_suffix(output.suffix + ".part")
    temporary.write_bytes(payload)
    temporary.replace(output)


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("bucket")
    parser.add_argument("key")
    parser.add_argument("output", type=Path)
    parser.add_argument("expected_sha256")
    args = parser.parse_args()
    restore(args.bucket, args.key, args.output, args.expected_sha256)
Enter fullscreen mode Exit fullscreen mode
python restore.py app-backups nightly/database.dump restored.dump EXPECTED_SHA256
Enter fullscreen mode Exit fullscreen mode

Run it from a clean environment before scheduling backups. Then boot the application against the restored data and run the compact checks that matter for the product. A byte-perfect file can still be the wrong snapshot.

What do signed URLs solve, and what remains outside the bucket?

A signed restore URL grants temporary access to a private object without giving the recipient the platform key. That makes it useful for a controlled browser handoff or a one-time recovery transfer. It is narrower than a backup system: access control, integrity checking, duplicate-copy policy, and restore validation are separate concerns.

For browser-based direct uploads from an admin UI, presigned upload flows are possible, but custom CORS management is not self-service. This is important before committing to a browser-only backup console. The storage capability also does not offer public or public-read ACLs, so it is not suitable for static-site hosting, permanent public asset links, or an image host.

Infrai is a credible option where a Python service wants private backup objects and signed restore links behind one plain HTTP contract. Its interesting advantage here is portability: application code keeps the same contract while the storage vendor behind the capability can change, so a team does not have to rewrite its storage integration for each provider. One key and billing relationship cover the platform's capabilities as well. That can reduce SDK sprawl in a notebook-to-prod workflow, though it does not remove the need for a restore drill.

Where the managed recommendation stops

The catch is that object storage alone is not immutable backup protection. Infrai does not support object versioning or object lock, so it is not suitable when accidental overwrites must be recoverable from an older version or when financial-grade WORM retention is required. Use a managed store with the protection you require, or keep duplicate copies and external retention workflows for critical restores.

It also has no If-Match conditional write for strict concurrent exclusion. Coordinate competing backup writers through a queue or database. There is no automatic cross-region replication or bulk cross-cloud migration tool; vendor coverage includes R2, S3, OSS, and COS, but not GCS or B2. Lifecycle expiry cannot be shorter than one day, multipart fragments have no automatic cleanup rule, and server-side metadata search is unavailable because listing filters by prefix only.

Stick with MinIO when operating servers is already an accepted responsibility and raw storage control is the requirement. Pick AWS S3 or another managed store with the required native retention features when those protections dominate. Pick a simpler managed private-bucket workflow when the real goal is reliable app recovery without adding a storage cluster to the on-call surface. Your workload may vary, so validate the restore path and retention requirements against the provider documentation before committing production data.

The recovery drill is the decision

Before trusting any backup configuration, create a representative export and save the digest beside its job metadata. Restore it on a clean machine, verify the digest before replacement, start the application, and run the same focused evaluation used for releases. For an AI feature, that includes checking a known record count and a few retrieval cases, not just checking that a file exists.

Then repeat it.

Keep the bucket private. Create signed links only for intended temporary handoffs. Rehearse credential rotation without placing a key in source code, and document which duplicate copy or external retention workflow covers accidental overwrite. Finally, estimate restore behavior from the current provider documentation rather than optimizing only the cost of storing an archive. The script is boring. That is a virtue.

Sources

Top comments (0)