DEV Community: Ethan Carter

Garage vs RustFS: Running S3 on a Single Node

Ethan Carter — Tue, 22 Sep 2026 23:59:57 +0000

Garage vs RustFS: Running S3 on a Single Node

Garage and RustFS are both Rust-written, single-binary S3-compatible stores you can run on one node. The decisive split is the license and the S3 feature set: Garage is AGPL-3.0 and skips versioning, Object Lock, and server-side encryption; RustFS is Apache 2.0 and ships all of them. Run Garage when you want a tiny, geo-distributed store on secondhand hardware and AGPL is fine; run RustFS when your app needs versioning, immutability, IAM, and a MinIO-compatible dialect on the simplest possible operational surface.

Key stats

	Garage	RustFS
License	AGPL-3.0 (network copyleft)	Apache 2.0 (permissive)
Implementation	Rust	Rust
Single-node start	`garage server --single-node`	one `docker run`
S3 object versioning	❌ Missing	✅ Available
Object Lock (WORM)	❌ Missing	✅ Available
Server-side encryption	❌ Missing	✅ Available
Bucket / site replication (S3 API)	❌ Missing	✅ Available
ACL / bucket policies	❌ (own permission model)	✅ (IAM / Policies)
Default S3 API port	3900	9000
Documented minimum RAM	1 GB	not published

Garage's "missing" rows are not bugs — they are its own S3-compatibility page stating the feature is not implemented. RustFS rows are its README Feature & Status table (re-checked 2026-09-21). We cite the docs, not benchmarks.

What is the real difference between Garage and RustFS?

Garage is an S3-compatible distributed object store designed for self-hosting at a small-to-medium scale, built by the French non-profit Deuxfleurs. Its own docs describe it as a service that "replicates data 3x over distant servers" and is "not as latency sensitive" because it avoids consensus algorithms like Paxos or Raft. You run one garage binary per node, point nodes at each other, and assign a cluster layout. The design center is geo-distributed clusters of cheap, mismatched machines across physical locations.

RustFS is narrower by design: a purpose-built, MinIO-compatible S3 object store written in Rust under the Apache 2.0 license. Its README Feature & Status table marks S3 Core, Versioning, Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode as available, with S3 Tables (Iceberg REST) and MinIO On-Disk Compatibility in preview. There is no POSIX layer and no FUSE mount. Both are single Rust binaries, but Garage optimizes for resilient distribution across locations; RustFS optimizes for S3 correctness and the smallest operational surface.

How do the licenses actually differ?

Garage's source code is released under the AGPL v3 license. The project states it plainly: "if you patch Garage and then use it to provide any service over a network, you must share your code." That is network copyleft — stronger than the ordinary GPL. If you modify Garage and expose it as a service, you must publish your modified source. For a homelab or an internal-only deployment that never serves third parties, AGPL is usually a non-event. The moment you ship a modified Garage as a product or a hosted service, the obligation attaches.

RustFS is released under the permissive Apache 2.0 license. The README notes it is "released under the permissible Apache 2.0 license," with no copyleft. You can run, modify, and embed it in a closed-source product or a commercial service without a source-disclosure obligation. For most companies evaluating storage, this is the practical differentiator: RustFS imposes no network-copyleft tax, while Garage's AGPL-3.0 does. If license friction is a concern, that single line settles the comparison before any feature table does.

What S3 features does Garage leave out?

Garage implements a deliberate subset of the S3 API, and its own compatibility page lists what is missing. Bucket versioning is "Missing." Object Lock — both legal hold and retention — is "Missing" across every lock endpoint. Server-side encryption (PutBucketEncryption and friends) is "Missing"; the project's stance is that you should encrypt at the partition or client side instead. Bucket replication via the S3 API is "Missing." ACLs and bucket policies are "Missing" — Garage has its own per-access-key, per-bucket permission model rather than AWS's ACL-or-policy system.

Lifecycle is only partially implemented: Garage supports AbortIncompleteMultipartUpload and Expiration and nothing that depends on versioning or storage classes. Multipart upload, presigned URLs, CORS, and static-website hosting are implemented. The honest read: Garage is a clean S3-compatible put/get store with replication built in at the cluster level, not a drop-in for apps that assume versioning, WORM immutability, SSE, or IAM policies. If your client needs those, test first or look at RustFS, which lists every one of them as available.

How does each one run on a single node?

Garage's documented single-node path is one command after you generate a config. The quick start generates a garage.toml (metadata_dir, data_dir, replication_factor = 1, ports 3901/3900/3902/3903) and then starts the server:

garage server --single-node --default-bucket

That flag auto-configures a single-node cluster with no replication and creates a default access key and bucket from environment variables. For Docker, the official quick start pins dxflrs/garage:v2.3.0 (newer stable releases exist) and maps the four ports:

docker run \
  -d \
  --name garage-container \
  -p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903 \
  -v $(pwd)/garage.toml:/etc/garage.toml \
  -e GARAGE_DEFAULT_ACCESS_KEY \
  -e GARAGE_DEFAULT_SECRET_KEY \
  -e GARAGE_DEFAULT_BUCKET \
  dxflrs/garage:v2.3.0 \
  /garage server --single-node --default-bucket

RustFS starts with one docker run (verbatim from its README):

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

That serves the S3 API on 9000 and a console on 9001. Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any real exposure. RustFS also ships a single binary you can drop on a host.

When should you pick Garage over RustFS?

Pick Garage when the requirement is resilient storage across locations on hardware you already own, and you are comfortable with AGPL-3.0. Its sweet spot is exactly what Deuxfleurs built it for: a few nodes in two or three physical locations replicating data so it stays available through failures and high-latency links. The operational surface is small — one binary, a TOML file, a layout command — and the memory footprint is tiny (the project cites 1 GB RAM and 16 GB disk as minimums). For homelab backups, media, or app assets, self-hosting wins on price the moment the hardware is sunk cost.

Garage also wins if you specifically want its model: CRDT-based federation instead of Raft/Paxos, so it tolerates slow or flaky inter-node links that would stall a consensus-based cluster. If your app only needs put/get/list plus presigned URLs and a static-site endpoint, Garage is lighter than running a full S3 engine. The catch is the missing feature set above — if your app leans on versioning, Object Lock, or IAM, Garage is the wrong tool and you should not try to bolt those on.

Where does RustFS win?

RustFS wins on the axis most S3 teams actually care about: a complete, MinIO-compatible S3 endpoint with the least operational surface. One docker run, no cluster-layout choreography, a web console on 9001. The feature set is broad and marked available in the README — Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode are all shipped and covered by CI gates. If your tools already speak MinIO's S3 dialect, RustFS lists MinIO On-Disk Compatibility as a preview feature and targets that dialect as its primary surface.

The Apache 2.0 license is the other win: no network-copyleft obligation, so you can wrap RustFS in a product or a hosted service without source-disclosure strings. For a team whose only contract with storage is the S3 API and whose app assumes versioning, immutability, or IAM policies, RustFS is the closer fit and the shorter path to a working, compliant endpoint. RustFS is younger as an ecosystem, and S3 Tables (Iceberg) plus MinIO on-disk compatibility are still preview — own that if a lakehouse or byte-exact MinIO migration is the requirement.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of September 2026. Garage rows come from garagehq.deuxfleurs.fr (quick start and the S3-compatibility page); RustFS rows come from its README Feature & Status table (re-checked 2026-09-21). No benchmarks, no marketing.

Capability	Garage	RustFS
S3-compatible API	✅ (subset)	✅ (S3 Core)
Object versioning	❌ Missing	✅ Available
Object Lock (WORM)	❌ Missing	✅ Available
Server-side encryption	❌ Missing	✅ Available
Bucket / site replication (S3 API)	❌ Missing	✅ Available
Lifecycle (ILM)	⚠️ Partial (Abort MPU + Expiration)	✅ Available
ACL / bucket policies	❌ (own model)	✅ (IAM / Policies)
Web console / admin API	✅ (admin API)	✅ (Web Console)
Geo-distribution model	CRDT federation, no Raft/Paxos	Distributed Mode (✅ Available)
Single binary	✅ (`garage`)	✅
License	AGPL-3.0	Apache 2.0
Implementation	Rust	Rust

Two rows deserve a call-out. RustFS lists S3 Tables (Iceberg REST) as preview, so a mature lakehouse over RustFS data is not a promise the project makes yet. And Garage's missing versioning, Object Lock, SSE, and ACL rows are by design — its strength is resilient distribution, not S3-feature parity.

How do you choose between them?

Use this rule of thumb: if the ticket says "backups across sites on hardware we own, AGPL is fine," start with Garage. If the ticket says "S3 endpoint for an app that needs versioning, immutability, or IAM, and we don't want copyleft," start with RustFS.

A split is legitimate and common. Run Garage for the geo-distributed, secondhand-hardware layer where its CRDT model and tiny footprint pay off, and run RustFS in front of the S3-facing applications where versioning, Object Lock, and a MinIO-compatible dialect are the whole ask. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job — Garage dragged into an app that assumes Object Lock becomes a missing-feature ticket, and RustFS pushed into a no-copyleft-averse, flaky-link geo cluster carries obligations it was not built to carry.

FAQ

Is Garage a drop-in MinIO replacement?

No. Garage implements a subset of the S3 API — its own compatibility page lists versioning, Object Lock, replication, SSE, and ACL/policy as missing — so most S3 clients work for put/get/list but apps that assume MinIO's full feature set will break. RustFS targets the MinIO S3 dialect as its primary surface and lists MinIO On-Disk Compatibility as a preview feature. If a MinIO-compatible endpoint is the requirement, RustFS is the closer fit.

Does Garage support object versioning?

No. Garage's S3-compatibility page marks bucket versioning as "Missing" and notes it "does not (yet) support object versioning." GetBucketVersioning returns a stub that always reports versioning disabled. If your workload needs versioned objects or point-in-time recovery through S3, Garage cannot provide it today; RustFS lists Versioning as available.

Is RustFS Apache 2.0 like Garage is AGPL?

No. RustFS is released under the permissive Apache 2.0 license with no copyleft, while Garage is AGPL-3.0. The practical difference: if you modify Garage and expose it as a network service, you must publish your modified source; RustFS imposes no such obligation. For a hosted or embedded product, that single license line is often the deciding factor.

Can Garage run on a single node?

Yes. Garage's quick start runs a single node with garage server --single-node --default-bucket, which auto-configures a no-replication cluster and creates a default key and bucket. The docs warn this provides no data redundancy and "should not be used in production." Garage is designed for three nodes across zones; single-node is for evaluation and dev, not for durable storage.

Which one should I run for a homelab backup?

If your backup needs versioning, immutability (Object Lock), or IAM-scoped keys, run RustFS — it ships all three and is Apache 2.0. If you want geo-distributed copies across secondhand machines, accept AGPL-3.0, and only need put/get plus presigned URLs, Garage is lighter and built exactly for that. Match the missing-feature list to your client before committing.

Sources

Garage quick start (commands, ports, single-node): https://garagehq.deuxfleurs.fr/documentation/quick-start/
Garage S3 compatibility status (versioning, Object Lock, SSE, replication, ACL missing): https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/
Garage license (AGPL v3) and design goals (1 GB RAM, 16 GB disk, geo-distribution): https://garagehq.deuxfleurs.fr/
Garage source repository and language (Rust, AGPL-3.0): https://git.deuxfleurs.fr/Deuxfleurs/garage
RustFS README (Feature & Status, docker run, default credentials, Apache 2.0): https://raw.githubusercontent.com/rustfs/rustfs/main/README.md

SeaweedFS vs RustFS: When a Tiny Footprint Wins

Ethan Carter — Sat, 19 Sep 2026 01:52:00 +0000

SeaweedFS and RustFS are both Apache 2.0, single-binary stores, but they solve different problems. SeaweedFS is a distributed file system that also speaks S3, with a 16-byte per-object index that lets one master track billions of files. RustFS is a purpose-built, MinIO-compatible S3 object store with a web console. Pick SeaweedFS when you need a POSIX/FUSE mount, an Iceberg lakehouse, or billions of small files; pick RustFS when S3 is your only interface and a drop-in MinIO replacement in one docker run is the goal.

Key stats

	SeaweedFS	RustFS
License	Apache 2.0	Apache 2.0
What one binary serves	S3 + POSIX file system + Iceberg lakehouse	S3-compatible object storage
In-memory index per object	16-byte entry per blob	not published
On-disk metadata per file	40 bytes	not published
1KB-file throughput (1M files, conc 16, single SSD)	15,708 write/s, 47,019 read/s	not published

The SeaweedFS throughput figure is its own README's unscientific single-machine number: 1 million 1KB files at concurrency 16 on a MacBook with an SSD. We cite it as a scale signal, not a hardware review.

What is the real difference between SeaweedFS and RustFS?

SeaweedFS is a simple and highly scalable distributed file system. Its own README says one weed binary "serves an S3 object store, a POSIX file system, and a lakehouse with S3 Tables, all over the same data." Under the hood, a master server tracks volumes (not individual files), volume servers store blobs in append-only volume files, and a filer exposes the data as a POSIX file system, an S3 endpoint, and an Iceberg catalog.

RustFS is narrower by design. It is S3-compatible object storage written in Rust and released under the Apache 2.0 license. Its README's Feature & Status table marks S3 Core, Versioning, Object Lock (WORM), Lifecycle Management, Bucket Replication, Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode as available today, with S3 Tables (Iceberg REST) in preview. There is no POSIX layer and no FUSE mount. The difference is the whole story: SeaweedFS is a file system that also does S3; RustFS is an S3 engine that does one thing.

How small is the metadata footprint, really?

This is where "tiny footprint" stops being marketing. SeaweedFS packs small files into append-only volume files, so there is no per-file inode and no per-file metadata file. The volume server keeps a 16-byte index entry per blob in memory and reads it in a single seek, even for erasure-coded data; on disk it stores 40 bytes of metadata per file. Because the master tracks volumes rather than files, a cluster holding billions of files still has only a few thousand volumes, so the master stays small and never sits in the read path — clients cache the volume-to-server mapping and talk to volume servers directly.

RustFS is a single Rust binary or container. The documented start is one docker run; there is no monitor quorum, no placement groups, no daemon zoo to reason about. We do not publish a per-object index size, and the comparison is not apples-to-apples — RustFS optimizes for S3 semantics, not for billions of tiny files on a single file system. The footprint win is real on different axes: SeaweedFS wins raw metadata scale; RustFS wins operational simplicity for an S3-only team.

How does each one deploy?

SeaweedFS ships as a single weed binary. The fastest path is weed mini, which starts the master, a volume server, the filer, WebDAV, the Iceberg REST catalog, and the Admin UI in one process, and exposes the S3 endpoint at http://localhost:8333:

AWS_ACCESS_KEY_ID=admin \
AWS_SECRET_ACCESS_KEY=secret \
S3_BUCKET=my-bucket \
./weed mini -dir=./data

To grow capacity, point more volume servers at the master:

weed volume -dir=/data -master=<master_host>:9333

RustFS ships as a single binary or a container. The documented start command is:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

That one line serves the S3 API on port 9000 and a console on 9001. For bare metal there is a one-line installer:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any production exposure.

When should you pick SeaweedFS over RustFS?

Reach for SeaweedFS when the workload is not "objects" but "files at extreme scale." Its design target — store billions of files, serve them fast — shows up in the numbers: a 16-byte per-blob index, 40 bytes per file on disk, and a master that tracks volumes, not files. If you are building a content-addressable store, a photo or video library, or an ML feature store with tens of millions of tiny shards, that metadata model is the difference between a master that fits in RAM and one that does not.

SeaweedFS also wins the moment you need a file system or a lakehouse over the same bytes. One weed binary serves an S3 endpoint, a FUSE mount on Linux/macOS/Windows, WebDAV, SFTP, HDFS, and an Iceberg REST catalog, all addressing the same data. If a data scientist wants to ls the lakehouse while an app writes through S3, SeaweedFS does both without a sync job. RustFS does not offer a FUSE/POSIX mount, so that requirement alone steers you to SeaweedFS.

Where does RustFS win?

RustFS wins on the axis most S3 teams actually care about: a clean, MinIO-compatible S3 endpoint with the least operational surface. One docker run, no quorum, no volume-server choreography. The feature set is now broad — Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, OIDC/SSO, Server-Side Encryption, and Distributed Mode are all marked available in the README, with MinIO On-Disk Compatibility in preview. If your tools already speak MinIO's S3 dialect, RustFS is a drop-in.

The Apache 2.0 license is the same as SeaweedFS's, so licensing is not the differentiator. The differentiator is focus: RustFS spends its complexity budget on S3 correctness and the web console, not on a file system layer. For a team whose only contract with storage is the S3 API, that focus means fewer knobs to misconfigure and a shorter path to a working endpoint.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of September 2026. SeaweedFS rows come from its GitHub README; RustFS rows come from its README Feature & Status table (2026-09-18). No benchmarks, no marketing.

Capability	SeaweedFS	RustFS
S3-compatible API	✅	✅ (S3 Core)
POSIX / FUSE file system	✅ (FUSE on Linux/macOS/Windows)	❌
Iceberg / lakehouse over same data	✅ (S3 Tables)	🧪 Preview (S3 Tables, Iceberg REST)
Erasure coding	✅ (warm data, background)	❌
Bucket / site replication	✅ (volume-level)	✅ (Bucket + Site Replication)
Web console / Admin UI	✅	✅ (Web Console)
Kubernetes	CSI driver + Operator	✅ (Helm Charts)
Single binary	✅ (weed)	✅
License	Apache 2.0	Apache 2.0
Implementation	Go	Rust

Two rows deserve a call-out. RustFS lists S3 Tables (Iceberg REST) as Preview, so if a mature lakehouse over the same bytes is the requirement, SeaweedFS is ahead today. And SeaweedFS's erasure coding runs on warm data in the background, so writes never pay the encoding cost up front — a model RustFS does not yet offer in its feature table.

What can RustFS not replace yet?

Be clear about the boundaries before you commit. RustFS's README has no FUSE/POSIX mount, so if you need a file system interface or a mounted lakehouse, SeaweedFS (or CephFS) is the answer today. RustFS's S3 Tables (Iceberg REST) is in preview, so a production lakehouse over RustFS data is not a promise the project makes yet. And while Distributed Mode is now marked available, a RustFS cluster at SeaweedFS-scale file counts is a different beast — SeaweedFS was built from the start to keep the master small at billions of files, and RustFS does not publish an equivalent per-object index claim.

None of this is a knock on RustFS. It is a scope statement: RustFS is an S3 engine, and it is good at being one. Pushing it into the file-system or exabyte-file-count role is the wrong tool for that job.

How do you choose between them?

Use this rule of thumb: if the words "file system," "FUSE," or "lakehouse" appear in the ticket, start with SeaweedFS. If only "S3" appears, and the real constraint is headcount, time, or MinIO compatibility, start with RustFS.

A split is legitimate and common. Run SeaweedFS for the data-lakehouse and file-serving layer where billions of files and a POSIX mount pay off, and run RustFS in front of the S3-facing applications where a MinIO-compatible endpoint and a web console are the whole ask. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job — SeaweedFS dragged into a pure-S3, small-team role carries a file system you will never mount, and RustFS pushed into a POSIX role it does not implement becomes a custom FUSE you have to build yourself.

FAQ

Is SeaweedFS a drop-in replacement for MinIO?

Not directly. SeaweedFS implements S3 as one of several interfaces on top of its distributed file system, so most S3 clients work, but it is not built to mirror MinIO's S3 dialect command for command. RustFS, by contrast, lists MinIO On-Disk Compatibility as a preview feature and targets the MinIO S3 dialect as its primary surface. If your only requirement is S3 and you want MinIO-compatible tooling, RustFS is the closer fit.

Does SeaweedFS support erasure coding?

Yes. SeaweedFS replicates hot data for speed and applies erasure coding to warm data in the background, so writes never pay the encoding cost up front. Volume servers replicate or erasure-code at the volume level, and the in-memory index stays 16 bytes per blob even for erasure-coded data. RustFS's feature table does not list erasure coding today.

Does RustFS have a FUSE mount?

No. RustFS's README Feature & Status table has no FUSE/POSIX mount, so there is no file system interface. If you need to mount storage as a local directory or expose a lakehouse over the same bytes, SeaweedFS provides a FUSE mount on Linux, macOS, and Windows. For pure S3 object access, the missing mount is irrelevant.

Which one scales to more files?

SeaweedFS is designed for it: the master tracks volumes rather than files, so a cluster with billions of files holds only a few thousand volumes and the master stays small. RustFS is a capable S3 object store, but it does not publish a comparable per-object index claim, and its strength is S3 semantics at operationally simple scale rather than billions of tiny files on one file system.

Are both projects Apache 2.0?

Yes. Both SeaweedFS and RustFS are released under the Apache License 2.0, a permissive license with no copyleft. If you run either as a service the practical difference is small; if you embed the engine in a shipped product, Apache 2.0 spares you the copyleft questions that a GPL/LGPL project would raise.

RustFS 1.0 Is GA After 2.5 Years: What's Actually Production-Ready, and What Isn't

Ethan Carter — Thu, 17 Sep 2026 14:08:01 +0000

RustFS 1.0.0 reached general availability on September 16, 2026. The core S3-compatible object storage engine is stable and supported for production. RustFS is a distributed, Apache 2.0-licensed object store written in Rust, now with 32,000+ GitHub stars and 2.7 million+ deployed instances worldwide.

Key stats

Metric	Value
GitHub stars	32,000+
Docker Hub pulls	10,000,000+
Instances deployed	2.7 million+
Contributors	160+
First commit to GA	Feb 2024 to Sep 2026 (2 yr 7 mo)
License	Apache 2.0

RustFS 1.0.0 GA: what actually shipped

RustFS 1.0.0 is the first generally available release of the project. "GA" here means one specific thing: the core object storage engine (PUT, GET, multipart upload, versioning, replication, and the S3 API surface) is stable and supported for production traffic. The team spent two years and seven months getting there, from the first commit in February 2024 to open sourcing in July 2025, then Beta in April 2026 and RC in August 2026.

What shipped is not a rewrite of someone else's storage engine. RustFS is built from scratch in Rust, which the team chose for memory safety and concurrency under heavy write loads. GA matters because it lets you stop evaluating RustFS and start running it where downtime costs money. The release also closes a promise the project made at open source: a self-hosted, S3-compatible store with no commercial license gate on the core engine.

What is RustFS, and why Rust?

RustFS is a distributed object storage system that speaks the S3 API. You point an existing S3 client (the AWS CLI, MinIO's mc, rclone, or your application's SDK) at a RustFS endpoint and it behaves like S3. Under the hood it is written in Rust, which is the part that actually matters to operators.

Rust brings two things to a storage engine. First, memory safety without a garbage collector, so there is no stop-the-world pause while you serve a hot bucket. Second, fearless concurrency: the borrow checker lets the team write parallel I/O paths that do not corrupt data under load. The practical result the project cites is steady throughput on PUT-heavy workloads. If you have run a Go-based store and fought GC tail latency, that is the class of problem RustFS targets. The trade-off is a smaller ecosystem and fewer prebuilt integrations than the incumbents.

Is RustFS production-ready today?

For the core object storage path, yes. RustFS 1.0.0 lists single-node and distributed deployment as available, along with versioning, object lock (WORM), bucket and site replication, durability through bitrot protection, and server-side encryption. Those are the features most teams actually turn on in production.

What I would not do on day one is assume RustFS matches MinIO or Ceph feature for feature. The RustFS feature matrix marks S3 Tables (Apache Iceberg) as preview, not GA, and the surrounding tooling (backup operators, third-party connectors, managed offerings) is thinner than the incumbents. My recommendation: run RustFS for a real workload you can measure, not just a hello-world bucket. Stand up a single node, point rclone at it, and watch p99 latency under your own traffic before you trust it with the primary copy of anything.

What can RustFS do that matters for storage teams?

RustFS covers the S3 surface most applications need: multipart upload for large objects, versioning, object lock for compliance hold, and S3 Select for server-side filtering. On access it is not S3-only. The GA release ships the Swift API, WebDAV, FTPS, and SFTP alongside the S3 endpoint, which matters if you are retiring an old Swift cluster or want file-style access without a separate gateway.

On the control plane, RustFS gives you IAM policies, OIDC single sign-on, a RustFS KMS, STS tokens, and audit logging. For operations there are event notifications, health inspection, multi-tenancy, and Helm charts for Kubernetes. Data movement is handled by bucket replication and site replication, plus ILM tiering to a remote S3 target. If your stack already speaks S3, the migration is mostly a DNS and credential change.

How do you run RustFS 1.0.0?

The fastest path is a single container. The official image is rustfs/rustfs:latest, and the README's verified command maps the API port 9000 and console port 9001, with volumes for data and logs:

docker run -d -p 9000:9000 -p 9001:9001 \
  -v $(pwd)/data:/data -v $(pwd)/logs:/logs \
  rustfs/rustfs:latest

[sourced from RustFS GitHub README, NOT EXECUTED IN CI]

Default credentials are rustfsadmin / rustfsadmin (change them before this touches a network). The web console is at http://localhost:9001. For a bare-metal box there is also a one-click script: curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh. Kubernetes users get Helm charts. For a quick distributed test, docker compose with the project's docker-compose-simple.yml brings up multiple nodes. Pick the method that matches where your workload already runs.

Where does RustFS still trail MinIO and Ceph?

Honesty first: RustFS is younger. Ceph has a decade of production deployments and MinIO has a far larger install base and connector ecosystem. On mixed and read-heavy workloads RustFS is still closing the gap. Our beta.10 benchmarks showed RustFS ahead on PUT-heavy patterns, but read-mixed tuning is where the incumbents remain ahead today.

S3 Tables (Iceberg) is in preview, so if your roadmap depends on in-storage analytics tables, that is not GA yet. The third gap is operational muscle memory: fewer community runbooks, fewer third-party backup tools with first-class RustFS support, and a smaller talent pool who have run it at scale. None of that is a reason to skip RustFS, but it is a reason to validate on your own workload and keep a rollback path. RustFS is betting that Rust's safety and write-path concurrency outweigh ecosystem lag over time.

What's next after 1.0.0 GA?

GA is a start line, not a finish. The RustFS team's stated next steps are two: harden what shipped and push further into AI infrastructure. On hardening, the plan is to keep absorbing feedback from production users and close rough edges in replication, tiering, and the console.

On the AI side, RustFS is adding S3 Vectors support and framing a RustFS 2.0 built around AI data centers, object storage that feeds training and inference pipelines directly. That direction fits the project's positioning as AI-native storage rather than a generic S3 bucket. For now, treat S3 Vectors and 2.0 as roadmap, not something you can deploy today. If vector storage or AI data lakes are on your 2027 plan, RustFS is worth watching, but build this year's system on what is GA.

Should you migrate from MinIO to RustFS now?

If you are greenfield or adding a new storage tier, RustFS 1.0.0 is reasonable to adopt for production workloads that are S3-centric and write-heavy. The S3 compatibility means your existing clients work unchanged, and the Apache 2.0 license means no commercial surprise.

If you already run MinIO or Ceph in production with mature tooling, do not rip it out. Migrate one workload, measure it, and expand only if RustFS earns it on your numbers. The lowest-risk first project is a secondary tier (backups, artifacts, or a staging bucket), where a write-path latency win shows up without betting the primary data path. RustFS is not asking you to trust a promise; it is asking you to run a benchmark on your own traffic. That is the right bar for a 1.0.

FAQ

Is RustFS compatible with the S3 API?

Yes. RustFS implements the S3 API, so existing S3 clients such as the AWS CLI, MinIO's mc, and rclone work against a RustFS endpoint without code changes. Multipart upload, versioning, and object lock are supported.

Can RustFS replace MinIO?

For S3-centric, write-heavy workloads, RustFS 1.0.0 is production-ready and a viable alternative. It does not have feature parity yet. MinIO's ecosystem and read-mixed performance lead. Migrate one workload first and measure.

Is RustFS free and open source?

Yes. RustFS is licensed under Apache 2.0, with the source on GitHub. There is no commercial license gate on the core object storage engine.

How do I install RustFS?

The fastest method is a single Docker container using the official rustfs/rustfs:latest image (API on 9000, console on 9001). A one-click script (install_rustfs.sh) covers bare metal, and Helm charts cover Kubernetes.

Does RustFS run distributed, and is it production-ready?

Distributed deployment is listed as available in RustFS 1.0.0. Single-node mode is the simplest production start. Durability comes from bitrot protection, versioning, and bucket or site replication. S3 Tables (Iceberg) remains in preview.

Originally published at the RustFS blog.

Ceph vs RustFS: When to Use What (2026)

Ethan Carter — Wed, 09 Sep 2026 12:15:00 +0000

Ceph vs RustFS: When to Use What (2026)

Ceph and RustFS both speak S3, but they solve different problems. Ceph is a unified object, block, and file platform that scales to exabytes and carries real operational weight. RustFS is a single-binary, S3-focused object store for teams that want MinIO-level simplicity without a storage team. Choose Ceph when one cluster must serve block, object, and file. Choose RustFS when the workload is S3 objects and you want it live today.

What is the real difference between Ceph and RustFS?

Ceph is a distributed object, block, and file storage platform. Its documentation states it uniquely delivers object, block, and file storage in one unified system, reaching from petabytes to exabytes of data across thousands of clients. Under the hood sits RADOS (Reliable Autonomic Distributed Object Store), with RBD for block, RGW for the S3/Swift gateway, and CephFS for POSIX file access layered on top.

RustFS is narrower by design. It is S3-compatible object storage written in Rust and released under the Apache 2.0 license. The project's README lists S3 Core Features, Versioning, Bucket Replication, Event Notifications, Bitrot Protection, Single Node Mode, Multi-Tenancy, Keystone Auth, and the Swift API as available today. There is no block layer and no file layer. That scope difference is the whole story: Ceph is the generalist that spans three storage paradigms, RustFS is the S3 specialist that does one thing with a small operational surface.

How does each one deploy?

Ceph is a cluster of cooperating daemons. The official cephadm tool bootstraps a tiny cluster on a single node with one Monitor and one Manager, then you expand it by adding hosts and provisioning OSDs (object storage daemons), MDS (metadata servers, needed for CephFS), and RGW (the S3 gateway). Each daemon type carries its own configuration, failure modes, and tuning surface, and a production deployment normally spans several nodes with redundant monitors. This is not a flaw; it is the cost of unifying three storage paradigms in one system.

RustFS ships as a single binary or a container. The documented start command is:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[sourced from https://github.com/rustfs/rustfs/blob/main/README.md, NOT EXECUTED IN CI]

That one line serves the S3 API on port 9000 and a console on 9001. For bare metal there is a one-line installer:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

[sourced from https://github.com/rustfs/rustfs/blob/main/README.md, NOT EXECUTED IN CI]

Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any production exposure.

When should you pick Ceph over RustFS?

Reach for Ceph when a single storage layer has to do more than one job. If your Kubernetes cluster needs RBD block volumes for databases, an S3 endpoint for applications, and a CephFS mount for shared files, all from the same hardware, Ceph is one of the few open-source systems that covers all three. Its RADOS gateway implements a large subset of the Amazon S3 RESTful API, so most S3 clients work against it with only minor differences in canned ACLs.

Ceph also earns its keep at serious scale. The project documents petabyte-to-exabyte deployments and a mature ecosystem with multiple vendors offering paid support. If you already employ storage engineers who know CRUSH maps, placement groups, and OSD recovery, Ceph's operational model is a feature rather than a tax. Choose it when scope and scale outrun simplicity, and when block or file access is a hard requirement rather than a nice-to-have.

Where does RustFS win?

RustFS wins on the axis most small and mid-size teams actually care about: time to a working S3 endpoint. One binary, one docker run, no monitor quorum to reason about, no placement groups to size. The Feature & Status table is honest about what is ready: S3 Core, Versioning, Bucket Replication, Event Notifications, and Bitrot Protection are marked available, which covers the common object-storage patterns, backup targets, and application data lakes.

The Apache 2.0 license matters more than people admit. Ceph is mainly LGPL 2.1/3.0 with some GPL and BSD components; if you ever embed the engine rather than run it as a service, Apache 2.0 spares you copyleft questions. And because RustFS is written in Rust, there is no garbage collector in the request path, which keeps tail latency easier to reason about under mixed workloads. For teams that just need S3 without becoming storage experts, that combination is the point.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of August 2026. No benchmarks, no marketing. Ceph's RGW feature rows come from the official RADOS Gateway S3 API reference; RustFS's come from its GitHub README Feature & Status table.

Capability	Ceph (RGW)	RustFS
S3 buckets, objects, multipart	✅	✅ (S3 Core)
Object Versioning	✅	✅
Bucket Lifecycle	✅	🚧 Under Testing
Bucket Replication	⚠️ Partial (across zones only)	✅
Bucket / Object Notifications	✅	✅ (webhook)
Block storage (RBD)	✅	❌ (object only)
File storage (CephFS)	✅	❌
K8s deployment	Rook operator / cephadm	✅ Helm Charts
Console / UI	ceph-mgr dashboard	✅ web console
License	LGPL 2.1/3.0 (mixed)	Apache 2.0
Implementation	C++ daemons	Rust (single binary)

Two rows deserve a call-out. RustFS lists Lifecycle Management as Under Testing, so do not promise automated tiering yet. Ceph's Bucket Replication is marked partial and permitted only across zones in RGW. Both gaps are real and both are documented by the projects themselves, not by us.

What can RustFS not replace yet?

Be clear about the boundaries before you commit. RustFS does not provide block storage or a POSIX file system; the README's Feature & Status table has no RBD and no FUSE/POSIX mount, and no Object Lock. If your requirement is WORM retention or a mounted file system, Ceph or a purpose-built tool is the answer today.

Distributed Mode is also Under Testing in RustFS. Single-node is a supported, solid story; multi-node production at petabyte scale is not a promise the project makes yet. Its ecosystem is younger than Ceph's by more than a decade, so you will find fewer third-party integrations and fewer war-story blog posts. I would not pitch RustFS as a drop-in for an existing exabyte Ceph fabric. For S3 object workloads that fit a single node or a modest cluster, it is a different conversation and one worth having.

How do you choose between them?

Use this rule of thumb. If the words "block", "file system", and "object" all appear in the same ticket, start with Ceph. If only "S3" appears and the real constraint is headcount or time, start with RustFS.

A hybrid is legitimate and common. Run Ceph for the infrastructure layer (RBD for stateful workloads, CephFS for shared files) and RustFS in front of the S3-facing applications where operational simplicity pays off. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job. Ceph dragged into a pure-S3, small-team role becomes expensive overhead. RustFS pushed past its single-node comfort zone becomes a bet on an untested distributed path. Match the tool to the layer and both stay boring in the good way.

FAQ

Is RustFS a replacement for Ceph?

Partially, and only for the object-storage slice. RustFS can replace the RGW (S3 gateway) portion of a Ceph stack for S3-focused workloads, with far less operational overhead. It cannot replace RBD block storage or CephFS file storage. If your stack needs block or file from the same cluster, Ceph stays the right call.

Is Ceph harder to operate than RustFS?

Yes, materially. Ceph runs multiple daemon types (MON, OSD, MDS, MGR, RGW), each with its own config and failure modes, and a real cluster spans several nodes. RustFS is one binary or one container; the documented start is a single docker run. The trade-off is scope: Ceph covers three storage paradigms, RustFS covers object storage with less to manage.

Does Ceph support the S3 API?

Yes. Ceph's RADOS Gateway implements a large subset of the Amazon S3 RESTful API, plus a subset of the OpenStack Swift API, with its own user management. Most S3 clients work against it, though Ceph notes differences in canned ACLs and a partial Bucket Replication (across zones only). It is not a 100 percent S3 clone, but it covers the common operations.

Can RustFS run on a single node?

Yes. RustFS lists Single Node Mode as an available feature, and the documented quick start is a single docker run on one host serving S3 on port 9000 with a console on 9001. Multi-node distributed mode is Under Testing, so production scale-out is not a promise yet. For many S3 object workloads a single node is enough.

What license is each project under?

RustFS is Apache 2.0, a permissive license with no copyleft. Ceph is mainly dual-licensed under LGPL 2.1 or 3.0, with some GPL and BSD components in the tree. If you run either as a service the practical difference is small; if you embed the engine in a shipped product, Apache 2.0 is the more flexible choice.

Wasabi vs RustFS: S3-Compatible Cost Showdown

Ethan Carter — Mon, 07 Sep 2026 12:14:10 +0000

Disclosure: I work on RustFS. Every comparison number below is pulled from a primary source and dated 2026-08-21, so check the Sources table before you budget.

Wasabi wins on simplicity: one flat $7.99 per TB per month with no egress or API fees, ideal when you have no hardware and want 16 regions by this afternoon. RustFS wins on raw dollars per TB once you already own or colocate servers, because storage becomes disks you bought instead of a recurring bill. The real question is not "which is cheaper" but "who carries the hardware, power, and operations."

Key Stats

Metric	Wasabi	RustFS
Storage price	$7.99 / TB / mo (NA, EMEA, APAC)	$0 / TB / mo — you supply disks
Egress	Free under fair-use (≤ stored volume / mo)	Free — it is your own network
API requests	Free under fair-use	Free
Minimum term	90-day retention charge	No minimum
Minimum commitment	1 TB billed even if unused	None
Storage regions	16 regions	Wherever you place the node
Durability claim	11 nines (99.999999999%)	Your redundancy
Object Lock (WORM)	Yes	Not in README Feature table
License	Proprietary	Apache 2.0

What This Cost Showdown Actually Measures

A cost showdown between a cloud and a self-hosted object store is not a single price tag. It is the recurring opex of renting capacity versus the capital plus operations of owning it. Wasabi quotes a flat rate per terabyte per month and bills nothing extra for moving data in, out, or for API calls, within fair-use limits. RustFS is software you run; the bill is the servers, disks, power, and the engineer who gets paged. The comparison only makes sense once you know your dataset size, how long it lives, and whether you already pay for a rack somewhere. Everything below keeps those three variables explicit so the numbers stay honest.

How Does Wasabi Price Storage?

Wasabi's pay-as-you-go Hot Cloud Storage lists at $7.99 per TB per month across North America, EMEA, and APAC, with no separate charge for ingress, egress, or API requests (Wasabi pricing page, checked 2026-08-19). That flat rate is the headline and the reason backup vendors default to it. For 100 TB, the math is 100 × $7.99 = $799 per month, or $9,588 per year, before any minimums. Reserved-capacity terms in 1, 3, or 5-year blocks lower that further if you can commit capacity up front. What you do not see on the front page is the small print that decides whether your real bill matches the calculator, and that is where the next section lives.

Where Does the Wasabi Bill Actually Bite?

Three clauses in the pay-as-you-go FAQ change the number for real workloads. First, a 90-day minimum storage duration: delete an object before 90 days and you still pay "Timed Deleted Storage" for the remaining days, so high-churn data costs more than its live size suggests. Second, a 1 TB monthly minimum means storing 200 GB still bills a full terabyte. Third, objects under 4 KB are billed as 4 KB, and enabling versioning or bucket logging grows your stored bytes and your bill. Egress is free only while monthly downloads stay at or below your active stored volume; above that, Wasabi calls it "not a good fit" and can limit or suspend service.

What Does RustFS Actually Cost to Run?

RustFS charges nothing per gigabyte because it is software, not capacity. Your cost is hardware. A 24 TB enterprise HDD listed for roughly $400 to $600 in 2025 (market retail range), so raw capacity capital lands near $17 to $25 per TB one-time. Amortize that drive over a five-year life and disk capEx is about $3.40 to $5 per TB per year, before you account for mirrors or backups. Power, cooling, and rack space are extra, but your bandwidth is free because the data never leaves your network through a metered exit. The trade you accept is operations: patching, monitoring, and recovery are on you, not a support ticket.

RustFS Feature Reality Check: What You Get, What You Don't

RustFS's README Feature & Status table is the only honest source for what the software does today. Marked Available: S3 Core, Upload/Download, Versioning, Bucket Replication, Logging, Event Notifications, K8s Helm Charts, Keystone Auth, Multi-Tenancy, Swift API, Bitrot Protection, and Single Node Mode. Under Testing: Lifecycle Management, Distributed Mode, and RustFS KMS. The table does not list Object Lock, erasure coding, or a FUSE mount, so do not plan a WORM archive or a POSIX filesystem on RustFS yet. Read it plainly: single-node RustFS is ready now, multi-node redundancy is not a production promise. I work on RustFS, so weigh that against the README, not my enthusiasm.

How Do the Two Compare on Cost-Relevant Features?

Dimension	Wasabi	RustFS
Footprint	16 managed regions	Your hardware, your site
Durability	11 nines (vendor claim)	What you build (mirror or backup)
Immutable retention	Object Lock: WORM, legal hold	Not available today
Lifecycle / tiering	Yes	Under Testing
Egress model	Free under fair-use	Your network, no meter
Lock-in	Proprietary account	Apache 2.0, you own it

The features that move cost are durability and lifecycle. Wasabi hands you 11 nines and Object Lock out of the box; RustFS hands you the software and expects you to supply redundancy until Distributed Mode ships. If compliance retention matters, that gap is the whole decision today.

When Should You Pick Wasabi?

Pick Wasabi when you have no servers and no interest in running storage. It is the faster path to 16 regions, data-residency choices across North America, Europe, and Asia, and immutable backups through Object Lock without standing up anything. The flat $7.99 per TB per month is easy to defend in a budget review because there is no egress line item to explain. For backup, DR, and active archive where someone else should carry the pager, the recurring fee buys absence of operational load. If your team's core product is not storage, renting it is usually the cheaper career move.

When Does RustFS Win on Total Cost?

RustFS wins when the dataset is large, stable, and already sitting on hardware you pay for. At 100 TB, Wasabi recurs at $9,588 per year; the same capacity as disks you own amortizes to a few dollars per TB per year in capital, and your egress is free because the traffic never crosses a metered provider link. The break-even is earlier the more you download, since Wasabi's fair-use cap bites exactly where RustFS has no meter. The catch is durability: until Distributed Mode leaves testing, mirror to a second node or back up to another S3 target yourself. You trade a recurring fee for owned infrastructure and Apache-2.0 freedom from lock-in.

How Do You Point Your Tools at Each?

Both speak the S3 API, so the same SDKs and CLI habits apply. For Wasabi, the MinIO Client connects with the commands from Wasabi's official doc:

# Wasabi via MinIO Client (mc) — verbatim from docs.wasabi.com
# [sourced from https://docs.wasabi.com/how-do-i-use-minio-client-with-wasabi, NOT EXECUTED IN CI]
wget https://dl.minio.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc config host add https://s3.wasabisys.com   # prompts for access/secret key
./mc ls

And the AWS CLI, also verbatim from Wasabi's doc:

# [sourced from https://docs.wasabi.com/v1/docs/how-do-i-use-aws-cli-with-wasabi, NOT EXECUTED IN CI]
aws s3 ls --endpoint-url=https://s3.wasabisys.com

RustFS starts with the README's docker command:

# [sourced from https://github.com/rustfs/rustfs, NOT EXECUTED IN CI]
docker run -d -p 9000:9000 -p 9001:9001 \
  -v $(pwd)/data:/data \
  -v $(pwd)/logs:/logs \
  rustfs/rustfs:latest

Default credentials are rustfsadmin / rustfsadmin; change them before exposing port 9000.

FAQ

Is Wasabi really cheaper than AWS S3?

At typical access volumes, yes. Wasabi's own comparison example sizes 100 TB stored with 20 TB downloaded per month at about $818 versus roughly $4,097 for AWS S3, once egress is included. The gap comes almost entirely from Wasabi's missing egress and API line items. Whether that beats self-hosting depends on whether you already own the hardware; the cloud comparison only holds against other clouds.

Does RustFS charge per gigabyte?

No. RustFS is open-source software under Apache 2.0; you run it on your own disks and pay no per-GB fee and no egress fee, because the data stays on your network. Your cost is the servers, drives, power, and the operations work to keep them healthy. Treat it as capital and labor, not a metered bill.

Can RustFS replace Wasabi for immutable backups?

Not today for compliance retention. RustFS's README Feature & Status table does not list Object Lock, while Wasabi and Backblaze B2 both offer S3 Object Lock with WORM and legal hold. If you need ransomware-immutable, audit-ready retention, use Wasabi, B2, or Ceph now and keep RustFS for analytical and batch object workloads where immutability is not the requirement.

What is Wasabi's 90-day minimum storage policy?

Objects deleted before they have been stored 90 days incur a "Timed Deleted Storage" charge for the remaining days, comparable to minimum-duration policies on some hyperscaler tiers. It mainly hurts high-churn workloads where data lives for days or weeks. For long-lived archives it is invisible, because the data stays past the threshold anyway.

Is RustFS production-ready?

Single-node RustFS is marked Available and is a reasonable place to start today. Distributed Mode, Lifecycle Management, and RustFS KMS are still Under Testing, so multi-node redundancy and automated tiering are not production promises yet. Run a single node and mirror or back it up to a second S3 target until Distributed Mode ships; do not bet a WORM archive on it yet.

Sources

All claims checked against these primary sources on 2026-08-21.

Claim	Source
$7.99/TB/mo, no egress/API fees, 1 TB monthly min, 90-day retention, 4 KB min object, fair-use egress cap	https://wasabi.com/pricing , https://wasabi.com/pricing/faq
16 storage regions, S3 + IAM API, Object Lock, 11 nines durability	https://wasabi.com/cloud-object-storage
Wasabi mc commands	https://docs.wasabi.com/how-do-i-use-minio-client-with-wasabi
Wasabi AWS CLI commands	https://docs.wasabi.com/v1/docs/how-do-i-use-aws-cli-with-wasabi
RustFS Feature & Status, docker run, default creds, Apache 2.0, `1.0.0-rc.3`	https://github.com/rustfs/rustfs
Enterprise 20TB+ HDD retail range $400-$600 (2025), used as raw-capacity cost anchor	https://www.accio.com/business/hdd-failure-rate (market summary, "Enterprise High-Capacity HDDs (20TB+)" section)

Self-Hosting S3-Compatible Storage on Bare Metal

Ethan Carter — Thu, 27 Aug 2026 12:52:00 +0000

You self-host S3-compatible storage on bare metal by installing a single Rust binary on a Linux server and pointing any S3 client at it. RustFS installs with one script, listens on port 9000 (S3 API) and 9001 (console), and is Apache 2.0 licensed. Single-node mode is production-ready today; multi-node clustering is still under testing.

Every command below is copied verbatim from the official source cited beside it. This sandbox has no Docker daemon, so none of the commands were executed here; they are marked accordingly.

Key Stats

Fact	Source
RustFS installs with one command and runs as a systemd service on x86_64 or aarch64 Linux	RustFS docs (Linux quick-start)
Default S3 API port is 9000; console port is 9001	RustFS GitHub README
Default credentials are `rustfsadmin` / `rustfsadmin` and must be changed	RustFS README + docs
RustFS is Apache 2.0 licensed and S3-compatible	RustFS GitHub README
Single-node mode is production-ready; distributed mode is still under testing	RustFS README Feature & Status

What is self-hosted S3-compatible storage?

A self-hosted S3-compatible storage server is a program you run on your own hardware that speaks the Amazon S3 API. Applications using AWS SDKs, the aws CLI, or MinIO's mc can talk to it without code changes, because the bucket, object, and credential model matches S3. The difference from a cloud bucket is ownership: the disks, the network path, and the uptime are yours.

RustFS is one such server, written in Rust and licensed under Apache 2.0. It exposes the S3 API on port 9000 and a web console on 9001, and it stores objects on the local filesystem. Because it is S3-compatible, the same client code that targets AWS S3 also targets a RustFS node. That compatibility is the whole point of self-hosting here: you get an S3 endpoint without renting one.

Why run object storage on bare metal?

Running object storage on bare metal means installing the server directly on a Linux machine instead of in a container or a managed cloud. The appeal is control. Your data stays on disks you own, in a network you define, and the cost is the hardware plus your time rather than a per-gigabyte cloud bill that grows with every upload.

The trade-off is real and worth stating plainly. You own the failure modes too. A single disk dies, the server reboots, the certs expire, and that is now your incident, not a vendor's. Bare metal suits teams that already run Linux servers, care about data residency, or have steady high-volume storage that makes a cloud bill painful. It is a poor fit if you want zero operational responsibility. Self-hosting trades a recurring fee for recurring attention.

What you need before you start

Minimum: a Linux server with systemd on x86_64 or aarch64, root or sudo access, the unzip utility, outbound network access to pull the package, and open ports 9000 (S3 API) and 9001 (console) so clients and the browser console can reach the service.

Plan the data disk before you install. The quick-start runs RustFS in single-node, single-disk (SNSD) mode, which stores everything under /data/rustfs0 with no redundancy. That is fine for evaluation and dev, but a single disk has no spare. For anything you would hate to lose, attach a second disk, schedule filesystem or object-level backups, or wait for distributed mode to leave testing. The installer lets you change the data path and ports at install time, so decide the layout now rather than after data accumulates.

How do you install RustFS on bare metal?

Run the official one-line installer. It downloads the binary, installs it to /usr/local/bin/rustfs, registers a systemd service, starts it, and stores data under /data/rustfs0 by default:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

[sourced from RustFS docs (Linux quick-start), NOT EXECUTED IN CI]

After it finishes you get a summary with the service port (9000), console port (9001), and data directory, plus a security warning to replace the default credentials. Open http://<server-ip>:9001 in a browser to reach the console, then log in with the access and secret keys you set. The same server answers S3 API calls on 9000. From here you can create a bucket in the console or from the command line, which the next sections cover.

How do you run RustFS with Docker instead?

If you prefer a container, the Docker image gives you the same server with no systemd dependency. The official command maps both ports, persists data and logs to local volumes, and detaches:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[sourced from RustFS GitHub README, NOT EXECUTED IN CI]

Do not drop the -p or -v flags. Without -p 9000:9000 the S3 API is unreachable from the host, and without the volume mounts a container restart loses everything under /data and /logs. The image tag :latest tracks the newest build; pin a specific release like 1.0.0-rc.2 if you need reproducibility. Podman users run the equivalent with :Z,U on the volume mounts for SELinux. Either path lands you at the same console on 9001 and the same S3 endpoint on 9000.

How do you point an S3 client at your server?

RustFS ships its own CLI, rc, which handles both object operations and admin checks. After installing it, register an alias that stores the endpoint and keys, then verify the server is alive:

rc alias set local http://localhost:9000 <your-access-key> <your-secret-key>
rc ping local
rc ready local

[sourced from RustFS docs (rc CLI), NOT EXECUTED IN CI]

Create a bucket and push a file the same way any S3 client would:

rc bucket create local/my-bucket
rc object copy /path/to/hello.txt local/my-bucket/hello.txt
rc object list local/my-bucket

[sourced from RustFS docs (rc CLI), NOT EXECUTED IN CI]

Because the API is S3-compatible, AWS SDKs, the aws CLI with --endpoint-url, and MinIO's mc all work against port 9000 with the same keys. You are not locked into rc; it is just the native client. Point your application at http://<server-ip>:9000 and the buckets behave like any other S3 endpoint.

What RustFS does today, and what it doesn't

RustFS today is a solid single-node S3-compatible server. The features marked available in the project's own status table include S3 core operations, upload/download, versioning, logging, event notifications, bucket replication, bitrot protection, single-node mode, multi-tenancy, Keystone auth, and the Swift API. If your workload fits one node, those are real and usable now.

Be honest about the gaps. Distributed mode, lifecycle management, and RustFS KMS are still under testing, so multi-node erasure-coded clusters and automated tiering or expiry are not production promises yet. There is no Object Lock, no FUSE or POSIX mount, and no built-in erasure coding in the current release. RustFS does not ship a turnkey multi-region setup. For a single bare-metal node serving S3 to your apps, that is plenty. If you need cross-node redundancy or compliance retention today, plan around those limits rather than assuming they exist.

How do you keep a bare-metal server healthy?

First, change the default credentials. The installer leaves placeholder keys in /etc/default/rustfs; set your own access and secret keys there, then restart:

RUSTFS_ACCESS_KEY=<your-access-key>
RUSTFS_SECRET_KEY=<your-secret-key>   # e.g. output of: openssl rand -base64 24

sudo systemctl restart rustfs
sudo systemctl status rustfs --no-pager

[sourced from RustFS docs (Linux quick-start), NOT EXECUTED IN CI]

Keep the service under systemd so it restarts on boot, and watch systemctl status rustfs for the active (running) line. Because single-node mode has no redundancy, back up the data directory or replicate the bucket to a second site on a schedule you trust. Check disk space before it is gone, since a full volume stops writes. None of this is exotic, but it is yours to do. The console on 9001 shows bucket and object state when you want a visual check.

FAQ

Is RustFS free to self-host?

Yes. RustFS is licensed under Apache 2.0, so you can run it on your own hardware at no license cost. You pay only for the server, disks, and your time to operate it.

What are the default RustFS credentials?

The default access key and secret key are both rustfsadmin. The installer warns you to replace them, and if you leave them unset the server falls back to those same defaults. Set RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY in /etc/default/rustfs and restart the service before exposing the port.

Can RustFS replace MinIO on a single Linux server?

For single-node S3-compatible needs, yes. Both speak the S3 API, so existing client code and SDKs work against either. RustFS is written in Rust, licensed Apache 2.0, and ships its own rc CLI alongside the standard S3 interface.

Does RustFS support multi-node distributed storage?

Distributed mode is listed as under testing in the project's Feature & Status table, so it is not a production promise yet. Single-node mode is available and production-ready today. If you need cross-node erasure coding now, that gap should factor into your decision.

What port does RustFS use?

The S3 API listens on port 9000 and the web console on 9001. Both must be reachable: 9000 for client and application traffic, 9001 for the browser console.

Build a Data Lake on S3-Compatible Storage

Ethan Carter — Tue, 25 Aug 2026 11:50:00 +0000

A data lake on S3-compatible storage is an object bucket holding open file formats (Parquet, ORC, JSON) partitioned by key, queried in place by engines like DuckDB, Spark, or Trino. You skip a proprietary warehouse, keep the raw data, and control the bill. The only hard requirement is a solid S3-compatible API.

Every command below is copied verbatim from the official source cited beside it. This sandbox has no Docker daemon, so none of the commands were executed here; they are marked accordingly.

Key Stats

Fact	Source
AWS S3 has delivered strong read-after-write consistency for all objects since Dec 1, 2020	AWS What's New (2020-12)
AWS limits each account to 10,000 buckets	AWS S3 docs (BucketRestrictions)
RustFS is Apache 2.0 licensed and S3-compatible	RustFS GitHub README
DuckDB reads Parquet directly from `s3://` through its httpfs extension	DuckDB httpfs S3 API docs
Iceberg, Delta Lake, and Hudi all store Parquet data files plus metadata on object storage	Project documentation

What is a data lake on S3-compatible storage?

A data lake on S3-compatible storage is a bucket that stores raw and transformed data as files, not rows in a database. The files use open formats like Parquet, ORC, or JSON, and they sit under key prefixes that act like folders. An analytics engine reads those files over the S3 API and returns rows, but the data never has to move into a separate warehouse.

The phrase S3-compatible matters because it widens your options. You can start on AWS S3, then run the same code against MinIO, Wasabi, Cloudflare R2, or a self-hosted RustFS cluster without rewriting your pipelines. The object store becomes a neutral substrate: cheap, durable, and reachable from every tool that speaks S3.

In other words, a data lake is less a product and more a layout convention. Get the layout right and any engine can read it. Get it wrong and you rebuild queries every quarter.

Why build your data lake on S3-compatible storage?

You build a data lake on S3-compatible storage because the S3 API is the closest thing the data world has to a universal port. Every query engine, every backup tool, every ETL job already knows how to talk to it. That reach means you are never locked into one vendor's query semantics or one warehouse's pricing page.

The cost angle is real but it is not automatic. Cloud S3 bills per GB-month plus per-request and egress charges, and egress is where lakes get expensive when you query from outside the region. Self-hosting flips that math: you pay for disks and bandwidth you already own, and you can co-locate compute so reads stay on the local network. The trade is operational. Someone runs the cluster, patches it, and owns the failures.

For a small team, starting on a managed S3 endpoint and keeping the code portable is usually the pragmatic first move. You can move the data later without touching the queries.

How do you lay out a data lake in object storage?

The layout is the part people skip and later regret. A data lake in object storage works best with Hive-style partitioning, where each key path encodes a dimension as a directory. A common shape is events/dt=2026-08-14/region=us/part-001.parquet. Engines prune those prefixes, so a query for one day reads a tiny slice instead of the whole bucket.

Keep file sizes sane. Object storage dislikes millions of tiny files: each is a separate GET with its own latency and listing overhead. Aim for files in the tens of megabytes, not kilobytes. If your producer emits small records, batch them before writing.

Avoid deep nesting that no tool can prune, and keep a clear raw zone and a curated zone. Raw holds exactly what landed; curated holds the cleaned tables you actually query. The discipline pays off the first time you need to replay a day from raw.

In other words, treat the key prefix as your primary index, because in object storage it is the only one you get for free.

Which table formats sit on top of S3?

On top of raw files, most teams add a table format so engines agree on schema, partitions, and snapshots. The three open options are Apache Iceberg, Delta Lake, and Apache Hudi. All three store Parquet data files plus JSON or AVRO metadata on object storage, and all three expose a table through the S3 API.

Iceberg is the one I reach for first. Its metadata is designed for cloud object storage: snapshot isolation, hidden partitioning, and schema evolution that does not rewrite files. Delta Lake came out of the Spark world and stays tight with that ecosystem. Hudi targets incremental pipelines and record-level upserts.

You can also skip a table format entirely and use Hive-style partitions with a catalog, which is plenty for append-only logs. The point is that the format is a layer on top of S3, not a replacement for it. Pick the engine-supported option and keep the files on S3 either way.

How do you query a data lake without moving it?

This is the part that surprised me the first time: you can query Parquet straight off S3 with a single-node tool. DuckDB reads it through its httpfs extension. You register credentials with a secret, then point a query at an s3:// path.

-- DuckDB S3 secret (sourced from DuckDB httpfs S3 API docs, NOT EXECUTED IN CI)
CREATE OR REPLACE SECRET lake (
    TYPE s3,
    PROVIDER config,
    KEY_ID 'rustfsadmin',
    SECRET 'rustfsadmin',
    REGION 'us-east-1',
    ENDPOINT 'localhost:9000',
    USE_SSL false
);

SELECT count(*) FROM read_parquet('s3://my-data-lake/raw/dt=2026-08-14/*.parquet');

The ENDPOINT and USE_SSL lines are the S3 secret parameters DuckDB documents for non-AWS hosts; for AWS you omit them. For heavier work, Spark or Trino scan the same prefixes across many nodes, and Iceberg or Delta catalogs give them schema and snapshot info.

The win is that the data stays put. You are not loading it into a warehouse first; you read the lake where it lives. That single property is what keeps a data lake cheap compared to a copy-everywhere architecture.

How do you stand up S3-compatible storage for a lake?

Standing up your own S3-compatible endpoint is a handful of commands if you use RustFS. One Docker container gives you a bucket API that DuckDB, Spark, and the AWS CLI all understand. The official run command is:

# sourced from RustFS GitHub README, NOT EXECUTED IN CI
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default credentials are rustfsadmin / rustfsadmin. Point the MinIO client at it:

# sourced from MinIO mc docs, NOT EXECUTED IN CI
mc alias set rustfs http://localhost:9000 rustfsadmin rustfsadmin

Then create a bucket and load data with the AWS CLI:

# mc mb sourced from MinIO mc docs; aws s3 cp recursive form per AWS CLI docs, NOT EXECUTED IN CI
mc mb rustfs/my-data-lake
aws s3 cp ./events/ s3://my-data-lake/raw/ --recursive --endpoint-url http://localhost:9000

That is a working lake: a bucket, partitioned data, and engines that can read it. Scale and durability are the parts you design next.

How do you keep a data lake safe?

A data lake is only useful if you can recover it. Two S3 features earn their keep here, and both are worth turning on early.

Versioning keeps every write as a new version instead of overwriting it. On AWS S3:

# sourced from AWS CLI docs, NOT EXECUTED IN CI
aws s3api put-bucket-versioning --bucket amzn-s3-demo-bucket --versioning-configuration Status=Enabled

For a self-hosted RustFS lake, versioning is an available feature, and bucket replication can mirror a bucket to a second site for disaster recovery. That replication path is available today.

The catch is that versioning without lifecycle rules fills the bucket forever. On AWS you pair versioning with a lifecycle policy that expires old versions. On RustFS, lifecycle management is still under testing, so plan a manual or external cleanup job until that ships.

In other words, protect the data first, then automate the cleanup. A lake you cannot restore is a liability, and a lake you never clean is a slowly growing bill.

What breaks when you self-host a data lake?

Self-hosting a data lake trades cloud convenience for control, and the rough edges show up in predictable places.

Small files are the first one. Object stores charge per request and list slowly, so a lake built from millions of tiny objects crawls no matter which engine you use. Batch before you write.

Consistency matters for pipelines that list then read. AWS S3 has been strongly consistent since December 2020, but self-hosted S3-compatible stores vary. Test list-after-write in your setup before you trust it for orchestration.

Egress is the quiet tax. If compute runs in a different region or account from the bucket, every scan leaves the network and shows up on the bill. Co-locate compute with storage to avoid it.

Finally, durability is your job now. A single node is a single point of failure. RustFS distributed mode is still under testing, so for production durability today you either replicate to a second node or back the bucket up offsite. Know which one you are doing.

FAQ

Do I need a warehouse like Snowflake to build a data lake?

No. A data lake is just files on object storage plus a query engine. Snowflake, BigQuery, and friends are warehouses that copy data into their own format. You can query Parquet on S3 directly with DuckDB or Trino and skip the warehouse until you actually need one.

Can DuckDB really query S3 directly?

Yes. DuckDB's httpfs extension reads Parquet over the S3 API using a CREATE SECRET for credentials and endpoint. It is single-node, so it suits interactive analysis and moderate scans, not petabyte parallel jobs. For those, use Spark or Trino against the same files.

Is S3-compatible storage strongly consistent for a data lake?

AWS S3 has been strongly consistent for all objects since December 2020, including reads after writes and list operations. Self-hosted S3-compatible stores do not all guarantee this, so verify list-after-write behavior before you build orchestration that depends on it.

Which table format should I pick: Iceberg, Delta, or Hudi?

Start with Apache Iceberg unless your stack is Spark-centric (then Delta) or you need record-level upserts from a streaming source (then Hudi). All three store Parquet plus metadata on S3 and are interoperable at the file level. The format is a layer on top of S3, not a lock-in.

How much does a self-hosted data lake cost versus AWS S3?

AWS S3 Standard bills per GB-month plus per-request and egress charges, with the first 100 GB of internet egress free per month across services. Self-hosting replaces that with your own disk and bandwidth, which is cheaper at scale if compute is co-located. The real cost is operational: someone runs and patches the cluster. Price it as engineering time, not just hardware.

Object Storage vs File Storage: When to Use Which (2026)

Ethan Carter — Sun, 23 Aug 2026 13:45:00 +0000

I still see engineers storing user-uploaded photos in /var/www/uploads/ on an ext4 volume and wondering why their server falls over at 10M files. Meanwhile, the team next door threw the same photos into an S3 bucket and scaled to 100M files without breaking a sweat. The difference is the storage paradigm. Pick the wrong one and you feel it at scale.

I still mix the two up in conversation sometimes, so I keep a short checklist: random writes and file locks mean file storage; HTTP PUTs and billions of objects mean object storage.

Short answer: use file storage when you need POSIX semantics — in-place edits, sub-millisecond random I/O, file locking (databases, OS files, NFS shares). Use object storage when you need scale, an HTTP API and rich metadata (user uploads, data lakes, backups, ML datasets). Most mature stacks run both, side by side.

Key Stats

Fact	Value	Source
Amazon S3 consistency	Strong read-after-write for new objects, overwrites and LIST — since Dec 1, 2020, at no extra cost	AWS What's New
s3fs-fuse random writes	"random writes or appends to files require rewriting the entire object"	s3fs-fuse README → Limitations
goofys write support	"only sequential writes supported"; no symlinks/hardlinks; cannot rename directories with >1000 children	goofys README → Current Status
Mountpoint for Amazon S3	Does "not implement all the features of a POSIX file system" — no directory renaming, no symlinks, no edits to existing files	awslabs/mountpoint-s3
RustFS license	Apache 2.0 (no AGPL restrictions)	rustfs/rustfs README
RustFS FUSE mount	Not offered. The README's Feature & Status table lists no FUSE / POSIX mount driver — use a third-party S3 FUSE client against its S3 endpoint	rustfs/rustfs README

What Is the Fundamental Difference Between Object and File Storage?

Dimension	File Storage	Object Storage
Data unit	File (named byte sequence)	Object (data + metadata + key)
Organization	Hierarchical (directories/subdirectories)	Flat (key namespace; `/` is cosmetic)
Access method	POSIX (open/read/write/seek/close)	HTTP REST API (PUT/GET/DELETE)
Mutability	In-place (change bytes 100-200 without touching 1-99)	Immutable (overwrite = new version/new object)
Metadata	Fixed attributes (name, size, permissions, timestamps)	Rich & extensible (custom key-value tags, content-type, etc.)
Scaling limit	Millions of files (inode exhaustion, metadata perf)	Billions+ of objects (distributed metadata)
Protocol	NFS, SMB, POSIX local (ext4, xfs, zfs)	S3 API (HTTP/HTTPS)
Consistency model	Strong (reads see writes immediately)	Strong read-after-write on AWS S3 since Dec 2020 — covers new objects, overwrites and LIST; S3-compatible systems vary
Typical latency	Sub-millisecond (local) to milliseconds (NFS)	Milliseconds (network round-trip)
Best for	OS-level operations, databases, home dirs	Unstructured data at scale, web/mobile apps, analytics

When Should You Use File Storage?

File storage is the right choice when your application (or OS) needs POSIX semantics:

Use Case 1: Operating System Files

/etc/hosts
/var/log/syslog
/home/user/.bashrc
/tmp/processing_12345.tmp

Your OS expects file storage. It uses open(), read(), write(), seek() — not HTTP PUT/GET. Don't fight this.

Use Case 2: Databases

PostgreSQL, MySQL, MongoDB, SQLite — they all expect block devices or file systems with:

Sub-millisecond random I/O (index lookups, page reads)
In-place mutation (UPDATE SET field = value changes specific bytes)
Strong consistency (ACID transactions depend on ordered fsync)
File locking (.lock files, advisory locks)

Object storage has millisecond-level latency and no in-place mutation. Databases on S3 perform terribly (with niche exceptions like Iceberg/Delta lakehouse patterns).

Use Case 3: Network File Sharing (NFS/SMB)

When multiple users/servers need shared access to the same files with familiar tools:

Design teams sharing Figma/Adobe files via SMB mount
Build servers sharing source code via NFS
Home directories in enterprise environments

These use cases need file-level permissions, directory browsing, and application transparency — all strengths of file storage.

Use Case 4: Small-Scale Applications (< 100K files, < 1TB)

For small datasets, file storage is simpler:

No API to learn
Familiar tools (ls, cp, rsync, grep)
Easy backups (tar, rsync)
Local access = fastest possible

When Should You Use Object Storage?

Object storage is the right choice when you need scale, simplicity of API, and rich metadata:

Use Case 1: User-Generated Content

Photos, videos, documents, uploads — the canonical object storage workload:

# User uploads photo
s3.put_object(
    Bucket="user-photos",
    Key=f"user-{user_id}/photo-{uuid}.jpg",
    Body=image_data,
    ContentType="image/jpeg",
    Metadata={"uploader": str(user_id), "camera": "iphone"}
)

Scale from 1K to 100M objects without changing code. I have watched teams try to stretch a filesystem to that size; inode exhaustion is not a fun afternoon.

Use Case 2: Data Lake / Analytics

Parquet/Avro/CSV files for Spark, Trino, DuckDB:

s3://data-lake/bronze/events/year=2026/month=07/day=24/event-*.parquet
s3://data-lake/gold/daily_active_users.parquet

Flat namespace, massive scale, accessed by query engines that speak S3 natively. This is where object storage dominates in 2026.

Use Case 3: Backup & Archive

Database dumps, VM snapshots, compliance records:

Immutable (versioning = accidental deletion protection)
Tiered (lifecycle policies move old data to cheap storage automatically)
Replicated (cross-region for DR)
Compliant (Object Lock for WORM retention)

File storage can do backups too, but at scale, object storage's tiering and replication features save significant cost and operational effort.

Use Case 4: Static Website / CDN Origin

S3 + CloudFront (or Cloudflare) is the standard pattern for serving static web content:

Objects = web assets (HTML, CSS, JS, images)
Global CDN = fast delivery everywhere
HTTPS + custom domain = zero-infrastructure frontend

Serving a global static site from an NFS mount is not something I'd want to run on-call for.

Use Case 5: Machine Learning & AI

Training data, model checkpoints, inference outputs:

Checkpointing: Write model state as object → resume from any saved point
Dataset versioning: Each dataset version = immutable object (reproducible training)
Feature stores: Parquet objects queried by ML frameworks via S3 API

ML workloads at scale (terabytes of training data) are almost always object-storage-backed in 2026.

Can You Mount S3 as a Filesystem?

What if you want S3's scale but need file-system semantics? I have been asked this in almost every object-storage migration. The honest answer is: you can, but the mount layer will lie to you in small ways. Here is what each project officially documents:

Tool	Language	What it does	Officially documented limits
s3fs-fuse	C++	Mounts an S3 bucket via FUSE on Linux/macOS/FreeBSD; preserves the native object format so `aws s3` still works	"random writes or appends to files require rewriting the entire object"; "no atomic renames of files or directories"; "no hard links"; "no coordination between multiple clients mounting the same bucket"
goofys	Go	A "Filey System" that "strives for performance first and POSIX second"; close-to-open consistency, no on-disk cache	"only sequential writes supported"; "does not support symlink or hardlink"; "cannot `rename` directories with more than 1000 children"; "`fsync` is ignored" — and the last commit was June 2023, so treat it as low-maintenance
Mountpoint for Amazon S3	Rust	AWS's own GA file client, tuned for high read throughput and sequential writes of new objects	AWS states it is "probably not the right fit" for apps that use "directory renaming or symlinks" or "make edits to existing files (don't work on your Git repository or run `vim` in Mountpoint)"; support for non-AWS S3-compatible stores is limited
rclone mount	Go	Mounts any of rclone's 70+ backends, including any S3-compatible endpoint	rclone's own docs warn the file system is not fully POSIX-compliant; behaviour depends on VFS cache mode

Own it first: RustFS does not ship a FUSE driver. Its README Feature & Status table covers S3 core, versioning, bucket replication, event notifications, bitrot protection, Swift/Keystone and Helm charts — no POSIX mount. If you want a mount, point one of the clients above at RustFS's S3 endpoint like you would at any other S3 service. Anyone telling you a "native RustFS mount" exists is reading a spec sheet that doesn't.

Performance reality: the translation layer is POSIX → HTTP, so each metadata operation becomes a network round trip. s3fs-fuse names this explicitly: "metadata operations such as listing directories have poor performance due to network latency." That's fine for bulk work — cp, tar, grep, feeding a training job. It is not fine for databases, build systems or anything doing high-IOPS random writes, because those turn into whole-object rewrites.

Which Should You Pick? A Decision Flowchart

Do you need sub-millisecond random I/O?
├─ YES → File Storage (database, OS files)
│         (or block storage)
│
└─ NO → Do you need POSIX semantics (ls, chmod, flock)?
    ├─ YES → File Storage (NFS/SMB shares, source code)
    │
    └─ NO → Will you exceed 1M files/objects?
       ├─ YES → Object Storage (S3/S3-compatible)
       │         (photos, data lake, backups, ML)
       │
       └─ NO → Either works; pick the simpler tool
                 for your team's skill set

TL;DR

File storage = hierarchical, mutable, POSIX, scales to millions. Use it for OS files, databases, NFS shares, small datasets.
Object storage = flat, immutable, HTTP API, scales to billions. Use it for user content, data lakes, backups/archives, static sites, ML data.
The crossover point is usually scale. Under 100K files/1TB: file storage is simpler. Over that: object storage wins on operational cost.
Databases always want file/block storage (not object). ML/analytics always want object storage (not file).
S3 FUSE clients (s3fs-fuse, goofys, AWS Mountpoint, rclone mount) give you file-system access to S3 data — every one of them documents real POSIX gaps (no atomic renames, sequential-writes-only, no edits to existing files). Fine for bulk I/O, wrong for databases. RustFS itself ships no FUSE driver; use one of these against its S3 endpoint.
You can use both. Most mature infrastructures I have worked with run file storage and object storage side-by-side. The cleanest architecture is usually "file storage for the OS and databases, object storage for everything else" rather than forcing one paradigm to cover both.

Need S3-compatible object storage you can run yourself? RustFS is Apache 2.0 licensed (no AGPL strings) and its README lists S3 core, versioning, bucket replication, event notifications, bitrot protection, multi-tenancy and Helm charts as Available; Lifecycle Management, Distributed Mode and KMS are still marked Under Testing — so plan accordingly. Try it in one command:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[sourced verbatim from the RustFS GitHub README — NOT EXECUTED IN CI]. Console on port 9001, default credentials rustfsadmin / rustfsadmin — change them before you expose anything. Binaries and the rc CLI: rustfs.com/download.

FAQ

Can I replace NFS with S3/object storage?

Sometimes. For plain file sharing I usually skip the FUSE shim and serve objects through a web UI or pre-signed URLs — one less POSIX lie to debug. If you really need a mount, read the limits first: s3fs-fuse has "no atomic renames of files or directories" and "no coordination between multiple clients mounting the same bucket"; goofys supports "only sequential writes"; Mountpoint refuses edits to existing files. Compilers, build systems, anything calling flock() — those stay on real file storage.

Which is faster, object storage or file storage?

It depends where the reader is sitting. A local NVMe filesystem wins for a single machine; object storage wins when you need a CDN in front of it. The question I ask is not "which is faster" but "which is fast enough at this distance". Databases need the local path. A photo served worldwide needs the CDN path. I ignore quoted millisecond figures unless they come with the test setup attached.

Can databases run on object storage?

Traditional OLTP databases — PostgreSQL, MySQL — no, not as their primary data directory. They need in-place mutation, ordered fsync and sub-millisecond random reads, none of which object storage provides. What does work, and works extremely well, is the lakehouse pattern: query engines such as DuckDB (via httpfs), Trino, Spark and ClickHouse (S3 table engine) read Parquet/ORC directly out of S3, and table formats like Apache Iceberg and Delta Lake add ACID semantics on top of immutable objects. Object storage is also the universal backup target for databases. So the accurate statement is: analytics on object storage, yes; transactional storage engine on object storage, no.

How do I migrate from file storage to object storage?

Gradually, and by workload rather than by directory. A path that works: (1) point all new workloads at S3 from day one; (2) move user-generated content first — uploads are the natural fit; (3) move analytics data next, as Parquet in a bucket queried by Spark/Trino/DuckDB; (4) leave the legacy file server on NFS/SMB and mirror it to object storage for DR and archive; (5) never move OS files. rclone sync handles filesystem-to-S3 copies against any S3-compatible endpoint, and aws s3 sync works for AWS. Budget for a coexistence period — both paradigms running side by side is the normal end state, not a failure.

Does RustFS provide a POSIX or FUSE mount?

No. As of the check date on this article, the RustFS GitHub README's Feature & Status table lists S3 Core Features, Upload/Download, Versioning, Logging, Event Notifications, K8s Helm Charts, Keystone Auth, Swift API, Bitrot Protection, Single Node Mode, Bucket Replication and Multi-Tenancy as Available, with Lifecycle Management, Distributed Mode and RustFS KMS marked Under Testing. There is no FUSE driver, no rustfs mount command and no POSIX mount feature anywhere in the README or on docs.rustfs.com. If you need a mount, run s3fs-fuse, goofys or rclone mount against the RustFS S3 endpoint on port 9000 — exactly as you would against any other S3-compatible service.

Sources

All claims above were checked against primary sources on 2026-08-07:

Amazon S3 strong read-after-write consistency (new objects, overwrites, LIST; Dec 1, 2020) — https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-s3-now-delivers-strong-read-after-write-consistency-automatically-for-all-applications/
s3fs-fuse "Limitations" section (random writes rewrite the whole object, no atomic renames, no hard links, no multi-client coordination) — https://github.com/s3fs-fuse/s3fs-fuse
goofys "Current Status" non-POSIX behaviours (sequential writes only, no symlink/hardlink, 1000-child rename cap, fsync ignored); last commit June 2023 — https://github.com/kahing/goofys
Mountpoint for Amazon S3 POSIX caveats (no directory renaming, no symlinks, no edits to existing files) — https://github.com/awslabs/mountpoint-s3
rclone mount documentation — https://rclone.org/commands/rclone_mount/
RustFS license, Feature & Status table, quickstart command and default credentials — https://github.com/rustfs/rustfs
RustFS installation documentation (no FUSE/POSIX mount path listed) — https://docs.rustfs.com/

Using S3 as Your Disaster Recovery Target: A Practical Guide

Ethan Carter — Fri, 21 Aug 2026 13:40:00 +0000

Using S3 as Your Disaster Recovery Target: A Practical Guide

S3 works as a disaster-recovery target because your data is already there, it is redundant by design, and cross-region replication is a configuration change rather than a new pipeline. The case for bothering is in the outage economics: Uptime Institute's 2022 Outage Analysis found that over 60% of failures now cost at least $100,000 — up from 39% in 2019 — and the share costing more than $1 million climbed from 11% to 15%. Yet when I audit DR plans, S3 is consistently the most underused asset in the room.

Key Stats

Metric	Figure	Source
Failures costing ≥ $100,000	60%+ (was 39% in 2019)	Uptime Institute, 2022 Outage Analysis
Failures costing ≥ $1M	15% (was 11%)	Uptime Institute, 2022 Outage Analysis
Orgs hit by a serious/severe outage in 3 years	1 in 5	Uptime Institute, 2022 Outage Analysis
Public outages lasting > 24 hours (2021)	~30% (was 8% in 2017)	Uptime Institute, 2022 Outage Analysis
Major outages caused by human error	~40% of orgs; 85% trace to procedure failures	Uptime Institute, 2022 Outage Analysis
AWS S3 designed durability	99.999999999% (11 nines)	AWS S3 official documentation

That last row on human error is the one worth sitting with: the failure mode is rarely "the storage broke." It is that nobody had run the restore.

I have audited enough DR plans to notice a pattern: the teams that survive outages are not the ones with the most elegant architecture; they are the ones that actually tested a restore. S3-compatible storage is just the cheapest place to start that habit.

Why does S3 work well as a DR target?

It's Already There

Most applications I look at already write something to S3, even if the team does not think of it as primary storage:

User-uploaded files (photos, documents, media)
Log files and audit trails
Database backups (mysqldump, pg_dump, volume snapshots)
ML model artifacts and datasets
Configuration backups

If this data is already in S3, making it DR-capable is mostly a configuration change, not a new pipeline.

It's Durable by Design

S3 (and serious S3-compatible implementations) stores data redundantly:

Implementation	Stated Durability	Mechanism
AWS S3	99.999999999% (11 nines), designed for	Redundant storage across multiple devices in ≥3 Availability Zones
Backblaze B2	99.999999999% (11 nines), annual	Reed-Solomon erasure coding
RustFS	No published nines figure	README lists Bitrot Protection and Bucket Replication as ✅ Available; Distributed Mode is 🚧 Under Testing
MinIO	No published nines figure	Erasure coding or replication, configurable

I am deliberately not inventing a durability number for RustFS or MinIO. Neither project publishes an audited nines figure, and a self-hosted cluster's real durability depends on your disk count, erasure set width and failure domains — not on the vendor's marketing page.

Single-disk failure = zero data loss. Single-node failure (in clustered setups) = zero data loss. This is better than most on-premises databases achieve out of the box.

It's Cheap (Relative to Alternatives)

DR is insurance — nobody wants to overpay for it. Concretely: AWS S3 Standard list price is $0.023/GB-month, so parking 10 TB of DR backups costs about $236/month. A warm-standby database instance sized for the same workload, plus its attached block storage, generally lands in four figures a month. I am not going to quote you a tidy multiple, because the ratio swings hard with instance class, retention and egress — run the numbers for your own shape.

Which DR architecture pattern should you use?

Pattern 1: Backup-to-S3 (What I recommend first)

Production DB ──[daily dump]──▶ S3 Bucket (primary region)
                                    │
                              Cross-region replicate
                                    │
                                    ▶
                              S3 Bucket (DR region)

How it works:

Nightly database dump → compressed file → PUT to S3
S3 Cross-Region Replication (CRR) copies to DR region
If primary region fails: spin up DB in DR region → restore from S3

RTO (Recovery Time Objective): 1-4 hours (depends on DB size + restore speed)
RPO (Recovery Point Objective): Up to 24 hours (backup frequency)

This is not exciting architecture, but it is the pattern that saves most teams. Get backups into a second region before you worry about streaming replication.

Tools: aws s3 sync, rclone, database-native S3 backup tools (pg_backrest, mysqldump + pipe)

Pattern 2: Continuous Log Shipping (Better RPO)

Production DB WAL/Binlog ──[stream]──▶ S3 (primary)
                                         │
                                   CRR / Custom forwarder
                                         │
                                         ▶
                                   S3 (DR region)
                                         │
                                   [Continuous restore]
                                         ▶
                                   Standby DB (DR region)

How it works:

Write-Ahead Logs (PostgreSQL) or Binlogs (MySQL) stream to S3 continuously
Standby DB in DR region applies logs in near-real-time
If primary fails: promote standby (seconds to minutes of RPO)

RTO: Minutes (standby is already running, just needs promotion)
RPO: Seconds to minutes (depends on log shipping lag)

Tools: WAL-G (PostgreSQL), pgBackRest, MySQL binlog-to-s3 tools, Debezium (CDC)

Pattern 3: Active-Active with S3 as Source of Truth (Avoid unless you need it)

Region A App ◄──── S3 (shared, replicated) ───► Region B App
                    │
              [Both regions read/write to same S3]
              [App-level conflict resolution required]

How it works:

Both regions' applications read/write to the same S3 bucket(s)
Conflict resolution is your application's job — object storage does not merge concurrent writes for you
If Region A fails: Region B continues serving with zero RPO

RTO: Zero (automatic)
RPO: Zero (both regions always current)

Complexity: High. I have seen this pattern look simple on a whiteboard and turn into weeks of conflict-resolution bugs in production. Use it only when zero RPO is a hard business requirement, not because it sounds modern.

Implementing Pattern 1: The Minimal Viable DR Plan

Step 1: Identify Critical Data

Not all data needs DR protection. Classify yours:

Tier	Data Type	Example	DR Requirement
T0 (Critical)	User data, financial transactions	User uploads, payment records	RPO < 1hr, RTO < 1hr
T1 (Important)	Business operational data	Logs, configs, ML models	RPO < 24hr, RTO < 4hr
T2 (Useful)	Analytics, historical	Aggregated data, old backups	RPO < 7 days, RTO < 24hr
T3 (Disposable)	Cache, temp files	Session stores, build artifacts	Best effort / none

Focus DR effort on T0 and T1. T2 and T3 are nice-to-have.

Step 2: Set Up Automated Backups to S3

#!/bin/bash
# dr-backup.sh — Daily backup to S3
DATE=$(date +%Y-%m-%d)
BUCKET="dr-backups-primary"

# PostgreSQL backup
pg_dump -Fc production_db | gzip | aws s3 cp - "s3://$BUCKET/postgres/$DATE/db.dump.gz"

# Application data sync
aws s3 sync /data/app-uploads "s3://$BUCKET/uploads/$DATE/" --delete

# Config backup
tar czf - /etc/myapp/config | aws s3 cp - "s3://$BUCKET/config/$DATE/config.tar.gz"

echo "[$(date)] DR backup complete" >> /var/log/dr-backup.log

Schedule via cron (every 6 hours for T0, daily for T1):

0 */6 * * * /opt/scripts/dr-backup.sh    # T0: Every 6 hours
0 2 * * * /opt/scripts/dr-backup-full.sh # T1: Daily at 2 AM

Step 3: Configure Cross-Region Replication

On AWS. Replication requires versioning on both buckets, and Role is a required top-level field in the replication configuration — a lot of copy-pasted snippets omit it and fail with InvalidRequest.

# [sourced from https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-replication.html, NOT EXECUTED IN CI]

# Prerequisite: versioning on BOTH source and destination
aws s3api put-bucket-versioning \
  --bucket dr-backups-primary \
  --versioning-configuration Status=Enabled

aws s3api put-bucket-versioning \
  --bucket dr-backups-dr \
  --versioning-configuration Status=Enabled

# CRR rule ("Role" is REQUIRED — the IAM role S3 assumes to replicate)
aws s3api put-bucket-replication \
  --bucket dr-backups-primary \
  --replication-configuration '{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [{
      "Status": "Enabled",
      "Priority": 1,
      "DeleteMarkerReplication": { "Status": "Enabled" },
      "Filter": { "Prefix": "" },
      "Destination": {
        "Bucket": "arn:aws:s3:::dr-backups-dr",
        "StorageClass": "STANDARD"
      }
    }]
  }'

On self-hosted S3. RustFS lists Bucket Replication as ✅ Available in its README Feature & Status table. If you want a scheduled, engine-agnostic copy between two independent clusters — which is what most self-hosted DR setups actually run — rclone is the pragmatic tool. Define two named S3 remotes, then sync:

# ~/.config/rclone/rclone.conf
# [sourced from https://rclone.org/s3/, NOT EXECUTED IN CI]
[primary]
type = s3
provider = Other
access_key_id = rustfsadmin
secret_access_key = rustfsadmin
endpoint = http://primary-site:9000

[dr]
type = s3
provider = Other
access_key_id = rustfsadmin
secret_access_key = rustfsadmin
endpoint = http://dr-site:9000

# [sourced from https://rclone.org/commands/rclone_sync/, NOT EXECUTED IN CI]
rclone sync primary:dr-backups-primary dr:dr-backups-dr --progress

Then put that behind cron every 30 minutes. Note rclone sync makes the destination match the source — it deletes objects at the destination that no longer exist at the source. If you want DR to survive an accidental mass-delete on the primary, use rclone copy instead, or enable versioning and Object Lock on the DR bucket.

Step 4: Document & Test the Restore Procedure

Your DR plan is only useful if the person on call can execute it without calling you. I have been that 3 a.m. call; write the runbook for someone who has not seen it before:

DR Runbook: Primary Region Failure

Trigger conditions:

Primary region unreachable for > 15 min
Major data corruption detected
Executive decision to failover

The whole procedure runs about 45–90 minutes. Each step is a separate command block you can paste as-is.

1.Verify DR bucket integrity (5 min)

   aws s3 ls s3://dr-backups-dr/ --recursive | wc -l
   # Compare count to expected object count

2.Promote DR database (15-45 min)

   # Start the instance in the DR region, then stream the dump straight out of S3.
   # NOTE: gunzip cannot read an s3:// URL — you must pipe through `aws s3 cp ... -`.
   LATEST=$(aws s3 ls s3://dr-backups-dr/postgres/ | sort | tail -n 1 | awk '{print $2}')
   aws s3 cp "s3://dr-backups-dr/postgres/${LATEST}db.dump.gz" - \
     | gunzip -c \
     | pg_restore -d production_db

3.Update DNS (2-30 min, depends on TTL)

   # Route53: Change A record to DR instance IP
   # Wait for DNS propagation (monitor with dig + health checks)

4.Verify application health (10 min)

   curl https://myapp.com/healthcheck
   # Expect: {"status":"ok","region":"dr"}

5.Communicate (ongoing)

Status page update
Internal Slack alert
Customer notification (if SLA impacted)

Test this runbook quarterly. An untested runbook is just a theory, and outages are bad at following theories.

What goes wrong with S3 DR?

Replication lag will surprise you

CRR is asynchronous. An object written at T0 might not appear in the DR bucket until T0 + 15 minutes (or longer under load). Your latest backup might not yet be in DR when you need it. This is the failure mode I check first in any DR drill.

What helps: check replication lag metrics before declaring disaster, and keep a force-sync procedure for critical objects.

Encryption keys live in one region

If your primary region uses AWS KMS-managed keys and that region is down... you can't decrypt backups in the DR region unless you've planned for cross-region KMS access. I have seen a perfectly good DR bucket become useless because the key was still in the failed region.

What helps: use client-side encryption (you hold the key) or ensure the KMS key is accessible from the DR region (multi-Region KMS key).

S3 is not a database DR mechanism

S3 is great for DR of things stored in S3. It is not a replacement for database replication. Your PostgreSQL primary still needs streaming replication, logical replication, or Patroni — S3 is the safety net, not the primary mechanism.

TL;DR

S3 is probably already your biggest DR asset — most critical data lands there eventually.
Three patterns: Backup-to-S3 (simplest, RPO=hours), Log shipping (better, RPO=minutes), Active-Active (best, complex).
Start with Pattern 1 (automated backups + CRR) — it's a 1-day setup that covers 80% of DR scenarios.
Test restores quarterly. An untested DR plan is a false sense of security.
S3 complements — doesn't replace — database replication. Use both.

Building a DR target on self-hosted S3? RustFS is Apache 2.0 licensed and its README Feature & Status table lists **Bucket Replication, **Versioning, **Bitrot Protection* and Event Notifications as ✅ Available — the four primitives a DR pipeline actually leans on. Being straight with you: Distributed Mode is still 🚧 Under Testing, so validate your multi-node topology yourself before betting a production DR plan on it. Download here.*

FAQ

Is S3 enough for disaster recovery, or do I need a database replica?

You need both. S3 handles the non-transactional layer — files, logs, backups, artifacts. For PostgreSQL or MySQL you still need streaming or logical replication if you want RPO measured in seconds. I treat S3 DR as the safety net and database replication as the thing that actually keeps me asleep at night.

How long does it take to restore from S3 after a disaster?

You will not know until you time it. Bandwidth to the S3 endpoint, I/O on the restore target, and whether you are doing a full restore or point-in-time recovery all dominate. Run a quarterly drill, record the real number in the runbook, and use that as your RTO. The planning-doc number is fiction.

Should I use AWS S3 or self-hosted S3 for DR?

Both work. AWS S3 with Cross-Region Replication is simpler to configure but costs more (storage plus request and egress fees) and keeps your DR data inside the same vendor blast radius as production. Self-hosted S3 (RustFS, MinIO, Ceph RGW) gives lower ongoing cost, no inter-site egress billing, and vendor independence, at the price of operating the DR storage yourself. Many teams run a hybrid: self-hosted for the primary site, a cloud bucket as the off-site archive tier.

Does S3 Cross-Region Replication protect me from accidental deletion?

Not by itself. CRR faithfully replicates deletions when DeleteMarkerReplication is enabled, so a mass-delete on the primary propagates to DR. Real protection comes from versioning plus Object Lock (WORM retention) on the destination bucket, and from keeping at least one backup copy outside the replication path. Replication is a redundancy mechanism, not a backup.

What is the required IAM setup for `put-bucket-replication`?

The Role field is a required top-level element of the replication configuration — it is the IAM role S3 assumes on your behalf. That role needs read permissions plus s3:GetReplicationConfiguration on the source bucket, and s3:ReplicateObject, s3:ReplicateDelete and s3:ReplicateTags on the destination. Versioning must be enabled on both buckets before the rule will apply.

Sources

All figures and commands in this article were checked against primary sources on 2026-08-06:

Uptime Institute, 2022 Outage Analysis — outage cost and human-error figures: https://www.businesswire.com/news/home/20220608005265/en/
AWS CLI reference, put-bucket-replication (required Role field, versioning prerequisite): https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-replication.html
rclone S3 backend configuration: https://rclone.org/s3/
rclone sync command semantics: https://rclone.org/commands/rclone_sync/
RustFS README, Feature & Status table + Apache 2.0 license: https://github.com/rustfs/rustfs

Commands marked NOT EXECUTED IN CI are reproduced verbatim from the linked official documentation but were not run in the environment used to write this article.

S3 Lifecycle Policies: How to Cut Your Storage Bill 40% (With the Actual Arithmetic)

Ethan Carter — Wed, 19 Aug 2026 13:14:00 +0000

An S3 lifecycle policy moves objects between storage classes on a schedule, or deletes them. The '40%' people quote is real, but only for one specific bucket shape, and only on the storage line. On AWS us-east-1 list prices, pushing 30% of a 100 TB bucket to Standard-IA and 40% to Glacier Instant Retrieval drops the storage line from $2,304.00 to $1,254.40 — a 45.6% cut. The arithmetic is the boring part. The saving leaks back out in the quarter after you ship the rule, in the line items nobody re-opens.

Every number here is pulled from the AWS Price List Bulk API (us-east-1 offer file, publicationDate 2026-08-07) or worked out from those rates. If a rate isn't in the offer file, I say so. I'm not inventing a number to fill the gap.

Key Stats

Metric	Figure	Source
S3 Standard, first 50 TB/mo	$0.023 / GB-mo	AWS Price List API, us-east-1
S3 Standard-IA	$0.0125 / GB-mo + $0.01/GB retrieval	AWS Price List API
S3 Glacier Instant Retrieval	$0.004 / GB-mo + $0.03/GB retrieval	AWS Price List API
S3 Glacier Flexible Retrieval	$0.0036 / GB-mo	AWS Price List API
Minimum object size that transitions by default	128 KB (since Sept 2024)	AWS S3 User Guide
Minimum storage duration	30 d (IA) / 90 d (Glacier IR, Flexible) / 180 d (Deep Archive)	AWS S3 User Guide
Lifecycle transition request, into Glacier Flexible	$0.03 / 1,000 requests	AWS Price List API

What is an S3 lifecycle policy?

An S3 lifecycle policy is a bucket-level ruleset: it transitions objects to a cheaper class, or expires them, once they hit an age you set. You attach JSON or XML; S3 runs it asynchronously, so there's no cron job to babysit.

The User Guide lists five actions: Transition, Expiration, NoncurrentVersionTransition, NoncurrentVersionExpiration, AbortIncompleteMultipartUpload, and a Filter that takes Prefix, Tag, ObjectSizeGreaterThan, ObjectSizeLessThan, or And to combine them.

Two things actually cost you money. One: transitions only go down the ladder. The User Guide is blunt that the Deep Archive transition 'can go only one way.' Two: billing starts the instant the rule is satisfied, not when the data finishes moving. You're paying the destination rate from day one, even if AWS is still copying in the background.

Where does the 40% actually come from?

It comes down to the price gap between classes, times how much of your data is cold. That's the whole formula. For a 100 TB bucket in us-east-1 (AWS bills in 1 TB = 1,024 GB, so that's 102,400 GB), baseline, everything in S3 Standard:

 51,200 GB × $0.023  = $1,177.60   (first 50 TB band)
 51,200 GB × $0.022  = $1,126.40   (next 450 TB band)
 ------------------------------------------------
 TOTAL                 $2,304.00 / month

Split that 100 TB three ways: hot in Standard, warm in Standard-IA, cold in Glacier IR:

Scenario	Hot / Warm / Cold	Monthly storage	Saving
Conservative	50% / 30% / 20%	$1,643.52	−28.7%
Typical	30% / 30% / 40%	$1,254.40	−45.6%
Aggressive	20% / 20% / 60%	$972.80	−57.8%

That 45.6% assumes roughly two-thirds of your bytes are over 30 days old, which is the usual shape for logs, backups, and ML snapshots. But it's the storage line only. Requests and retrievals bill on top, and they're exactly where the saving disappears.

Which storage class should each tier of data land in?

Pick the class by how often you actually read the data, then check the minimum duration. The two numbers that drive cost are per-GB storage and per-GB retrieval. Here's the full ladder from the same offer file:

Storage class	$/GB-mo	Retrieval	Min duration	Min billable size	Availability SLA
S3 Standard	$0.023	—	none	none	99.99%
S3 Standard-IA	$0.0125	$0.01/GB	30 days	128 KB	99.9%
S3 One Zone-IA	$0.010	$0.01/GB	30 days	128 KB	99.5%
S3 Glacier Instant Retrieval	$0.004	$0.03/GB	90 days	128 KB	99.9%
S3 Glacier Flexible Retrieval	$0.0036	$0.01/GB standard, $0.00/GB bulk	90 days	n/a*	99.99% after restore
S3 Glacier Deep Archive	see note	$0.02/GB standard, $0.0025/GB bulk	180 days	n/a*	99.99% after restore

* Glacier Flexible and Deep Archive add 40 KB of overhead per object, of which 8 KB is billed at S3 Standard rates and 32 KB at the destination Glacier rate.

Note on Deep Archive: the us-east-1 S3 offer file exposes Deep Archive's transition, retrieval, and checksum SKUs but no per-GB timed-storage SKU. I won't quote a storage rate I can't pull from a first-party file, so get it from the pricing console before you budget on it. The closest verifiable neighbour in the same file is Intelligent-Tiering's Deep Archive Access tier at $0.00099/GB-mo.

One Zone-IA is 20% cheaper and lives in a single AZ at 99.5% availability. Fine for regenerable stuff: thumbnails, transcodes, a second backup copy. Never the only copy you've got.

How do you write and apply the rule?

With aws s3api put-bucket-lifecycle-configuration. This is the official AWS CLI example, copied verbatim from the command reference:

aws s3api put-bucket-lifecycle-configuration --bucket amzn-s3-demo-bucket --lifecycle-configuration  file://lifecycle.json

{
    "Rules": [
        {
            "ID": "Move rotated logs to Glacier",
            "Prefix": "rotated/",
            "Status": "Enabled",
            "Transitions": [
                {
                    "Date": "2015-11-10T00:00:00.000Z",
                    "StorageClass": "GLACIER"
                }
            ]
        },
        {
            "Status": "Enabled",
            "Prefix": "",
            "NoncurrentVersionTransitions": [
                {
                    "NoncurrentDays": 2,
                    "StorageClass": "GLACIER"
                }
            ],
            "ID": "Move old versions to Glacier"
        }
    ]
}

[Both blocks sourced verbatim from docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-lifecycle-configuration.html — NOT EXECUTED IN CI.]

A workable rule, using only documented LifecycleRule fields: it tiers logs down, skips objects too small to move, expires old versions, and reclaims abandoned multipart uploads:

{
  "Rules": [
    {
      "ID": "logs-tier-down",
      "Status": "Enabled",
      "Filter": {
        "And": {
          "Prefix": "logs/",
          "ObjectSizeGreaterThan": 131072
        }
      },
      "Transitions": [
        { "Days": 30,  "StorageClass": "STANDARD_IA" },
        { "Days": 120, "StorageClass": "GLACIER_IR" }
      ],
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30 },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

ObjectSizeGreaterThan: 131072 is just 128 KB written out, and the next section explains why that number matters. The Glacier IR hop sits at day 120, not 90, because Standard-IA has a 30-day minimum and the User Guide says the second transition 'must occur after at least' the first one's minimum has passed. Tighten that gap and AWS rejects the rule outright.

Why do small objects break lifecycle economics?

You pay per transition request, not per byte. The first lifecycle bill that ever surprised me was a 200 KB-heavy bucket where transition cost ate the whole storage saving, which is the trap most people hit before they check object size. A 200 KB object costs the same request as a 200 MB one. AWS made that the default in September 2024: 'the default behavior prevents objects smaller than 128 KB from being transitioned to any storage class.'

Payback (transition fee divided by the monthly saving) at list rates:

Object size	→ Standard-IA	→ Glacier IR	→ Glacier Flexible
128 KB	7.8 months	8.6 months	12.7 months
256 KB	3.9 months	4.3 months	6.3 months
1 MB	1.0 month	1.1 months	1.6 months
10 MB	~0.1 month	~0.1 month	~0.2 months

Scale that to a whole bucket. Same 100 TB, one transition each, at $0.01/1,000 into Standard-IA and $0.03/1,000 into Glacier Flexible:

Average object size	Object count	One-time bill → Standard-IA	→ Glacier Flexible
64 MB	1,638,400	$16.38	$49.15
2 MB	52,428,800	$524.29	$1,572.86
200 KB	536,870,912	$5,368.71	$16,106.13

Same 100 TB, but a 328x swing in transition cost driven purely by object size. AWS says it straight: 'for smaller objects, the transition costs can outweigh the storage savings.' Check your average object size in S3 Storage Lens first, and put ObjectSizeGreaterThan on every transition.

What does the policy cost when the data comes back?

Retrieval fees are the second place the win leaks out. Standard-IA saves $0.0105/GB-mo versus Standard but charges $0.01/GB to read. Those two numbers give you a hard break-even:

Class	Monthly saving vs Standard	Retrieval	Break-even
Standard-IA	$0.0105/GB	$0.01/GB	1.05 full reads/month
One Zone-IA	$0.0130/GB	$0.01/GB	1.30 full reads/month
Glacier Instant Retrieval	$0.0190/GB	$0.03/GB	0.63 full reads/month

Read a Glacier IR object more than about twice a quarter and you'd have been cheaper in Standard. The '$0.004/GB' headline hides how tight that budget really is.

Early deletion is the third trap. Delete or overwrite before the minimum and AWS bills the rest pro-rated: $0.0125/GB-mo for Standard-IA, $0.004 for Glacier IR, $0.0036 for Glacier. Point a 30-day retention rule at a 90-day-minimum class and you pay for 90 days of Glacier every month, indefinitely.

Intelligent-Tiering or hand-written rules?

Intelligent-Tiering adds a monitoring fee of $0.0025 per 1,000 objects per month and moves data between tiers for you, with no retrieval fee on the Frequent, Infrequent, or Archive Instant tiers. Hand-written rules are free to run but assume you already know your access pattern.

What decides it is object count, not total data:

Object count	Intelligent-Tiering monitoring
1,000,000	$2.50/month
50,000,000	$125.00/month
536,870,912	$1,342.18/month

I use Intelligent-Tiering for buckets with unpredictable reads and big objects, and explicit rules where the access pattern is already obvious. Logs and backups are the obvious case: nobody reads last quarter's access logs spontaneously, so a monitoring fee to find that out is pure waste. The other direction: half a billion thumbnails is $1,342/month in monitoring, and most of those objects sit under the 128 KB threshold anyway.

Does lifecycle work the same on self-hosted S3?

Not quite. On AWS a transition shuffles bytes between AWS-run classes. On self-hosted S3 there's no Glacier, so a 'lifecycle rule' means expire, clean up old versions, or move to a remote tier you set up yourself.

MinIO implements this through mc ilm. From the official command reference:

mc ilm rule add --expire-days 90 --noncurrent-expire-days 30  myaistor/mydata

mc ilm rule add --transition-days 30 --transition-tier "COLDTIER" myaistor/mydata

[Verbatim from min.io/docs — the myaistor alias is MinIO's own example alias. NOT EXECUTED IN CI.]

RustFS is straight about this one. Its README Feature & Status table shows Lifecycle Management as 🚧 Under Testing, next to Distributed Mode and RustFS KMS. Versioning, Bucket Replication, Event Notifications, Bitrot Protection, and Single Node Mode are ✅ available; there's no Object Lock row and no tiering row. If tiered lifecycle is a hard requirement today, RustFS isn't the answer yet.

What RustFS does give you is the structural argument: your own disks mean no per-GB retrieval fee and no per-request transition fee, so the break-even math above stops applying. The catch is you buy and run the hardware. If you're already running RustFS for other reasons, expire-and-cleanup lifecycle is free and worth turning on. If you'd stand it up purely to dodge egress, the hardware usually costs more than the bandwidth you save, so run the numbers first.

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[Verbatim from github.com/rustfs/rustfs README — NOT EXECUTED IN CI.]

S3 API on :9000, Console on :9001. The README lists default credentials rustfsadmin / rustfsadmin, and docs.rustfs.com explicitly tells you not to keep them: "Do not use the well-known rustfsadmin value for either credential."

The five mistakes that eat the savings

Transitioning objects under 128 KB. Payback is 8–13 months. Filter them out with ObjectSizeGreaterThan.
Chaining transitions inside a minimum duration. Standard-IA → Glacier IR at day 60 is invalid; the IA minimum is 30 days and Glacier IR's own 90-day clock only starts on arrival.
Forgetting NoncurrentVersionExpiration. On a versioned bucket, every overwrite leaves a billable version. Storage grows forever while your object count looks flat.
Skipping AbortIncompleteMultipartUpload. Failed uploads leave parts that are invisible in ListObjects but fully billable.
Enabling Intelligent-Tiering on hundreds of millions of small objects. Objects under 128 KB are never monitored for tiering, but you can still be paying monitoring fees at scale for the rest.

FAQ

How much can S3 lifecycle policies actually save?

On AWS us-east-1 list prices, a 100 TB bucket costs $2,304.00/month entirely in S3 Standard. Moving 30% to Standard-IA and 40% to Glacier Instant Retrieval brings it to $1,254.40/month — a 45.6% cut. A conservative 50/30/20 split still saves 28.7%. Savings scale linearly with the fraction of cold data, and apply to the storage line only.

What is the minimum object size for an S3 lifecycle transition?

128 KB. Since September 2024, AWS's default behavior prevents objects smaller than 128 KB from transitioning to any storage class. Configurations created before September 2024 keep the old behavior until you edit them. You can override with the x-amz-transition-default-minimum-object-size header on PutBucketLifecycleConfiguration, but you usually should not: a 128 KB object takes 7.8 months to pay back a single Standard-IA transition.

Does S3 charge for lifecycle transitions?

Yes, per request. In us-east-1: $0.01 per 1,000 transitions into Standard-IA, One Zone-IA, or Intelligent-Tiering; $0.02 per 1,000 into Glacier Instant Retrieval; $0.03 per 1,000 into Glacier Flexible Retrieval; $0.05 per 1,000 into Glacier Deep Archive. For 536 million small objects, that single move to Glacier Flexible is $16,106.

When is Standard-IA more expensive than S3 Standard?

When you read the object more than 1.05 times per month. Standard-IA saves $0.0105/GB-month on storage but charges $0.01/GB on retrieval, so a little over one full read per month erases the discount. Glacier Instant Retrieval breaks even at 0.63 reads/month because its $0.03/GB retrieval fee is three times higher.

Do self-hosted S3 servers support lifecycle policies?

Partially, and support varies. MinIO implements lifecycle through mc ilm rule add with expiration and remote-tier transition. RustFS lists Lifecycle Management as 🚧 Under Testing in its README Feature & Status table, so it is not production-ready today. The upside of self-hosting is that per-GB retrieval fees and per-request transition fees do not exist, which removes the break-even math entirely.

Sources

All figures checked 2026-08-10.

Source	Used for
AWS Price List Bulk API — `pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonS3/current/region_index.json`, us-east-1 offer file, `publicationDate 2026-08-07`	All per-GB storage, retrieval, request, and transition rates
Transitioning objects — general considerations	128 KB default, minimum durations, 40 KB Glacier overhead, one-way Deep Archive, early-billing rule
Understanding and managing Amazon S3 storage classes	Minimum billable size, availability SLAs, AZ counts
S3 Lifecycle configuration elements	Action and Filter element names
aws s3api put-bucket-lifecycle-configuration	Verbatim CLI command and JSON example
mc ilm rule add	Verbatim MinIO ILM commands
rustfs/rustfs README	Verbatim docker run, default credentials, Feature & Status table
docs.rustfs.com — Docker	API port 9000, Console port 9001, credential warning

Before you trust any of this on your own bucket, pull object count and average size from S3 Storage Lens and drop them into the payback table to see whether a rule actually pays off. In my experience most don't, or only barely.

If retrieval and transition fees, not the storage rate, are what's blowing up your bill, self-hosted S3-compatible storage deletes both line items. RustFS is Apache 2.0, S3-compatible, and runs single-node today: github.com/rustfs/rustfs.

Cross-Region S3 Replication Without the Gotchas (2026)

Ethan Carter — Mon, 17 Aug 2026 13:07:00 +0000

Cross-Region S3 Replication Without the Gotchas (2026)

Cross-region S3 replication copies objects from a source bucket to a destination in another region, continuously. AWS S3, MinIO, and RustFS all do it, but the failures almost never come from the copy itself. They come from versioning, IAM, and delete-marker defaults you forgot to set.

Key facts before you start

Check	What the official docs say
Versioning	AWS requires versioning enabled on both the source and destination buckets before replication works.
Pre-existing objects	AWS does not replicate objects written before the replication configuration; use S3 Batch Replication to backfill.
IAM role	AWS replication needs a role with 8 S3 actions (GetReplicationConfiguration, ListBucket, GetObjectVersionForReplication/Acl/Tagging, ReplicateObject/Delete/Tags).
MinIO	`mc replicate add` requires versioning on the source bucket; objects without a version ID are excluded.
RustFS	Site replication (`rc admin replicate add`) links whole deployments and needs versioning + TLS + root creds at every site.

What is cross-region S3 replication, in plain terms?

CRR is a server-side rule that copies every new object from a source bucket to a destination in another region, asynchronously. 'Asynchronously' matters here: the source write returns success before the copy lands, so the two buckets are eventually consistent, not locked in step. Anything that speaks the PUT Bucket Replication API does the same: AWS, MinIO, RustFS. The point is disaster recovery and reads closer to the user. But 'just copy my bucket to another region' hides a dozen small defaults. Versioning state, IAM trust, delete-marker handling, KMS keys: each one fails silently instead of throwing an error you'd notice.

Versioning is the prerequisite nobody can skip

This is the one I see missed most in postmortems. Versioning has to be on, at both ends, or replication doesn't work. I've spent multiple sessions debugging broken replication from skipping the 15-minute propagation wait after enabling versioning, so now I turn it on and walk away before writing anything. AWS tracks objects by version ID, so the destination needs it too. Enabling it is one CLI call (verbatim below). AWS flags a gotcha here that's worth taking seriously:

# Enable versioning on BOTH the source and destination bucket
aws s3api put-bucket-versioning \
    --bucket amzn-s3-demo-bucket \
    --versioning-configuration Status=Enabled

AWS note: "When you enable versioning on a bucket for the first time, it might take a short amount of time for the change to be fully propagated... We recommend that you wait for 15 minutes after enabling versioning before issuing write operations."

If you attach the rule to a bucket without versioning, AWS rejects the config outright. If only the destination is missing it, the source accepts writes but every replica write fails silently, and nothing shows up remotely. That's the failure I get pinged about more than any other: replication looks healthy, the destination is just empty.

How do you set up the AWS IAM role without a silent AccessDenied?

S3 assumes an IAM role to do the copy. The role has two halves: a trust policy letting s3.amazonaws.com assume it, and a permissions policy granting the exact replication actions. Both blocks below are verbatim from the AWS replication permissions docs. The trust policy:

{
   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Principal":{
            "Service":"s3.amazonaws.com"
         },
         "Action":"sts:AssumeRole"
      }
   ]
}

The permissions policy (source bucket = amzn-s3-demo-source-bucket, destination = amzn-s3-demo-destination-bucket):

{
   "Version":"2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetReplicationConfiguration",
            "s3:ListBucket"
         ],
         "Resource": [
            "arn:aws:s3:::amzn-s3-demo-source-bucket"
         ]
      },
      {
         "Effect": "Allow",
         "Action": [
            "s3:GetObjectVersionForReplication",
            "s3:GetObjectVersionAcl",
            "s3:GetObjectVersionTagging"
         ],
         "Resource": [
            "arn:aws:s3:::amzn-s3-demo-source-bucket/*"
         ]
      },
      {
         "Effect": "Allow",
         "Action": [
            "s3:ReplicateObject",
            "s3:ReplicateDelete",
            "s3:ReplicateTags"
         ],
         "Resource": "arn:aws:s3:::amzn-s3-demo-destination-bucket/*"
      }
   ]
}

Where this goes wrong is quiet: drop the s3.amazonaws.com principal from the trust policy, or leave s3:ReplicateDelete out of the permissions, and replication just doesn't happen. No CloudTrail error, no SNS alert; the destination stays empty. Check the role ARN in the replication config matches what you attached, then test with one object.

The replication configuration file, field by field

With versioning and the role set, you attach a replication config to the source bucket. The command is verbatim from the AWS CLI reference:

aws s3api put-bucket-replication \
    --bucket amzn-s3-demo-bucket1 \
    --replication-configuration file://replication.json

The minimal replication.json (verbatim from AWS docs) already bakes in two defaults that bite people:

{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [
        {
            "Status": "Enabled",
            "Priority": 1,
            "DeleteMarkerReplication": { "Status": "Disabled" },
            "Filter" : { "Prefix": ""},
            "Destination": {
                "Bucket": "arn:aws:s3:::amzn-s3-demo-bucket2"
            }
        }
    ]
}

An empty Filter prefix means 'all objects'; Priority decides which rule wins when two rules match the same object. The two fields worth adding for production are Metrics / ReplicationTime, which switch on lag tracking. Note they live inside the Destination block, per the AWS schema:

{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [
        {
            "ID": "replicate-all-with-metrics",
            "Status": "Enabled",
            "Priority": 1,
            "Filter": { "Prefix": "" },
            "DeleteMarkerReplication": { "Status": "Disabled" },
            "Destination": {
                "Bucket": "arn:aws:s3:::amzn-s3-demo-bucket2",
                "Metrics": { "Status": "Enabled", "EventThreshold": { "Minutes": 15 } },
                "ReplicationTime": { "Status": "Enabled", "Time": { "Minutes": 15 } }
            }
        }
    ]
}

Metrics and ReplicationTime are real fields in the AWS replication schema; enabling them is what makes replication lag visible instead of invisible.

Why are your old objects missing from the replica?

This trips up almost everyone, and it's burned me on migrations that looked green until someone noticed six months of history missing from the replica. AWS says it plainly: 'Objects that existed before you set up replication aren't replicated automatically. In other words, Amazon S3 doesn't replicate objects retroactively.' A rule you add today only covers objects written or changed after it goes live. That 2 TB of history already in the source? It sits there until you backfill it explicitly.

On AWS, the supported way to handle that is S3 Batch Replication, a one-time job that copies existing objects on demand (the older per-rule ExistingObjectReplication option is no longer supported — AWS removed it and points to Batch Replication). Note that backfilling is a large, billable operation for cross-region transfers. MinIO does the same through the --replicate flag instead of a JSON field:

mc replicate add myaistor/mybucket \
   --remote-bucket https://user:secret@minio.mysite.tld/remotebucket \
   --replicate "delete,delete-marker,existing-objects"

The existing-objects value tells MinIO to replicate objects already there. MinIO needs versioning on first: objects written before versioning has no version ID and get excluded. Check the current MinIO docs for the exact mc version subcommand before running that, since the flag name drifts between versions.

Delete markers and KMS keys: the two settings that bite later

Delete markers and KMS encryption both default to the safe-but-surprising option, and neither one errors when it's wrong. First, delete markers. With versioning on, deleting an object writes a delete marker instead of removing data. DeleteMarkerReplication is Disabled by default, so a delete in the source does not delete in the destination, so your DR replica keeps serving the 'deleted' object. Set "DeleteMarkerReplication": { "Status": "Enabled" } if you want deletes to follow, but in a bidirectional setup that can trigger delete storms, so decide on purpose. From production experience I rarely run bidirectional replication with delete-marker sync on; a delete storm across sites is painful and slow to unwind, so I keep delete propagation off and clean up the destination by hand when something is actually gone.

Second, KMS-encrypted objects. If the source uses SSE-KMS, replication needs SourceSelectionCriteria.SseKmsEncryptedObjects.Status = "Enabled" plus a Destination.EncryptionConfiguration.ReplicaKmsKeyID, and the IAM role has to be allowed kms:Decrypt on the source key and kms:Encrypt on the destination key (kms:GenerateDataKey is only needed when you replicate plaintext objects into a destination bucket that has default SSE-KMS/DSSE-KMS encryption). Miss any of that and encrypted objects never show up at the destination while unencrypted ones do. That's a partial, silent failure that's a pain to diagnose. Safest move until the KMS grants are confirmed: keep the same key, or SSE-S3, on both ends.

How do self-hosted engines replicate across regions?

If you run your own S3-compatible storage, the model holds: a rule or a site link copies objects between two endpoints, but the tooling is different. On MinIO, the mc replicate add above is the whole flow (after versioning). On RustFS, cross-region is site replication, which links two or more independent deployments and syncs buckets, object versions, and IAM across them. The commands are from the RustFS rc client:

# Configure one alias per site (repeat for each deployment)
rc alias set site1 https://site1.example.com:9000 \
    <your-access-key> <your-secret-key> \
    --region us-east-1 --bucket-lookup path

# Confirm both sites are reachable
rc ready site1

# Link the sites (first alias receives the admin request)
rc admin replicate add site1 site2

# Inspect replication status from either site
rc admin replicate status site1

RustFS is explicit about what this needs: two or more independent deployments, a stable S3 API endpoint per site, bidirectional connectivity on the S3 API port (normally 9000), trusted TLS certs, the rc client on a secured host, root admin creds at every site, and bucket-versioning support at every site. Bring each deployment up with the verified Docker command:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default console credentials are rustfsadmin / rustfsadmin, and the console listens on port 9001.

The RustFS docs are clear about the limits: site replication is asynchronous (a successful write at one site doesn't mean the other already has it) and provides no DNS failover, no traffic routing, no application recovery orchestration. Plan your own recovery. Distributed Mode is still 'Under Testing' in the Feature & Status table, so treat each linked site as single-node-ready today, not a multi-node cluster, and test the workflow on empty sites before you link production data. RustFS gives you cross-region sync, but not a turnkey active-active setup that resolves conflicts. The docs don't pretend otherwise. If replication is the only reason you'd stand up a second cluster, do the egress math first: the bandwidth you save is usually smaller than a second deployment's run cost, and self-hosting only wins when you already own the hardware or you have a data-residency requirement.

Monitoring replication lag and the cost of cross-region traffic

You can't see replication lag until you turn on metrics. On AWS that means setting Metrics and ReplicationTime in the rule (above) and alarming on the CloudWatch metrics they produce, and without those blocks AWS emits no replication-latency metric at all. ReplicationTime lets you request a target replication window in minutes; AWS also offers S3 Replication Time Control (S3 RTC) for workloads that need a contracted SLA (99.9% of objects replicate within 15 minutes). Alarm on sustained backlog, not a single sample. Async replication falls behind temporarily under bursty writes.

On cost, cloud and self-hosted aren't a tweak apart, they're structurally different. AWS bills inter-region transfer for every replicated byte (rates vary by region pair), so a high-churn bucket racks up real egress. A self-hosted pair, RustFS via rc admin replicate add or MinIO via mc replicate add, moves bytes over your own network, so there's no per-GB cloud egress for the replication itself (you still pay your bandwidth provider). I'm not quoting a per-GB number here because AWS inter-region rates are region-pair-specific and move around. The point that holds: the engine choice decides who sends the egress bill, not whether bytes move.

FAQ

Does S3 cross-region replication copy existing objects?

No. AWS states that objects created before the replication configuration is attached are not replicated automatically. To move historical data, run an S3 Batch Replication job (the older per-rule ExistingObjectReplication option is no longer supported). MinIO covers the same case with the existing-objects value in mc replicate add --replicate.

Does cross-region replication replicate delete markers?

Not by default, and I'd leave it off unless you have a reason. In the AWS replication schema, DeleteMarkerReplication defaults to Disabled, so deleting an object in the source does not delete its replica. Flip it to "Status": "Enabled" only when the destination is a true mirror and you accept that a bad delete now spreads to both sides.

Does the destination bucket need versioning for S3 replication?

Yes, on both ends, no exceptions. Replication keys off version IDs, so the destination has to track them too. One gotcha: the change isn't instant. AWS says wait about 15 minutes after you flip versioning on before you start writing, or you'll replicate into a half-propagated bucket and wonder why objects are missing.

Can MinIO or RustFS do cross-region S3 replication?

Yes. MinIO uses mc replicate add with a --replicate flag that controls delete, delete-marker, and existing-object replication. RustFS uses site replication via the rc client: rc admin replicate add site1 site2 links two deployments and syncs buckets, object versions, and IAM. Both require versioning on the source and (for RustFS) bidirectional network access on the S3 API port.

How do I monitor S3 replication lag?

On AWS, enable the Metrics and ReplicationTime blocks inside the replication rule; only then does AWS emit replication-latency CloudWatch metrics you can alarm on. Without those blocks, lag is not exposed by default. RustFS exposes per-site replication status through rc admin replicate status site1 --metrics, which reports backlog and metrics from each deployment.

Originally published at RustFS blog. If you want S3-compatible replication without the per-GB cloud egress, RustFS is open source under Apache 2.0 and links regions through site replication. read the docs or download to try it.

Backblaze B2 vs Self-Hosted S3: Which Saves More Money?

Ethan Carter — Sun, 16 Aug 2026 13:21:00 +0000

Backblaze B2 vs Self-Hosted S3: Which Actually Saves More Money in 2026?

Backblaze B2's pricing is about as simple as cloud storage gets: $0.00695/GB/month (~$6.95/TB/mo) for storage, no charge for uploads, and free egress up to 3× your stored capacity (then $0.01/GB). No storage tiers, no retrieval classes, no egress spreadsheet. For backups, archives, and media libraries that combination tends to be the practical choice.

Simple pricing and low total cost are not the same thing once your data passes a certain size. This article compares where B2 wins, where self-hosted S3 (RustFS, MinIO) wins, and the data volume where the two cross over.

Key Stats (Backblaze figures verified 2026-08-04)

Metric	Figure	Source
B2 pay-as-you-go storage	$6.95 / TB / month ($0.00695/GB)	Backblaze pricing
B2 egress	Free up to 3× monthly storage, then $0.01/GB	Backblaze pricing
Always-free tier	First 10 GB of storage	Backblaze pricing
Transactions	Class A/B/C free; Class D $0.004 per 10,000 calls	Backblaze pricing
B2 regions	4 — US West, US East, EU Central, CA East	Backblaze docs
Self-hosted crossover	~7 TB (our cost model, assumptions below)	This article

Backblaze B2: What You Get for $0.00695/GB

Feature	Included?	Notes
S3-Compatible API	✅	"S3 Compatible API" — mostly core operations
Storage	$0.00695/GB/month (~$6.95/TB/mo)	~70% cheaper than AWS S3 Standard
Uploads (Class A)	Free	No ingress fee
Downloads (Class B)	Free up to 3× storage/mo, then $0.01/GB	Generous egress, no per-GB surprise under 3×
Minimum commitment	None	Pay for what you use
Data durability	11 nines claimed	Standard for industry
Presigned URLs	✅	Downloads and uploads; browser POST uploads not supported
Server-side encryption	✅	SSE-B2 (Backblaze-managed) or SSE-C (customer-managed)
API transactions	Free (Class A/B/C)	Class D $0.004 per 10,000 calls; first 2,500/day free
Multipart Upload	✅	Yes
Lifecycle rules	⚠️ Basic	Hide/delete only, no tier transitions
Object Lock / WORM	✅	Governance + compliance retention, legal hold, bucket default retention
Cross-region replication	⚠️ Cloud Replication	Within or between regions, but a cross-region target needs a second account; Native API/console only, 2 rules per bucket
ACLs / IAM roles / object tagging	❌	Officially listed as unsupported in the S3-Compatible API

Where B2 shines: Backups, archives, media asset libraries, and disaster-recovery targets. These are workloads where you write once, read occasionally, and want a bill you can predict from one line item.

Where B2 struggles: Heavy egress above 3× your stored capacity, where the $0.01/GB overage applies. Also any workload that needs lifecycle tier transitions, object tagging, ACLs, or IAM roles. Backblaze lists each of those as unsupported in its S3-Compatible API, and B2 lifecycle rules only hide, delete, or cancel unfinished large uploads.

The Crossover Math: When Self-Hosted Wins

Monthly Data Stored	Backblaze B2	Self-Hosted (RustFS)	Winner
1 TB	$6.95	~$38 (hardware amortized + ops)	B2
10 TB	$69.50	~$53	RustFS (-24%)
50 TB	$347.50	~$142	RustFS (-59%)
100 TB	$695	~$253	RustFS (-64%)
500 TB	$3,475	~$917	RustFS (-74%)

Assumptions: B2 egress within the free 3×-storage tier (so B2 cost ≈ storage cost). Self-hosted hardware amortized over 36 months. Ops cost = 0.15 FTE allocation.

For a typical backup or archive workload the crossover lands near 7 TB, inside the 5–10 TB band. Below it, B2's convenience usually wins on price and on not having to run anything. Above it, self-hosted pulls clearly ahead on cost.

The Hidden Factor: Download Costs

Backblaze changed its pricing model: egress is now free up to 3× your average monthly storage, and free beyond that through CDN or compute partners. Only egress above 3× storage is billed, at $0.01/GB. For most backup and archive workloads that makes download cost effectively zero, a real change from the old per-GB download charges.

Scenario	Monthly Egress Cost (B2)
1 TB stored, 10% egress/mo (100 GB)	$0 (within 3× free tier)
10 TB stored, 25% egress/mo (2.5 TB)	$0 (within 3× free tier)
50 TB stored, 50% egress/mo (25 TB)	$0 (within 3× free tier)
100 TB stored, 400% egress/mo (400 TB)	$1,000 (100 TB overage × $0.01)

Low-egress backup and archive workloads pay nothing for B2 downloads. Heavy egress (a CDN origin, ML training pulls, analytics above 3× storage per month) starts to cost. Self-hosted S3 stays at $0 either way, because the bandwidth is yours.

Self-hosted S3: download cost = $0 (it's your network, your bandwidth).

Decision Framework

Choose Backblaze B2 If:

✅ You store < 10TB
✅ Primary workload is backup/archive (write-once, read-rarely)
✅ You want zero operational responsibility
✅ Download ratio is < 20% of stored data monthly
✅ You don't need lifecycle tier transitions, object tagging, ACLs, or IAM roles

Choose Self-Hosted S3 (RustFS/MinIO) If:

✅ You store > 10TB
✅ Workload is active (frequent reads AND writes)
✅ You want predictable $0 egress at any scale (B2's free egress caps at 3× storage/mo)
✅ You have infra capacity or want to build it
✅ You need S3 surface B2 doesn't expose (object tagging, ACLs, IAM roles, lifecycle transitions)

Choose Both (Hybrid):

A lot of teams keep B2 as their off-site cold target and run self-hosted S3 for hot, active data:

Write → Self-hosted S3 (hot, fast, zero egress)
       │
       └── Lifecycle copy → B2 (cold archive, off-site)

The S3 API sits on both sides, so the same tooling moves data either way. The two tiers just carry different cost profiles.

TL;DR

B2 has the simplest cloud storage pricing ($0.00695/GB store, free egress up to 3× storage, then $0.01/GB). Strong fit for backups and archives.
Self-hosted S3 becomes cheaper around 7TB in our model, depending on download activity.
B2 egress is now free up to 3× storage — only heavy egress (>3×) costs; self-hosted is always $0.
B2 does support Object Lock (governance/compliance + legal hold) and bucket replication, but not object tagging, ACLs, IAM roles, or lifecycle tier transitions. Check your requirements against the list above.
Hybrid is common: self-hosted for hot data, B2 for cold archive. The shared S3 API makes the split workable.

RustFS — Apache 2.0, S3-compatible object storage you run yourself, with bucket replication, versioning, and event notifications available today (lifecycle management is still marked under testing). Download here or read the code on GitHub.

FAQ

Is Backblaze B2 really S3-compatible?

Mostly. B2's S3-Compatible API covers the core operations (PutObject, GetObject, ListObjects, Multipart Upload, DeleteObject), so S3-aware tools like rclone, restic, and Cyberduck work against it unchanged. Presigned URLs work for both downloads and uploads, and server-side encryption is supported via SSE-B2 or SSE-C. Backblaze officially lists five gaps: ACLs, IAM roles, object tagging, website configuration, and browser-based POST uploads to presigned URLs. Cloud Replication is also absent from the S3 API. It runs through the Native API or the web console instead.

Is Backblaze B2 cheaper than AWS S3?

On storage, clearly: $0.00695/GB against $0.023/GB for S3 Standard, about 70% less. Egress widens it further. B2 is free up to 3× your monthly storage and $0.01/GB above that, while AWS charges $0.09/GB after its free allowance. The catch is reach and feature surface. B2 runs 4 regions (US West, US East, EU Central, CA East) against AWS's 30-plus, and it has no storage classes to transition between. For backup and archive workloads, B2 ends up far below AWS S3 once egress is in the picture. In our cost model that gap runs to roughly 60–80%.

Does Backblaze B2 have a free tier?

Yes, and it doesn't expire. The first 10 GB of storage is always free, egress is free up to 3× your monthly storage, and Class A, B, and C transactions are free on pay-as-you-go. Class D calls cost $0.004 per 10,000, with the first 2,500 per day free. That is enough room to run a personal backup, a small-business archive, or a proof of concept without ever generating a bill, and it is more generous than most cloud storage free tiers.

When does self-hosted S3 storage become cheaper than Backblaze B2?

Around 7 TB in our model, which amortizes hardware over 36 months and charges 0.15 FTE for operations. At 1 TB, B2 wins outright ($6.95 versus roughly $38 self-hosted). At 10 TB, self-hosted runs about 24% less; at 100 TB, about 64% less; at 500 TB, about 74% less. Egress shifts the line. B2's free 3× allowance covers most backup patterns, but a CDN origin or an ML training set pulling more than 3× storage per month pays $0.01/GB, while self-hosted egress stays at $0.

Does Backblaze B2 support Object Lock and cross-region replication?

Both, with caveats. Object Lock is fully supported — governance and compliance retention modes, legal hold, and bucket-level default retention — and lifecycle rules will not delete a version that Object Lock protects. Replication runs through Backblaze Cloud Replication, which copies buckets within or between regions, but a cross-region target requires a second Backblaze account (an account is tied to one region), it allows only two rules per bucket, and it is driven by the Native API or web console rather than the S3 API.

Sources

Every Backblaze figure and feature claim above was checked against Backblaze's own documentation on 2026-08-04:

Storage price, free egress, free tier, transaction classes — Backblaze B2 pricing
Object Lock (governance/compliance modes, legal hold, default retention) — Backblaze Object Lock docs
Cloud Replication and the cross-region second-account requirement — Backblaze Cloud Replication docs
Lifecycle rule actions (hide, delete, cancel unfinished large files) — Backblaze Lifecycle Rules docs
S3-Compatible API gaps (ACLs, IAM roles, object tagging, website config, POST uploads) — Backblaze S3-Compatible API docs
Region list — Backblaze data centers and regions
RustFS license and feature status — RustFS on GitHub

AWS S3 list prices ($0.023/GB Standard, $0.09/GB egress) are AWS published pricing.