DEV Community: Ethan Carter

Benchmarking S3 Throughput: How to Measure Without Lying

Ethan Carter — Wed, 14 Oct 2026 13:58:00 +0000

An S3 throughput benchmark is only as honest as its setup. Most published numbers hide three variables that move results by 5x or more: where the client runs, how big the objects are, and which second of the run got reported. This article shows a workflow with warp that keeps those variables visible, on a real single-node server.

Metric (first-hand run)	Value
GET throughput, 1 MiB objects, 8 workers	663 MiB/s avg; 1s splits ranged 530 to 749 MiB/s
PUT throughput, same setup	102 MiB/s avg; 1s splits ranged 63 to 128 MiB/s
GET throughput, 10 KiB objects, 8 workers	136 MiB/s, 13,965 obj/s (27 s run); repeat run: 107 obj/s, 10,708 obj/s
PUT throughput, 10 KiB objects, 8 workers	3.5 MiB/s, 360 obj/s; median latency 21 ms
Mixed workload total (GET/STAT/PUT/DELETE)	266 MiB/s, 443 obj/s
GET TTFB median	10 ms (1 min run), 5 ms (concurrency 4)
Environment	RustFS 1.0.1, single node, laptop (Ryzen 7 5800H, 16 GB RAM, NVMe SSD), warp 1.3.1, loopback

Treat every number in that table as a relative reference point, not a capacity claim. The server ran on loopback, on a developer machine, with a client on the same host. That limitation is the first thing this article teaches you to disclose.

What a throughput number actually measures

A throughput number is the output of three components in series: the benchmark client, the network path, and the storage server. Whichever one is slowest sets the ceiling, and the benchmark report will not tell you which one it was. When I ran warp get against RustFS 1.0.1 on loopback, the client and the server shared one machine, so the network component was effectively zero and the disk did most of the talking.

That setup is fine for comparing versions of the same server. It is useless for predicting what a remote client will see. In my run, GET averaged 663 MiB/s with 8 concurrent workers on 1 MiB objects. A client on another host, behind one 1 GbE link, would cap near 125 MiB/s and blame the server for the difference. Name the path in every result you publish, or the number means nothing.

PUT is not GET

The fastest way to spot an untrustworthy storage benchmark is one that reports a single throughput number. On the same server, same run length, same concurrency, my PUT result was 101.6 MiB/s while GET was 663.3 MiB/s, a 6.5x gap on one node. The write path persists data before it acknowledges; the read path serves from page cache after the first fetch. No amount of tuning closes that gap entirely, and pretending it does not exist misleads every reader.

The mixed benchmark shows the interaction effect too. Under a combined GET/STAT/PUT/DELETE workload, GET dropped from 663 MiB/s standalone to 199.6 MiB/s, and PUT fell from 101.6 to 66.3 MiB/s. Writes steal IOPS and cache from reads. If your real workload writes while others read, the mixed number, not the standalone one, is your planning baseline. Publish both or publish neither.

How long should a benchmark run?

Long enough that warm-up stops mattering. The first seconds of a run are cold: empty caches, open connections, and the server allocating space. warp handles this explicitly. Per its documentation, the analysis window starts when every thread has completed one request and ends when the last thread starts its final one, which is why the analyzed time is slightly below the requested duration.

Run length also drives variance. My 1 minute GET run split into 57 one-second buckets: fastest 749 MiB/s, median 672.7, slowest 530.5. That is a 1.4x spread between best and worst second, on loopback, with nothing else running. A 10 second run would have waved that spread around as noise, and a 5 minute run narrows the confidence interval further. Short runs also disagree with each other: two independent 10 KiB GET runs, one 30 seconds and one 20 seconds, averaged 13,965 and 10,708 obj/s, a 23 percent gap between runs measured minutes apart on an idle machine. For anything you plan to publish or reuse in capacity planning, run at least a minute per operation type, run it more than once, and report the spread, not a single second.

Why object size changes the answer

Throughput in MiB/s and objects per second are different claims, and object size decides which one you can honestly make. At 1 MiB objects, my GET run delivered 663 obj/s. At 10 KiB objects on the same server, the same 8-worker setup delivered 13,965 obj/s but only 136 MiB/s: 21 times more objects, a fifth of the bandwidth. Per-request overhead dominates at small sizes, and which of the two numbers matters depends entirely on what your traffic looks like.

The write side told an even sharper story. PUT at 1 MiB managed 101.6 obj/s; PUT at 10 KiB managed 360 obj/s. Objects a hundred times smaller bought only 3.5 times more objects per second, and median latency sat at 21 ms per request. Whatever the write path is doing per request, it does not shrink in proportion to object size, so small-object write workloads hit that per-request floor long before they stress bandwidth. If your production traffic is millions of tiny uploads, benchmark at that size, not at 1 MiB, because the 1 MiB number will flatter the server.

warp defaults each benchmark type to its own object size and lets you override it with --obj.size. It also supports randomized sizes (--obj.randsize), which the documentation describes as log2-distributed: equal object counts per doubling of size, so the average lands well below the maximum you pass. Pick the size your production traffic actually has. If you do not know it yet, measure that first, because a benchmark with the wrong object size is an answer to a question nobody asked.

Report the distribution, not the best second

Every benchmark produces a distribution, and the dishonest version of a report quotes its maximum. warp prints three 1-second aggregates by default: fastest, 50% median, and slowest. My GET run: 749, 672.7, 530.5 MiB/s. My PUT run: 128.2, 101.7, 63.4 MiB/s. The honest summary line is the median with the range next to it, because your production p50 will look like the median, and your worst tenancy-neighbor day will look like the slowest second.

Latency deserves the same treatment. In the GET run, TTFB median was 10 ms, the 99th percentile 16 ms, the worst single request 36 ms. In the mixed run, STAT requests answered in a median 2.5 ms while DELETE sat at 39.1 ms median, because deletes wait on real I/O. Quote percentiles with the operation name attached. An average latency across mixed operations is a number that describes nothing.

Where should the benchmark client run?

As close to the server as honesty allows, which is rarely on the same box. Loopback removes the network, so it measures the server plus the client kernel, and results only transfer to deployments with a similar path. My numbers came from loopback on purpose: the goal was comparing server versions, not predicting a datacenter. When you need a network-realistic number, run the client on a separate machine on the same segment, and run it from several at once.

warp supports both modes. Multiple hosts can be passed comma-separated (--host=10.0.0.1:9000,10.0.0.2:9000), and for larger setups a warp server can drive remote warp client processes with --warp-client=client-{1...10}, each applying the same concurrency. The documentation's example of a distributed GET run uses 10 clients against 16 servers. Distributed runs also change interpretation: parameters apply per client, so total load multiplies. Write down the client topology next to the result.

Watch the server while the client runs

The warp report tells you what the client saw. It never tells you why. While a 20-second GET run against 10 KiB objects was in flight, I sampled the server machine every two to three seconds: CPU sat between 94 and 100 percent for the entire window, and physical disk reads stayed between 0 and 1.1 MB/s while warp reported about 105 MiB/s being served. Every one of those bytes came from the operating system's page cache, not from the disk. On a loopback setup the benchmark client shares those same saturated cores with the server, so the number measures the whole machine, not just storage.

Sampling during the run is what turns a suspicious number into an explained one. If CPU is pegged, you are client- or processor-bound; if the disk is pegged, the storage path is the ceiling; if neither is busy while throughput looks high, you are benchmarking the cache. Any process-level monitor works: CPU percentage, disk read/write bytes per second, free memory (watch it fall while the cache fills). One caveat from my own setup: my CIM-based sampler needed two to three seconds per sample, so treat those numbers as coarse, and put the sampler on the machine the server runs on.

Your GET number is probably a cache number

Cold reads are what your cache-miss traffic will actually see, and my runs did not measure them. The GET figures in this article are warm-cache numbers: the prefill wrote every object moments before the reads, and the page cache absorbed them. Getting closer to a cold number takes one of three things: a working set larger than physical memory so the cache cannot hold it, dropping the OS cache between runs (Linux has /proc/sys/vm/drop_caches; Windows exposes no supported equivalent, which is why my Windows-hosted numbers stay warm), or --obj.randsize to spread reads across a much wider object set. Say which one you did, because a GET figure quoted without its cache state is a number with an asterisk nobody wrote down.

Benchmarks need capacity too

This one cost me a failed run. The 1 minute PUT benchmark at 101.6 MiB/s wrote roughly 5.8 GB of test objects; the earlier GET run had already filled most of a bucket. When I raised the object size to 10 MiB, uploads failed with a generic InternalError from the server, and the benchmark aborted before it started. The server log showed the real cause: os error 112, storage full, on the data volume.

Two lessons worth the tuition. First, size your test volume before the run: benchmark data is real data, warp writes it at full speed, and a bucket can grow by gigabytes per minute. Second, a client-side error message is not a diagnosis. The S3 error XML said "please try again"; only the server log said "disk full". When a benchmark fails, read the server log before you conclude anything about the server.

A warp workflow you can copy

warp is MinIO's open-source S3 benchmark tool, and the workflow below uses only flags and syntax documented in its README. It creates and completely cleans a bucket called warp-benchmark-bucket before and after each run, so never point it at a bucket holding data. Three runs cover the planning basics:

warp get --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8
warp put --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8
warp mixed --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8

Each run prints the per-operation report with fastest/median/slowest 1-second splits, then saves raw request data for later analysis with warp analyze --analyze.op=GET --analyze.v <file>.csv.zst, which reproduces segmented statistics per request-size band. One caveat from my run: warp 1.3.1 on Windows wrote the .json.zst run metadata but not the raw CSV, so treat the analyze step as version- and platform-dependent and capture the console report as your primary record.

Frequently Asked Questions

What is warp?

warp is an open-source S3 benchmarking tool from MinIO that measures GET, PUT, DELETE, STAT, and mixed workloads, saves raw request data, and reports throughput, request latency, and TTFB distributions with warm-up excluded from the analysis window.

Why is PUT so much slower than GET on the same server?

The write path persists objects before acknowledging while the read path is served after a single fetch, often from cache. In my single-node loopback run the gap was 6.5x (102 MiB/s PUT vs 663 MiB/s GET at 1 MiB objects); the exact ratio depends on hardware, but a large gap is normal, and a benchmark showing only GET throughput is incomplete.

What concurrency should I use for an S3 benchmark?

Start where your application actually operates, then sweep. warp's --concurrent flag sets workers per client; too few workers leave the server idle and too many measure your client's queuing instead of the server. My 8-worker loopback runs saturated a single node comfortably; the honest method is reporting throughput at several concurrency levels.

How do I benchmark more than one S3 server?

Pass hosts comma-separated (--host=10.0.0.1:9000,10.0.0.2:9000) so warp alternates between them, or use distributed mode with remote warp client processes driven by --warp-client=client-{1...10}. Note that flags apply to each client individually, so total load scales with client count.

Why are my GET numbers suspiciously high?

Check whether the operating system's page cache is serving the reads. In my runs, warp reported 105 MiB/s of GET traffic while physical disk reads sat near zero; the objects had been written moments earlier and never left memory. Options for a more realistic figure: a dataset larger than RAM, dropping the cache between runs on Linux, or --obj.randsize to spread reads across a wider object set.

What number should I publish from a benchmark?

The median with its range and the full environment: server version, node count, network path, object size, concurrency, and run duration. For example: 663 MiB/s median-bucket 673 MiB/s (530 to 749 range), RustFS 1.0.1 single node, loopback, 1 MiB objects, 8 workers, 1 minute. That sentence is falsifiable, which is exactly what makes it worth publishing.

A note on how this was verified

Article type: this is a hands-on benchmark methodology report based on runs I executed, plus documentation-sourced tool behavior.

The benchmark runs in this article were executed first-hand on 2026-10-12 against a single-node RustFS 1.0.1 (Windows x86_64 binary, GitHub release build dated 2026-10-03) on loopback, using warp 1.3.1 (143bf4f): GET 1 MiB / 8 workers / 1 min, PUT 1 MiB / 8 workers / 1 min, GET 1 MiB / 4 workers / 20 s, mixed 1 MiB / 8 workers / 40 s, GET 10 KiB / 8 workers / 30 s, PUT 10 KiB / 8 workers / 30 s, and a repeat GET 10 KiB / 20 s, all quoted with real console output. Server-side resource samples (CPU, free memory, disk read/write bytes per second) were collected on the same machine during the repeat GET run with a CIM-based sampler polling every two to three seconds; they are coarse and cover one machine. The tool-behavior claims (bucket cleanup, warm-up exclusion, randomized size distribution, multi-host and warp-client flags, the mixed-workload operation mix of GET 45% / STAT 30% / PUT 15% / DELETE 10%) come from the warp v1.3.1 README, not from my runs. Not exercised: any network path (all runs were loopback), distributed warp-client runs, cold-read or cache-bypassed measurements (every GET figure here is warm-cache; the cache section explains the options), object sizes above 1 MiB (the 10 MiB attempt aborted on a full test volume), runs longer than one minute, and the warp analyze step, which could not run because warp 1.3.1 did not persist the raw CSV on Windows in my environment; that limitation is quoted above rather than worked around. Nothing here predicts a production cluster. Re-run the same commands against your own hardware before you plan capacity on any of it.

Sources

warp v1.3.1 README (commands, defaults, warm-up behavior, mixed-workload mix): https://github.com/minio/warp/blob/v1.3.1/README.md
warp releases (v1.3.1 Windows binary used for the runs): https://github.com/minio/warp/releases/tag/v1.3.1
RustFS GitHub repository and releases (1.0.1 Windows binary used for the runs): https://github.com/rustfs/rustfs/releases/tag/1.0.1
RustFS documentation (deployment and compatibility): https://docs.rustfs.com

RustFS S3 Tables: A Built-In Iceberg REST Catalog in an Apache-2.0 Object Store

Ethan Carter — Mon, 12 Oct 2026 13:18:00 +0000

RustFS S3 Tables is an Apache Iceberg REST Catalog built directly into the RustFS object storage kernel, so one process on one port serves both standard S3 and the Iceberg REST Catalog protocol. That removes the separate Hive Metastore or Glue catalog a self-hosted lakehouse normally needs. It is Apache 2.0 with no separate commercial edition.

Item	Value
Feature	Iceberg REST Catalog embedded in the object-store kernel
License	Apache 2.0, no separate commercial edition
Ships in	RustFS 1.0.0
Endpoint	Same port as S3 (`9000`)
Status	Built into the object-store kernel; fully open source (Apache 2.0)
Iceberg format	v1 and v2 supported (v2 default); v3 unsupported

What is RustFS S3 Tables?

RustFS S3 Tables is a catalog service embedded in the storage server rather than deployed next to it. An Iceberg client uses the REST catalog to discover tables and commit metadata changes. It uses the S3 API to read and write the underlying Parquet files. RustFS serves both interfaces on the S3 API port.

That collapses one moving part: a typical self-hosted lakehouse pairs the object store with a catalog such as Hive Metastore, AWS Glue, Nessie, or Polaris, and RustFS folds the catalog into the store. Because the catalog speaks the open Iceberg REST protocol, engines with an Iceberg REST client need no vendor plugin — Apache Spark, PyIceberg, and DuckDB are documented as connecting without one, and enabling a table bucket does not require registering existing Parquet files first. One consequence worth stating plainly: the catalog's metadata lives inside RustFS itself (Iceberg metadata JSON and manifests written to a reserved prefix on the same bucket), so there is no separate metadata failure domain. Your RustFS durability and backup story is also your catalog backup story, and you cannot fall back to an independent metastore.

The control plane and data plane split

Writes separate into two paths. Table creation and snapshot commits travel through the control plane, which is the Iceberg REST Catalog. Parquet data files are written by the compute engine straight into the bucket over the S3 API, with no intermediate layer in the data path.

Two details keep that boundary safe. Metadata pointers live under a reserved internal prefix, so ordinary S3 requests cannot create, overwrite, or delete catalog metadata; a stray aws s3 cp will not corrupt table state. On commits, the catalog runs compare-and-swap version checks to reject stale concurrent writers and uses idempotency keys to absorb duplicate submissions. The transaction boundary is one table. RustFS does not claim multi-table atomic transactions, and the limit is concrete: you cannot commit a schema change to one table and a data load to a second table in a single atomic operation. Workloads that need cross-table consistency — updating a dimension table and a fact table together, for example — must coordinate that in application logic or accept eventual consistency.

From an empty bucket to a working table

The official walkthrough starts with a dedicated bucket, then flips it into a table bucket with a signed request that carries an empty body. Using my-bucket on localhost:9000:

export RUSTFS_ENDPOINT="http://localhost:9000"
export AWS_ACCESS_KEY_ID="<your-access-key>"
export AWS_SECRET_ACCESS_KEY="<your-secret-key>"
export AWS_DEFAULT_REGION="us-east-1"

aws --endpoint-url "$RUSTFS_ENDPOINT" s3api create-bucket --bucket my-bucket

curl --fail-with-body --silent --show-error \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  --header "x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" \
  --request PUT "$RUSTFS_ENDPOINT/iceberg/v1/buckets/my-bucket"

Both the enable call and the GET status check return HTTP 200, with enabled: true and catalog-type: iceberg-rest in the response. I confirmed this end to end on a single-node RustFS 1.0.1 run; see the note at the end for exactly what was executed.

Namespaces and tables are plain REST calls under the warehouse prefix. Namespace creation with curl:

curl --fail-with-body --silent --show-error \
  --aws-sigv4 "aws:amz:us-east-1:s3" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  --header "Content-Type: application/json" \
  --header "x-amz-content-sha256: UNSIGNED-PAYLOAD" \
  --request POST "$RUSTFS_ENDPOINT/iceberg/v1/my-bucket/namespaces" \
  --data '{"namespace":["my-namespace"]}'
# {"namespace":["my-namespace"],"properties":{}}

One signing detail cost me a retry: the enable call signs an empty body, so it carries the well-known empty-sha256 header, but reusing that header on the namespace POST (which carries a JSON body) is rejected with Sha256Mismatch (HTTP 400). Body-carrying requests need x-amz-content-sha256: UNSIGNED-PAYLOAD or the actual body hash instead.

Table creation follows the same pattern against /namespaces/my-namespace/tables and returns the full Iceberg metadata document. Three fields in that response are worth reading: metadata-location points at the metadata JSON under a reserved .rustfs-table/ prefix on the same bucket, which is visible confirmation that catalog state lives in ordinary S3 objects next to your data; metadata.format-version is 2, matching the documented v2 default; and config reports rustfs.credential-vending: unsupported, so the client must hold its own S3 credentials rather than receiving temporary ones from the catalog.

From there an Iceberg client needs four settings. The REST catalog URI is http://localhost:9000/iceberg, and the client appends /v1. The warehouse is the bucket name, not an S3 URI or an AWS ARN. REST auth is SigV4 with signing name s3 and region us-east-1. The S3 file endpoint is the same host, with path-style addressing.

The docs flag one trap that first-timers hit: REST request signing and S3 file access are configured separately, even for the same account. Configure only the S3 side and the client returns 403 until you add the catalog's SigV4 credentials.

Which engines work today?

RustFS maintains a support matrix with different confidence levels per client. Two are automated. PyIceberg covers namespace and table creation, append, reload, scan, and catalog operations. DuckDB Iceberg 1.5.5 covers single-table create, insert, update, delete, and merge, plus schema evolution, snapshots, and concurrent writes.

Apache Spark has an opt-in live harness, so validate your exact Spark and Iceberg versions. Trino has a manual read-only probe and write compatibility is not claimed. StarRocks is documented as an external-catalog read path only. Databend has a live harness limited to an S3-stage read probe of table data files, with no claimed Iceberg REST integration.

At the endpoint level, /iceberg/v1 is the canonical prefix and /_iceberg/v1 is a MinIO AIStor-style compatibility alias. RustFS supports both.

My own run puts one caveat on the matrix's confidence levels. With PyIceberg 0.12.0 against RustFS 1.0.1, the catalog read path works over SigV4: namespace listing and table load return clean responses. The append commit does not. PyIceberg signs catalog requests with botocore's generic SigV4 signer, which sets the S3-style x-amz-content-sha256 header on empty-body requests but not on body-carrying ones; RustFS requires that header on its catalog endpoints, so the commit is rejected with missing header: x-amz-content-sha256. The matrix lists PyIceberg as automated-tested, so treat this as a version-sensitive area rather than a permanent gap: pin your PyIceberg version and run one append before you build on it. Signing is where client compatibility actually breaks today. A separate, still-open issue (#8103, filed September 2026) reports Spark failing metadata-table lookups for a different signing reason: Iceberg's signer double-encodes a URI path that RustFS canonicalizes the S3 way, turning what should be a 404 into a 403.

What's in scope, and what isn't

Before you build on it, here is the scope RustFS documents. Iceberg format v1 and v2 are supported with v2 as the default. Staged table creation, purge-on-drop, and Iceberg v3 are not supported. There is no SQL execution engine, no multi-table atomic transactions, and no independent active-active writes across regions. RustFS implements the open Iceberg REST Catalog protocol and does not claim full AWS S3 Tables control-plane compatibility, so "open-source S3 Tables" means a free, open-protocol Iceberg catalog rather than a drop-in AWS S3 Tables clone. Metadata deletion and background maintenance are off by default, with no built-in periodic scheduler. Dropping a table removes the catalog entry but keeps the underlying objects, so recursive deletion of paths still referenced by snapshots would lose data.

Where the catalog state lives, and how it fails closed

The default catalog backing is object-backed: RustFS persists catalog state as ordinary S3 objects — Iceberg metadata JSON, manifests, and the current pointer file — under a reserved internal prefix on the same bucket, not in a separate embedded database. That is why the backup story for your data is also the backup story for your catalog. The backing is configurable with RUSTFS_TABLE_CATALOG_BACKING; a durable-strong mode exists for deployments that want stronger durability than the default object-backed state, at the cost of a more involved cutover.

Because the catalog is objects plus a CAS-driven pointer, it fails closed by design. The support matrix tracks concrete cases: a stale commit token or stale base metadata returns a conflict without advancing the table pointer; missing metadata, manifest, data, or delete objects fail closed before a commit or maintenance step can advance state; concurrent writers resolve to a single winning CAS with retryable conflicts for the losers; catalog and ordinary S3 permission denials both block a data-plane bypass; and a stale maintenance plan fails closed before any object deletion. Across those cases the direction is the same: the system refuses to advance rather than risk corruption, and each guarantee is tied to a specific RustFS build and client version, not something to assume without testing your exact stack.

Upgrade risk concentrates in the durability-strong cutover. Moving to durable-strong backing requires a preflight reporting SNAPSHOT_MATERIALIZED with no blockers, an exclusive migration fence that drains in-flight writes, and a restart before admitting writers. Snapshot format uses a two-gate switch (RUSTFS_TABLE_CATALOG_STRONG_SNAPSHOT_V2 plus a fleet-confirmed gate); once a version-2 snapshot is persisted, rolling catalog writers back to a binary that only reads version 1 fails closed rather than replacing in-process state. The rollback red line is explicit: after v2 is on disk, do not revert to a v1-only binary. Keep the object-backed catalog backup in your restore set until durable-strong has cleared your retention window, and rehearse the disaster-recovery path on an isolated bucket before it carries data you cannot lose.

RustFS S3 Tables next to the alternatives

MinIO AIStor Tables. MinIO offers a comparable embedded-catalog capability, but only inside AIStor, its commercially licensed product line. AIStor is a separate binary from the AGPL-3.0 MinIO Object Store, and MinIO prices AIStor by capacity at roughly $0.02 per GB per month. The MinIO Object Store entered maintenance mode in December 2025 and no longer takes new features, so the open-source line does not carry this capability. RustFS ships the same class of capability in its Apache-2.0 core, with no community-versus-enterprise split. Licensing and deployment model are where they differ, not feature scope: if you want to drop the separate catalog without taking on a commercial license, RustFS is the lower-friction option, while AIStor brings a more structured commercial-support and compliance story.

Standalone open-source catalogs (Apache Polaris, Project Nessie). Polaris and Nessie are independent, Apache-2.0 REST catalogs you deploy and operate yourself, and both sit in front of any S3-compatible store. They are the genuine alternative if you want catalog/object separation rather than embedding: one catalog can serve several storage backends, and you control its versioning and access policy directly. The cost is exactly the moving part RustFS removes — you run, secure, back up, and upgrade a second service, and you own the wire between catalog and storage. RustFS trades that separation for a single process with one upgrade surface and one backup domain.

On maturity the trade is real: Polaris and Nessie are established projects with broad production adoption and active communities, while RustFS S3 Tables shipped with the 1.0.0 GA and has a thinner third-party deployment track record so far. If an existing operational playbook and catalog maturity weigh more than avoiding a second service, the standalone catalogs are the lower-risk starting point today; RustFS is the bet on a smaller operational footprint that is still closing that gap.

How to choose. The line is "embed vs separate," not "RustFS vs everyone else." If your priority is not running an extra service, RustFS or AIStor fit. If you already standardize on Polaris or Nessie across several backends, RustFS can sit underneath as just the S3 layer. If you need a managed, vendor-run catalog, the comparison shifts to AWS Glue or a cloud warehouse's native catalog rather than self-hosted Polaris/Nessie or RustFS.

Before you take this to production

Pin the version. Use RustFS 1.0.0 or later with a fixed tag rather than latest, and test in an isolated bucket. Give the table bucket its own lifecycle thinking, because table buckets are excluded from ordinary lifecycle expiration and enabling the mode on an existing bucket changes how those rules apply.

Configure both credential paths. Use least-privilege IAM for the catalog actions (admin:SetTableBucket, admin:GetTableBucket, and the namespace and table actions) plus separate S3 permissions for table file reads and writes. Leave metadata deletion and background maintenance off until you have reviewed the retained references, and never recursively delete S3 paths that snapshots may still point to. Start with DuckDB or PyIceberg on a single node to prove create, write, read, and drop before you scale out.

Namespaces group tables, but most of the isolation that matters lands on S3 prefixes, not on the namespace itself. The bucket-level enable response reports credential-scope: warehouse-prefix, and each table's config narrows it to a per-table UUID prefix (s3://my-bucket/tables/<uuid> in my run), which is the scope the catalog expects client credentials to cover. Two consequences follow. You cannot pre-provision a data-access policy for a table's prefix before that table exists, so namespace-level separation gives you organization rather than enforcement. And the realistic granularity options are coarser than the object model suggests: one credential set per producing application, strict prefix conventions, or a bucket per tenant when boundaries must be hard. Keep the catalog admin actions on a separate identity from data-plane writers either way.

Plan metadata maintenance up front. Metadata deletion and background maintenance are off by default and there is no built-in periodic scheduler, so dropping a table removes only the catalog entry and leaves the underlying Parquet files in place — live snapshots keep referencing them and keep occupying storage. Before a table carries real volume, set up a periodic cleanup job (your engine's expire_snapshots / remove_orphan_files, or a scheduled sweep of unreferenced paths) and never recursively delete S3 paths that live snapshots still reference.

The cleanup jobs themselves have teeth. remove_orphan_files deletes any data file current metadata does not reference, so a writer that crashes or pauses mid-commit leaves uncommitted files behind, and a cleanup pass with an aggressive older_than window can delete files a writer is about to commit — the Iceberg docs default that window to days, not minutes, for exactly this reason. expire_snapshots shrinks history on purpose, which is precisely what breaks point-in-time recovery: expire snapshots before your backup has copied them, and a restore lands on a newer state than the one you meant to keep. And every commit writes a fresh metadata JSON plus manifest list, so a table fed by many small commits accumulates metadata objects even while its data barely grows; until snapshots expire, all of it stays in the prefix your catalog reads on every table load.

Treat the catalog's reserved prefix as part of your backup set, since there is no independent metadata store to restore from.

FAQ

Is RustFS S3 Tables free and open source?

Yes. RustFS is Apache 2.0, and S3 Tables ships as a built-in feature of the storage server with no separate commercial edition.

Do I still need Hive Metastore or AWS Glue?

No. The Iceberg REST Catalog runs inside RustFS, so you do not deploy or operate a separate catalog service.

Which query engines work with RustFS S3 Tables?

Any client that speaks the Iceberg REST Catalog protocol. PyIceberg and DuckDB are automated in the RustFS test matrix; Spark and Trino have live harnesses; StarRocks and Databend are narrower.

Is RustFS S3 Tables production-ready?

It is a built-in feature of the Apache-2.0 storage server with no separate commercial edition. As with any feature, validate it against your own workload and keep a rollback path before it carries critical data.

How does RustFS S3 Tables differ from AWS S3 Tables?

AWS S3 Tables is a managed commercial service. RustFS implements the open Iceberg REST Catalog protocol on a self-hosted, Apache-2.0 stack and does not claim full AWS S3 Tables control-plane compatibility.

A note on how this was verified

Article type: hands-on control-plane walkthrough plus a systematic read of the official RustFS documentation, GA announcement, and published support matrix.

The walkthrough above was executed first-hand on a single-node RustFS 1.0.1 (Windows x86_64 binary, release build dated 2026-10-03) on 2026-10-11: bucket creation, table-bucket enablement, status check, namespace creation, table creation, and table load. A PyIceberg 0.12.0 client was also connected over SigV4: catalog reads (namespace listing, table load) succeeded, and the first append commit was rejected with the missing header: x-amz-content-sha256 error quoted in this article. The responses quoted here are taken from those runs, including the Sha256Mismatch rejection. Not exercised: successful engine-side data writes or scans (the commit path blocked them), DuckDB, concurrency benchmarking of metadata commits, large-volume snapshot or metadata-growth stress testing, failure-injection recovery drills, and the durable-strong cutover with its v2 snapshot gate. The operational cautions reflect documented behavior and known limits, not first-hand war stories. Validate against your own workload and keep a rollback path before it carries data you cannot lose.

Sources

RustFS GA announcement, "RustFS 1.0.0 GA" (2026-09-16): https://rustfs.com/blog/announcing-rustfs-1-0-0-ga/
RustFS S3 Tables documentation (EN): https://docs.rustfs.com/en/administration/data/s3-tables
RustFS S3 Tables support matrix (main): https://github.com/rustfs/rustfs/blob/main/docs/architecture/s3-tables-support-matrix.md
RustFS blog, "RustFS Launches S3 Tables" (2026-09-10): https://rustfs.com/blog/rustfs-s3-tables-iceberg-rest-catalog-quickstart
MinIO AIStor feature set and pricing: https://www.min.io/blog/aistor-overview
RustFS release 1.0.1 binaries (Windows x86_64, used for the hands-on run): https://github.com/rustfs/rustfs/releases/tag/1.0.1
rustfs/rustfs issue #8103, Spark SigV4 metadata-table failure (open as of 2026-10-11): https://github.com/rustfs/rustfs/issues/8103
Apache Iceberg maintenance procedures (remove_orphan_files / expire_snapshots semantics and defaults): https://iceberg.apache.org/docs/latest/spark-procedures/

S3 vs NFS for Kubernetes: Where Each One Actually Wins

Ethan Carter — Fri, 02 Oct 2026 13:15:00 +0000

S3 and NFS solve different Kubernetes storage problems. Use NFS or a CSI file driver when pods need shared, POSIX-style read-write-many file access. Use S3-compatible object storage when apps call the S3 API for immutable blobs, backups, and data lakes. RustFS gives you that S3 layer self-hosted under Apache 2.0.

Key Stats

Dimension	NFS (file)	S3 object storage (RustFS)
Access protocol	NFS v3 / v4 over TCP	HTTP REST (S3 API)
Concurrent pods	ReadWriteMany, native	One writer per object; concurrency via versioning / ETag
Kubernetes mount	csi-driver-nfs (`nfs.csi.k8s.io`, GA, K8s 1.21+)	No native volume type; needs s3fs-fuse or the app SDK
POSIX semantics	Yes (close-to-open consistency)	No — object store; a FUSE layer is only partial
RustFS status	—	Apache 2.0, S3-compatible, single binary; S3 Tables (Iceberg) & MinIO on-disk compat are Preview

What is NFS storage in Kubernetes?

NFS is a network filesystem that lets multiple pods read and write the same files across the network. In Kubernetes you usually consume it through the CSI NFS driver, whose plugin name is nfs.csi.k8s.io and which sig-storage ships as GA for Kubernetes 1.21 and later. The driver needs an existing NFSv3 or NFSv4 server and provisions Persistent Volumes by creating one subdirectory per PVC. NFS is the canonical ReadWriteMany filesystem: many pods on many nodes open the same file at once, which a block volume such as EBS or a local PV cannot do. We reach for it with shared model checkpoints, Jupyter home directories, and CMS uploads. The catch is that throughput tracks your filer, so a single NetApp or TrueNAS appliance becomes both the ceiling and the single point of failure. NFS itself has no built-in replication or tiering, so that lives on the backing storage.

What is S3-compatible object storage in Kubernetes?

S3-compatible object storage presents data as immutable blobs addressed by a key over an HTTP API, not as a POSIX filesystem. In Kubernetes there is no native "s3" volume type, so pods reach it through the application's AWS SDK or an S3 client such as aws s3 or mc, with credentials normally injected from a Secret. RustFS is an Apache 2.0, S3-compatible server built in Rust; its Feature & Status table lists S3 Core, Versioning, Object Lock (WORM), Lifecycle, and Distributed Mode all as Available. The mental model matters more than any benchmark: objects are addressed by key, versioned, and immutable until overwritten. There is no file handle, no in-place append, and no directory tree the kernel understands. For backups, artifacts, data lakes, and any app that already speaks the S3 API, this is the natural fit. We treat NFS and S3 as two separate tiers, not competitors.

The POSIX gap: why S3 doesn't mount like NFS

Object storage is not a filesystem, and that gap is the whole story. If a pod needs a path such as /data/shared/models/ckpt.bin with real POSIX semantics, NFS hands it over natively. S3 gives you a bucket and a key; to make it look like a directory tree you mount it through a FUSE layer such as s3fs-fuse. We have run s3fs-fuse in production, and its own README is blunt about the limits: random writes or appends rewrite the entire object, directory listings suffer network latency, there are no hard links and no atomic renames, and multiple clients mounting the same bucket get no coordination. s3fs-fuse preserves only a "large subset of POSIX" — mode, uid/gid, symlinks — not the full contract. So if your app does in-place edits, file locking, or expects rename() to be instant, a FUSE mount will bite you. Own it up front: S3 wins as an API, loses as a filesystem.

ReadWriteMany: where NFS still wins

ReadWriteMany, meaning many pods writing the same files at once, is NFS territory. Kubernetes access modes are ReadWriteOnce, ReadOnlyMany, and ReadWriteMany; a block volume or local PV is ReadWriteOnce by default, so a second node cannot attach it. NFS was built for concurrent multi-client access over the network, so the csi-driver-nfs driver happily serves 50 replicas of a web frontend from one share. We use NFS for Jupyter home directories and shared training caches where every pod must see the same file tree. The operational cost is real: you run and monitor an NFS server, and a file lock held by a crashed pod can stall others until the lock expires. If your requirement is "shared mutable files across nodes," deploy csi-driver-nfs and stop debating. S3 cannot do this without a FUSE shim, and that shim gives you no POSIX locking.

Immutable blobs, backups, and data lakes: where S3 wins

S3 pulls ahead the moment your data is immutable, append-only, or key-addressed. Backups are the obvious case: our nightly PostgreSQL dumps and CI artifacts land in RustFS as versioned objects, so a bad overwrite is one versionId away from recovery, not a restore-from-tape drama. Object storage also scales horizontally without a single filer ceiling; RustFS Distributed Mode is marked Available, so you grow capacity by adding nodes rather than buying a bigger appliance. Data lakes and Iceberg tables are S3-native: you write Parquet through the S3 API and let the catalog track partitions. RustFS lists S3 Tables (Iceberg REST) as Preview, so the Iceberg control plane is still maturing — we flag that honestly rather than pretend it is GA. For ML datasets, log archives, and any pipeline that streams objects, S3 is the lingua franca, and NFS would only add latency and inode headaches.

How do you choose between them on a real cluster?

Start from the access pattern, not the technology. Ask four questions. First, do pods need POSIX file APIs and in-place edits? If yes, NFS. Second, do many pods write the same files at once? If yes, NFS or a CSI file driver. Third, is the data immutable blobs, backups, or a data lake? If yes, S3. Fourth, does the app already call the S3 API? If yes, S3, and skip the filesystem debate. We draw the line at mutability: mutable-shared-files maps to NFS, immutable-keyed-objects maps to S3. RustFS slots into the S3 side as a self-hosted, Apache 2.0 drop-in you control, with no egress fees and no vendor lock. The anti-pattern is mounting object storage as a filesystem just because the URL is shorter; you pay the FUSE tax for zero benefit. Pick the primitive that matches the workload and stop there.

Running RustFS as your S3 tier in Kubernetes

You can stand up the S3 tier in minutes. RustFS ships Kubernetes Helm charts, marked Available in its Feature & Status, or you can run the container directly. The documented start command, verbatim from the project README, is:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default credentials are rustfsadmin / rustfsadmin — rotate these before production, since they ship as defaults. In Kubernetes we expose the S3 endpoint (port 9000) and the Web Console (9001) through a Service, inject the access and secret key from a Secret, and point pods at the endpoint with the AWS SDK or mc. RustFS is Apache 2.0 and built in Rust; a 3-node Distributed Mode deployment fits commodity hardware. Because it is S3-compatible, existing tooling — aws s3, rclone, the MinIO client — works unchanged. NFS stays a separate concern; RustFS complements it rather than replacing it.

FAQ

Can I mount S3 as a filesystem in Kubernetes?

Yes, via a FUSE layer such as s3fs-fuse (GPLv2), which mounts a bucket as a local path. But s3fs-fuse's own README warns of real limits: random writes rewrite the whole object, there are no hard links, no atomic renames, and no coordination between multiple clients mounting the same bucket. Treat it as a read-mostly or write-once mount, not a POSIX replacement.

Is the in-tree NFS volume deprecated in Kubernetes?

The nfs volume type still appears in the Kubernetes docs, but the maintained, recommended path is the CSI NFS driver (nfs.csi.k8s.io), which sig-storage ships as GA for Kubernetes 1.21 and later. It requires an existing NFSv3 or NFSv4 server and provisions PVs by creating a subdirectory per PVC. For new clusters, install csi-driver-nfs with Helm.

Which should I use for AI/ML training data?

Immutable, large datasets (Parquet, image shards, model checkpoints stored as objects) belong in S3-compatible object storage; RustFS serves them over the S3 API at scale. Shared, frequently-mutated small files or Jupyter home directories belong on NFS with ReadWriteMany. Mix both: object store for the corpus, NFS for scratch.

Can RustFS replace NFS?

No. RustFS is an S3-compatible object store and has no built-in POSIX or FUSE filesystem. Its Feature & Status table lists S3 Tables (Iceberg) and MinIO on-disk compatibility as Preview and does not list FUSE or POSIX. Use RustFS for the object tier and keep a CSI file driver such as csi-driver-nfs for shared files.

How do pods access RustFS?

Through the S3 API: inject the access and secret key from a Kubernetes Secret and point the AWS SDK or mc at the RustFS Service endpoint (port 9000). There is no native Kubernetes volume type, so apps must speak S3. If you must have a filesystem view, run s3fs-fuse as a sidecar or init container that mounts the bucket.

S3 Storage Tiering: Pushing Cold Data Down the Ladder

Ethan Carter — Thu, 01 Oct 2026 13:14:00 +0000

S3 storage tiering is the practice of moving objects from hot, expensive storage classes to colder, cheaper ones as access frequency drops, using lifecycle rules. It cuts your recurring storage bill without deleting a single byte. Standard sits at the top of the ladder, Glacier Deep Archive at the bottom.

Class	Min storage duration	Retrieval charge	First-byte latency	Notes
S3 Standard	None	None	Milliseconds	General-purpose hot data
S3 Intelligent-Tiering	None	None (no retrieval fee)	Milliseconds	Auto-tiers; saves up to 40% / 68% / 95%
S3 Standard-IA	30 days	Per GB	Milliseconds	128 KB minimum object size
S3 Glacier Instant Retrieval	90 days	Per GB	Milliseconds	Archive with instant access
S3 Glacier Flexible Retrieval	90 days	Per GB (restore)	Minutes to hours	Formerly S3 Glacier
S3 Glacier Deep Archive	180 days	Per GB (restore)	Hours	Cheapest class on the ladder

What is S3 storage tiering?

S3 storage tiering is the practice of placing each object in the cheapest storage class that still meets its access-latency needs, then moving it downward as it cools. AWS S3 exposes six classes on a single ladder: Standard, Intelligent-Tiering, Standard-IA, Glacier Instant Retrieval, Glacier Flexible Retrieval, and Glacier Deep Archive. You do not pick the class by hand for every object. Instead, you set lifecycle rules that transition objects after a given number of days of age or inactivity. The win is recurring: Standard carries the highest per-GB rate on the ladder, while Glacier Deep Archive is the lowest. Tiering captures that spread automatically. The catch is that colder classes add a per-GB retrieval charge and a minimum storage duration (30, 90, or 180 days), so the move only pays off for data you rarely read. Done right, it is one of the most effective recurring cost controls on an S3 bill. It changes nothing about your application, because the object key and API stay the same.

Why does cold data quietly drain your budget?

Most S3 bills are dominated by storage, not requests. A dataset that lands hot on launch day is often untouched three months later, yet it keeps sitting in Standard, accruing the highest per-GB rate on the ladder every single day. Teams rarely delete it (it might be needed) and rarely move it (the move is not automatic). That gap is avoidable waste. The structural fix is a lifecycle rule, not a cleanup sprint. Consider a 1 TB log archive read once a quarter: in Standard it costs the full hot rate 90 days out of 90; in a Glacier class the per-GB storage is substantially lower and you pay a per-GB retrieval only on the one day you actually read it. S3 Intelligent-Tiering quantifies the upside publicly: its Infrequent Access tier saves up to 40% on storage versus Standard, the Archive Instant Access tier up to 68%, and the opt-in Deep Archive Access tier up to 95%. Those are AWS-published figures, not marketing ranges.

How do lifecycle rules push objects down the ladder?

A lifecycle rule is just XML attached to a bucket. The Transition action names a storage class and a day count; S3 moves objects automatically once they cross that age. Here is AWS's own example that tiers the logs/ prefix over a lifetime (verbatim from the S3 User Guide):

<LifecycleConfiguration>
  <Rule>
    <ID>example-id</ID>
    <Filter><Prefix>logs/</Prefix></Filter>
    <Status>Enabled</Status>
    <Transition><Days>30</Days><StorageClass>STANDARD_IA</StorageClass></Transition>
    <Transition><Days>90</Days><StorageClass>GLACIER</StorageClass></Transition>
    <Expiration><Days>365</Days></Expiration>
  </Rule>
</LifecycleConfiguration>

Apply it with the AWS CLI (verbatim command form from the AWS CLI reference):

aws s3api put-bucket-lifecycle-configuration --bucket <bucket> --lifecycle-configuration file://lifecycle.json

Because RustFS is S3-compatible and lists Lifecycle Management (ILM) as Generally Available, the same S3 API call works against a RustFS endpoint. One rule, three actions: tier to IA at day 30, to Glacier at day 90, expire at day 365.

S3 Standard vs Standard-IA: when does the first step pay off?

Standard-IA trades a lower per-GB storage rate for two costs: a per-GB retrieval fee on every GET, and a 30-day minimum storage duration. The break-even is purely about read frequency. If you read an object more than about once a month, the retrieval fees erase the storage savings; keep it in Standard. If you read it quarterly or less, IA wins. AWS enforces a 128 KB minimum billable object size for IA, so transitioning a bucket of tiny files can cost more in overhead than it saves. Filter those out with <ObjectSizeGreaterThan>. The transition itself is billed as a request, so tiering millions of tiny objects has its own price. Practical rule: IA is the right first rung for objects larger than 128 KB that you access roughly once a quarter or less. Everything hotter stays in Standard; everything colder keeps falling.

Glacier tiers: at what point do milliseconds stop mattering?

Glacier is three rungs, and the only difference is retrieval time and minimum duration. Glacier Instant Retrieval gives you millisecond access at a per-GB retrieval fee and a 90-day minimum. Use it for archives you might need right now. Glacier Flexible Retrieval (the former S3 Glacier) takes minutes to hours to restore and also carries a 90-day minimum; it is the cheapest of the three for true cold storage you read a few times a year. Glacier Deep Archive is the bottom: hours-long retrieval and a 180-day minimum, the lowest per-GB rate on the entire ladder.

Tier	Minimum storage duration	Retrieval time
Glacier Instant Retrieval	90 days	Milliseconds
Glacier Flexible Retrieval	90 days	Minutes to hours
Glacier Deep Archive	180 days	Hours

The trap is the minimums. Delete or re-tier an object before its window and AWS charges you for the remaining days as a prorated fee. So Glacier only pays off for data with a real retention horizon: backups and compliance archives.

Intelligent-Tiering: automatic savings without the ops burden

S3 Intelligent-Tiering is the only class that moves data for you. You opt in, pay a small monthly monitoring and automation charge per 1,000 objects, and AWS watches access patterns. Objects not touched for 30 days drop to the Infrequent Access tier (up to 40% storage savings), and after 90 days of silence to Archive Instant Access (up to 68%). There are no retrieval charges and no minimum storage duration, which removes the biggest risk of manual tiering: getting the break-even wrong and paying penalties. Objects smaller than 128 KB are not auto-tiered; they stay at the Frequent Access rate and incur no monitoring fee. The trade-off is the monitoring charge itself. At very high object counts it can exceed what you would save, so it suits data lakes and user-generated content where access is genuinely unpredictable. For known-access patterns, explicit lifecycle rules are cheaper.

The traps that turn tiering into a loss

Tiering is not free, and four costs bite. Minimum storage durations apply: Standard-IA holds objects for 30 days, Glacier Instant and Flexible for 90, Deep Archive for 180. Delete or transition out early and AWS bills the remaining days as a prorated charge. The 128 KB minimum billable size on IA and Glacier classes means tiny objects cost more to tier than to leave alone, so filter them with <ObjectSizeGreaterThan>. Every transition is billed as a request, so moving millions of objects spikes a one-time request cost. And lifecycle only goes down the ladder: to bring an object back to Standard you must copy it explicitly with aws s3 cp or aws s3 sync, never a transition. Miss any of these and the savings go negative.

Self-hosting tiering with RustFS

If you run storage yourself, the same playbook applies. RustFS is Apache-2.0, S3-compatible object storage and lists Lifecycle Management (ILM) plus ILM Tiering (Remote S3) as Generally Available in its current Feature & Status table. A self-hosted RustFS cluster can apply S3 lifecycle rules that push cold objects to a remote S3 tier (another RustFS pool or an external S3 endpoint) once they cool, instead of holding everything on expensive local disk. Be clear on the model: RustFS does not replicate AWS's six in-place local storage classes. Its tiering is ILM-driven movement to a remote tier, not a drop-in Glacier class switch. Spin up a node to try it:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default credentials are rustfsadmin/rustfsadmin; set real keys before production.

Can I move objects back to S3 Standard with a lifecycle rule?

No. Lifecycle transitions only go down the ladder. To bring an object back to Standard you must copy it explicitly with aws s3 cp or aws s3 sync, never a transition action.

What is the cheapest S3 storage class?

S3 Glacier Deep Archive: the lowest per-GB storage rate on the ladder, with a 180-day minimum storage duration and hours-long retrieval. AWS publishes the exact per-GB rates on aws.amazon.com/s3/pricing.

Does S3 Intelligent-Tiering have retrieval fees?

No retrieval charges and no minimum storage duration. You pay a small monthly monitoring and automation fee per 1,000 objects, plus standard storage. Objects under 128 KB are not auto-tiered and incur no monitoring fee.

What happens if I delete an object before the minimum storage duration?

You are charged a prorated amount equal to the storage cost for the remaining days. Standard-IA enforces 30 days, Glacier Instant and Flexible enforce 90, and Deep Archive enforces 180. Match your rules to real retention horizons.

Can RustFS do S3 storage tiering?

Yes. RustFS lists Lifecycle Management (ILM) and ILM Tiering (Remote S3) as Generally Available. It moves cold objects to a remote S3 tier through S3-compatible lifecycle rules. It does not implement AWS's local Glacier class ladder in place; tiering is movement to a remote tier.

Sources

AWS S3 storage classes: https://aws.amazon.com/s3/storage-classes/ (checked 2026-09-28)
AWS S3 Intelligent-Tiering: https://aws.amazon.com/s3/storage-classes/intelligent-tiering/ (checked 2026-09-28)
AWS S3 lifecycle configuration examples: https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html (checked 2026-09-28)
AWS S3 pricing (per-GB list rates): https://aws.amazon.com/s3/pricing/ (checked 2026-09-28; rendered dynamically, verify current rates there)
AWS CLI put-bucket-lifecycle-configuration: https://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-lifecycle-configuration.html (checked 2026-09-28)
RustFS README Feature & Status: https://github.com/rustfs/rustfs (checked 2026-09-28)

RustFS is Apache-2.0, S3-compatible object storage. Run it yourself: github.com/rustfs/rustfs

A 5-Minute RustFS Stack with Docker Compose

Ethan Carter — Sun, 27 Sep 2026 13:52:00 +0000

A 5-Minute RustFS Stack with Docker Compose

RustFS is an Apache 2.0, S3-compatible object store written in Rust that you can run as a single Docker Compose stack and reach in about five minutes. This guide walks through the official docker-compose-simple.yml. One command brings up a RustFS node on ports 9000 (S3 API) and 9001 (web console) with default credentials rustfsadmin / rustfsadmin. You get a MinIO-compatible S3 endpoint on your own hardware, with versioning, Object Lock, IAM, and server-side encryption included, and no cloud account required. One caveat before we start: the simple file is a single node writing across four local volumes, and S3 Tables (Iceberg) is still in preview.

Key stats

	RustFS (docker-compose-simple)
License	Apache 2.0
Image	`rustfs/rustfs:latest`
S3 API / console ports	9000 / 9001
Default credentials	`rustfsadmin` / `rustfsadmin` (change before exposure)
Storage layout	4 local volumes: `/data/rustfs{0...3}`
Preview features	S3 Tables (Iceberg REST), MinIO On-Disk Compatibility

All rows are from the RustFS README and the docker-compose-simple.yml in the repo root (links in Sources). No benchmarks invented.

What is a RustFS Docker Compose stack?

A RustFS Docker Compose stack is a single declarative file that launches the RustFS server as a container plus a tiny helper that fixes volume ownership, so you get a working S3-compatible endpoint without hand-writing docker run flags. It is the easiest path to self-hosting. Instead of remembering -p 9000:9000 -p 9001:9001 -v ..., you describe the desired state once and let Compose reconcile it. The official repo ships two files. docker-compose-simple.yml runs a bare node. docker-compose.yml adds Prometheus, Grafana, Tempo, Jaeger, and an optional Nginx. For a five-minute start you want the simple one. It is Apache 2.0 end to end, and the only thing you install is Docker itself.

Why Docker Compose instead of a bare docker run?

A bare docker run works, but it is one long command you have to retype and remember. The simple Compose file captures the same setup as a reproducible artifact. Ports, volumes, environment, restart policy, and a health check all live in one place you can commit to git. The README's own quick start offers both paths. The fast path is docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest. The maintained path is docker compose -f docker-compose-simple.yml up -d. Compose also handles the ownership gotcha for you. RustFS runs as UID/GID 10001:10001, and the volume-permission-helper service chowns the named volumes before the server starts. With a raw docker run and bind mounts, that permission step is on you.

What does docker-compose-simple.yml actually do?

The file defines two services. The rustfs service pulls rustfs/rustfs:latest, maps ports 9000 and 9001, and sets the environment that turns a generic image into your node. RUSTFS_VOLUMES=/data/rustfs{0...3} lays out four storage volumes. RUSTFS_ADDRESS=0.0.0.0:9000 and RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001 bind the API and console. RUSTFS_CONSOLE_ENABLE=true switches the web UI on. Credentials come from RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY, both set to rustfsadmin in the file and marked # CHANGEME. A healthcheck hits /health on 9000 and /rustfs/console/health on 9001 every 30 seconds. The second service, volume-permission-helper, is an Alpine container that chowns the volumes to 10001:10001 and exits. That is the whole stack.

Here is the file, verbatim from the repo root:

# Copyright 2024 RustFS Team
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

services:
  # RustFS main service
  rustfs:
    image: rustfs/rustfs:latest
    container_name: rustfs-server
    depends_on:
      - volume-permission-helper
    security_opt:
      - "no-new-privileges:true"
    ports:
      - "9000:9000" # S3 API port
      - "9001:9001" # Console port
    environment:
      - RUSTFS_VOLUMES=/data/rustfs{0...3}
      - RUSTFS_ADDRESS=0.0.0.0:9000
      - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
      - RUSTFS_CONSOLE_ENABLE=true
      - RUSTFS_CONSOLE_CORS_ALLOWED_ORIGINS=*
      - RUSTFS_ACCESS_KEY=rustfsadmin # CHANGEME
      - RUSTFS_SECRET_KEY=rustfsadmin # CHANGEME
      - RUSTFS_OBS_LOGGER_LEVEL=info
      - RUSTFS_OBS_LOG_DIRECTORY=/app/logs
      # Optional TLS:
      # - mount your cert directory to /opt/tls
      # - set RUSTFS_TLS_PATH=/opt/tls
      # - when TLS is enabled, health checks automatically switch to HTTPS
      # Keep strict disk topology checks enabled by default.
      # For local testing only, set `RUSTFS_UNSAFE_BYPASS_DISK_CHECK=true` explicitly.
      - RUSTFS_UNSAFE_BYPASS_DISK_CHECK=${RUSTFS_UNSAFE_BYPASS_DISK_CHECK:-false}

    volumes:
      - rustfs_data_0:/data/rustfs0
      - rustfs_data_1:/data/rustfs1
      - rustfs_data_2:/data/rustfs2
      - rustfs_data_3:/data/rustfs3
      - logs:/app/logs
    networks:
      - rustfs-network
    restart: unless-stopped
    healthcheck:
      # Default behavior:
      # - HTTP when RUSTFS_TLS_PATH is unset
      # - HTTPS when RUSTFS_TLS_PATH is set
      # - loopback TLS uses `-k` unless strict host+CA values are provided
      #
      # Strict TLS example:
      # - set RUSTFS_HEALTHCHECK_HOST to a SAN-covered hostname
      # - optionally set RUSTFS_HEALTHCHECK_CA (defaults to /opt/tls/ca.crt when present)
      test:
        [
          "CMD",
          "sh", "-ec",
          "host=\"$${RUSTFS_HEALTHCHECK_HOST:-127.0.0.1}\"; scheme=\"http\"; set -- -fsS; \
          if [ -n \"$${RUSTFS_TLS_PATH:-}\" ]; then \
            scheme=\"https\"; \
            ca_path=\"$${RUSTFS_HEALTHCHECK_CA:-}\"; \
            if [ -z \"$${ca_path}\" ] && [ -f /opt/tls/ca.crt ]; then ca_path=/opt/tls/ca.crt; fi; \
            case \"$${host}\" in 127.0.0.1|localhost) strict_host=false ;; *) strict_host=true ;; esac; \
            if [ \"$${strict_host}\" = true ] && [ -n \"$${ca_path}\" ]; then \
              set -- \"$${@}\" --cacert \"$${ca_path}\" --resolve \"$${host}:9000:127.0.0.1\" --resolve \"$${host}:9001:127.0.0.1\"; \
            else \
              set -- \"$${@}\" -k; \
            fi; \
          fi; \
          curl \"$${@}\" \"$${scheme}://$${host}:9000/health\" && \
          curl \"$${@}\" \"$${scheme}://$${host}:9001/rustfs/console/health\""
        ]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  # RustFS volume permissions fixer service
  volume-permission-helper:
    image: alpine
    volumes:
      - rustfs_data_0:/data0
      - rustfs_data_1:/data1
      - rustfs_data_2:/data2
      - rustfs_data_3:/data3
      - logs:/logs
    command: >
      sh -c "
        chown -R 10001:10001 /data0 /data1 /data2 /data3 /logs &&
        echo 'Volume Permissions fixed' &&
        exit 0
      "
    # Permission baseline:
    # - default RustFS runtime user is 10001:10001
    # - alternatively, run rustfs service with host-matched `user: \"<uid>:<gid>\"`
    restart: "no"

networks:
  rustfs-network:

volumes:
  rustfs_data_0:
  rustfs_data_1:
  rustfs_data_2:
  rustfs_data_3:
  logs:

How do you start the stack in under five minutes?

Three steps. First, install Docker Engine and the Compose plugin, or Podman plus podman compose. Second, fetch the official file with curl -O https://raw.githubusercontent.com/rustfs/rustfs/main/docker-compose-simple.yml, or copy it from the repo root. Third, launch it: docker compose -f docker-compose-simple.yml up -d. Compose pulls the image, starts the helper, then the server. The health check flips to healthy after roughly 40 seconds, the start_period. Verify the containers are up with docker compose -f docker-compose-simple.yml ps. You now have an S3 endpoint on localhost:9000 and a console on localhost:9001. If you prefer Podman, the README notes podman compose -f docker-compose-simple.yml up -d is a drop-in replacement.

How do you verify the S3 endpoint is alive?

The health check already proves the server answers, but a real S3 call is the better proof. Hit the API health endpoint directly: curl -fsS http://localhost:9000/health should return 200. For an actual object operation, point any S3 CLI at it. mc (MinIO Client) speaks RustFS's S3 dialect. Run mc alias set rustfs http://localhost:9000 rustfsadmin rustfsadmin, then mc mb rustfs/demo, then mc ls rustfs. A bucket listing means the endpoint, credentials, and console path all work. Open http://localhost:9001 in a browser and log in with rustfsadmin / rustfsadmin to see buckets, users, and IAM policies in the UI. Five minutes, done.

What should you change before exposing it to the network?

Two non-negotiables. First, change the credentials. The file ships rustfsadmin / rustfsadmin, which the inline # CHANGEME comment flags as public knowledge. Set RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY to real secrets, or pass them via your env file, before any port faces the internet. Second, decide on TLS. The file shows the opt-in path: mount certs to /opt/tls and set RUSTFS_TLS_PATH=/opt/tls. The health check switches to HTTPS automatically when TLS is on. Leave RUSTFS_UNSAFE_BYPASS_DISK_CHECK at its default false in real deployments. Setting it true is for local throwaway testing only. Bind to localhost behind a reverse proxy unless you have a reason not to.

Where does RustFS fit next to MinIO and the rest?

RustFS positions itself as a MinIO-compatible, Apache 2.0 S3 store built in Rust, so the migration story is the draw. Your existing mc scripts, SDKs, and bucket policies mostly keep working. The README Feature & Status table marks S3 Core, Versioning, Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode as available. The same surface MinIO users expect. The gap to name plainly: RustFS is a younger project, and two features are still preview, S3 Tables (Iceberg REST) and MinIO On-Disk Compatibility. If you need Iceberg tables in production today, that row is not your answer yet. For general S3 put/get, versioning, immutability, and IAM, the simple Compose stack is a credible replacement for MinIO.

What is still in preview, and what is ready?

Ready in the simple stack: single-node S3 on four local volumes, versioning, Object Lock, IAM, SSE, bucket replication, and the web console. All marked available in the README. In preview: S3 Tables (Iceberg REST) and MinIO On-Disk Compatibility, both shipped behind an opt-in flag or a bounded compatibility claim. There is no POSIX/FUSE mount either, so do not expect to mount RustFS like a filesystem. The simple file is one node. Distributed Mode exists and is marked available, but standing up erasure coding across hosts is a separate, larger topology, not the five-minute job. Read the Feature & Status table before you assume a capability.

FAQ

Can I run the RustFS Compose stack without Docker?

Yes. The README documents a drop-in Podman path: podman compose -f docker-compose-simple.yml up -d. Podman Compose reads the same file, so the stack is identical. You still need a container runtime; there is no native binary-only Compose equivalent.

Do I have to use the four-volume layout?

The simple file sets RUSTFS_VOLUMES=/data/rustfs{0...3}, four named volumes on the host. You can change the count or point at bind mounts, but keep every path writable by UID/GID 10001:10001 or run the rustfs service with a host-matched user:. The volume-permission-helper only fixes the named volumes it mounts.

Why does the console show rustfsadmin / rustfsadmin?

Those are the built-in defaults, carried in the Compose file as RUSTFS_ACCESS_KEY / RUSTFS_SECRET_KEY with a # CHANGEME note. They are public and well-known, so change them before exposing the listener. The README confirms the same default credentials.

Is RustFS free to self-host?

The server is Apache 2.0, so self-hosting, modifying, and embedding it carries no license fee and no network copyleft. You pay for the hardware and bandwidth, not the software. The observability add-ons in docker-compose.yml (Prometheus, Grafana, and so on) are their own licenses.

How do I upgrade the stack?

Change the image tag, rustfs/rustfs:latest or pin rustfs/rustfs:1.0.1, then docker compose -f docker-compose-simple.yml up -d again. Compose recreates the container; the named volumes persist your data across the recreate.

Sources

RustFS README: docker run command, default credentials, Feature & Status table, Compose commands: https://github.com/rustfs/rustfs
docker-compose-simple.yml (verbatim, repo root): https://raw.githubusercontent.com/rustfs/rustfs/main/docker-compose-simple.yml
docker-compose.yml (full observability stack): https://raw.githubusercontent.com/rustfs/rustfs/main/docker-compose.yml

S3 Presigned URLs: Security Pitfalls You'll Regret

Ethan Carter — Sat, 26 Sep 2026 13:24:00 +0000

S3 Presigned URLs: Security Pitfalls You'll Regret

A presigned URL is a time-limited, signed link that grants temporary S3 access without sharing your credentials. The regrets come from treating it like a password: over-long expirations, plaintext HTTP, no network lock, and signing with permanent keys. The hardening below is about the URL model itself, so it applies to Amazon S3 and to any S3-compatible endpoint that speaks SigV4.

Key stats

Limit / control	Value (Amazon S3)	Why it matters
CLI / SDK max expiration	604,800 s (7 days)	`--expires-in` caps here; the console caps at 12 hours
Default expiration	3,600 s (1 hour)	Set it explicitly; never leave the default for sensitive data
Temporary-credential URLs	Expire with the session	STS AssumeRole defaults to 1 hour; EC2 profile ~6 hours
Enforce HTTPS	`aws:SecureTransport = false` → Deny	Blocks plaintext HTTP requests
Max signature age	`s3:signatureAge > 600000` ms → Deny	Kills URLs older than 10 minutes

Every number above is from AWS's own presigned-URL and security-best-practices docs (links in Sources). We cite the docs, not blog lore.

What is an S3 presigned URL, really?

A presigned URL is a signed Amazon S3 link that wraps your security credentials and grants one specific operation — usually a GET to download or a PUT to upload — for a fixed window. Anyone holding the link can perform that operation, with no AWS account of their own. The signature is computed with your credentials under Signature Version 4 (SigV4), and AWS now uses SigV4 for every presigned URL, so the region must be set explicitly. You generate one with the AWS CLI:

aws s3 presign s3://amzn-s3-demo-bucket/mydoc.txt --expires-in 604800

That command returns a URL carrying X-Amz-Algorithm, X-Amz-Credential, X-Amz-Expires, and X-Amz-Signature. The operation is limited to the permissions of whoever signed it. In other words, a presigned URL is a short-lived, single-purpose capability delegation — not an account, and not a password you can rotate after the fact.

How long can a presigned URL stay valid?

The ceiling depends on how you create it. From the AWS CLI or SDK, --expires-in accepts up to 604,800 seconds — exactly seven days — and defaults to 3,600 seconds (one hour) if you omit it. The S3 console is stricter: it lets you set between 1 minute and 12 hours, with no path to a 7-day link through the UI. For programmatic generation, 7 days is the hard stop for URLs signed with long-term IAM user credentials under SigV4.

A longer window is rarely the right answer. A presigned URL is a bearer token: the moment it leaks, whoever holds it has the access it grants until it expires. If you set 7 days "just in case," you have given a stolen link a week of life. Favor minutes for download links and single-use uploads. The only reason to push toward the maximum is a genuinely long, unattended transfer where the recipient polls over days — and even then, prefer a refresh loop over one giant timeout.

Why do presigned URLs expire earlier than you set?

Because the URL is only as alive as the credential that signed it. AWS states this plainly: a presigned URL expires when its underlying credential is revoked, deleted, or deactivated, even if you asked for a later expiration. This bites hardest with temporary credentials. A URL signed by an IAM role session dies when that session ends, regardless of the --expires-in you specified. The same is true for STS AssumeRole (default session of 1 hour) and EC2 instance-profile credentials (which rotate with a maximum validity of about 6 hours).

The practical consequence: if you generate a "7-day" URL inside a Lambda using a role, it may stop working in under an hour. For long-lived links, sign with a dedicated IAM user whose key you can rotate, or build a service that re-issues the URL on demand. Treat the configured expiration as an upper bound, never a guarantee, and log which principal signed each URL so you can revoke the source fast.

The biggest pitfall: presigned URLs are bearer tokens

AWS calls them out directly: presigned URLs are bearer tokens that grant access to whoever possesses them, so they must be protected appropriately. The trap is forgetting that. Teams paste them into Slack, commit them to repos, log them with the request, or stuff them into client-side JavaScript where anyone can read them. Once a URL is out, you cannot rotate it like a password — you can only wait for it to expire or revoke the signing credential entirely, which kills every URL that credential ever made.

Three habits cut most of the risk. Never log the full URL; log a hash or just the object key. Generate the URL at request time on the server, not once at build time. Scope it to the smallest action and shortest time the caller needs. If a URL must travel to a browser, hand it over through a short-lived endpoint and expire it aggressively. The bearer-token model means the link itself is the secret — handle it like one.

How do you keep presigned URLs short-lived by default?

You can enforce a maximum signature age at the bucket level, so even a URL generated with a long --expires-in gets rejected once its signature is too old. AWS documents this with the s3:signatureAge condition key. The example below denies any presigned request on a bucket if the signature is more than 10 minutes old (600,000 milliseconds):

{
    "Version":"2012-10-17",
    "Statement": [{
        "Sid": "Deny a presigned URL request if the signature is more than 10 min old",
        "Effect": "Deny",
        "Principal": {"AWS": "*"},
        "Action": "s3:*",
        "Resource": "arn:aws:s3:::amzn-s3-demo-bucket/*",
        "Condition": {"NumericGreaterThan": {"s3:signatureAge": "600000"}}
    }]
}

This is defense in depth: the URL's own expiration might be an hour, but the bucket refuses anything signed more than 10 minutes ago. Tune the millisecond value to your workflow. Pair it with short expirations at generation time, and a leaked link becomes useless fast even if someone sits on it. Note this works for SigV4-signed requests and complements — not replaces — a small --expires-in.

How do you force HTTPS on every presigned request?

Presigned URLs carry credentials in the query string, so a plaintext HTTP handoff is a credential leak waiting to happen. Amazon S3's security guidance is to allow only encrypted connections and deny the rest with the aws:SecureTransport condition. The documented example bucket policy denies every request that is not TLS:

{
    "Version":"2012-10-17",
    "Statement": [{
        "Sid": "RestrictToTLSRequestsOnly",
        "Action": "s3:*",
        "Effect": "Deny",
        "Resource": [
            "arn:aws:s3:::amzn-s3-demo-bucket",
            "arn:aws:s3:::amzn-s3-demo-bucket/*"
        ],
        "Condition": {"Bool": {"aws:SecureTransport": "false"}},
        "Principal": "*"
    }]
}

With that in place, a presigned URL opened over HTTP returns Access Denied instead of serving the object. This applies to all requests to the bucket, presigned or not. One caveat: aws:SecureTransport is an AWS condition key; if you run an S3-compatible endpoint such as RustFS behind your own proxy, enforce TLS at the proxy or load balancer instead. The goal is the same — never let a signed URL traverse the wire in clear text.

How do you lock presigned URLs to a network?

By default a presigned URL works from anywhere on earth. If your recipients are internal, that is more reach than you want. AWS lets you constrain the network path with IAM condition keys. For requests hitting the public S3 endpoint, use aws:SourceIp; for traffic through a VPC endpoint, use aws:SourceVpc or aws:SourceVpce. The documented IAM statement denies all access unless it originates from a specific IP range:

{
    "Sid": "NetworkRestrictionForIAMPrincipal",
    "Effect": "Deny",
    "Action": "*",
    "Resource": "*",
    "Condition": {
        "NotIpAddressIfExists": {"aws:SourceIp": "IP-address-range"},
        "BoolIfExists": {"aws:ViaAWSService": "false"}
    }
}

This restricts the credential itself, so any URL it signs is also bound to that network. Apply it at the IAM principal, the bucket policy, or both. For a VPC-only architecture, the aws:SourceVpc / aws:SourceVpce pair keeps URLs from ever leaving your private path. Combine with the short signature-age and HTTPS rules above and a leaked link is both geofenced and short-lived.

Which credential should sign the URL?

The signing credential decides how much damage a leak causes and how fast you can contain it. Long-term IAM user access keys are the worst choice for high-churn URL generation: they do not rotate, and revoking one means editing every system that depends on it. AWS's own best-practice guidance is to use IAM roles for applications and services rather than embedding long-term credentials, precisely because roles hand out temporary, scoped permissions.

For presigned URLs, that means generating them inside a role session — a Lambda execution role, an EC2 instance profile, or an STS AssumeRole call — and accepting that the URL lives only as long as that session. If you genuinely need a multi-day link, isolate a single IAM user with a tight policy (only s3:GetObject on the specific prefix) and rotate its key on a schedule. Never sign URLs with an admin or root credential. The principle: the credential behind the URL should be the easiest thing in the chain to revoke.

RustFS and other S3-compatible endpoints: same rules, your perimeter

The hardening above is about the URL model, not about Amazon S3 specifically. Any S3-compatible endpoint that speaks SigV4 — RustFS among them — accepts the same presigned URLs your AWS SDK produces, and the same bearer-token risks apply. The difference is the perimeter: AWS gives you aws:SecureTransport, aws:SourceIp, and s3:signatureAge as managed condition keys; a self-hosted store puts that enforcement on you, at the proxy, the network policy, or the bucket policy if the server implements it.

RustFS ships under Apache 2.0 with S3 Core, Versioning, Object Lock, Server-Side Encryption, and IAM/Policies marked available, and it starts with one command:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

Default credentials are rustfsadmin / rustfsadmin — rotate them before exposure. If you run your own S3 surface, you own the HTTPS enforcement and network lock the cloud gives you for free, so bake them into the ingress. The security pitfalls are identical; only the place you close them changes.

FAQ

Are presigned URLs safe to send over email?

Yes, if they are short-lived and HTTPS-only. A presigned URL in email is a bearer token, so keep --expires-in to minutes for downloads and never send one that grants PUT to a writable prefix. Enforce aws:SecureTransport so the link fails on plaintext HTTP.

Can a presigned URL be revoked before it expires?

Not the URL itself. You revoke the signing credential — delete or deactivate the IAM user key, or end the role session — and every URL it created stops working immediately, even with time left. There is no per-URL kill switch.

Why is my presigned URL returning AccessDenied?

Most often the signing principal lacks the underlying permission (for example s3:GetObject), the bucket policy denies it, the region is wrong, or the request headers (such as Content-Type on a PUT) differ from generation. Clock drift also breaks SigV4 signatures.

What is the maximum expiration for a presigned URL?

Seven days (604,800 seconds) from the AWS CLI or SDK with IAM user credentials under SigV4. The console caps at 12 hours. URLs signed with temporary credentials expire with the session — STS AssumeRole defaults to 1 hour.

Do presigned URLs work with S3-compatible stores like RustFS?

Yes. Any endpoint that implements SigV4-signed S3 requests accepts the same presigned URLs. The security model is identical; you just supply the HTTPS and network restrictions yourself instead of relying on AWS condition keys.

Sources

AWS — Share an object with a presigned URL (CLI command, 604800 s max): https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html
AWS — aws s3 presign CLI reference (--expires-in default 3600, max 604800): https://docs.aws.amazon.com/cli/latest/reference/s3/presign.html
AWS — Download and upload with presigned URLs (expiration by credential type, bearer-token warning, s3:signatureAge, aws:SourceIp): https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-presigned-url.html
AWS — Upload objects with presigned URLs (curl PUT example, SDK patterns): https://docs.aws.amazon.com/AmazonS3/latest/userguide/PresignedUrlUploadObject.html
AWS — Example bucket policies (RestrictToTLSRequestsOnly / aws:SecureTransport): https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html
AWS — Security best practices for Amazon S3 (IAM roles, least privilege, HTTPS): https://docs.aws.amazon.com/AmazonS3/latest/userguide/security-best-practices.html
RustFS README (Apache 2.0, Feature & Status, docker run, default credentials): https://raw.githubusercontent.com/rustfs/rustfs/main/README.md

Re-checked 2026-09-23.

Garage vs RustFS: Running S3 on a Single Node

Ethan Carter — Tue, 22 Sep 2026 23:59:57 +0000

Garage vs RustFS: Running S3 on a Single Node

Garage and RustFS are both Rust-written, single-binary S3-compatible stores you can run on one node. The decisive split is the license and the S3 feature set: Garage is AGPL-3.0 and skips versioning, Object Lock, and server-side encryption; RustFS is Apache 2.0 and ships all of them. Run Garage when you want a tiny, geo-distributed store on secondhand hardware and AGPL is fine; run RustFS when your app needs versioning, immutability, IAM, and a MinIO-compatible dialect on the simplest possible operational surface.

Key stats

	Garage	RustFS
License	AGPL-3.0 (network copyleft)	Apache 2.0 (permissive)
Implementation	Rust	Rust
Single-node start	`garage server --single-node`	one `docker run`
S3 object versioning	❌ Missing	✅ Available
Object Lock (WORM)	❌ Missing	✅ Available
Server-side encryption	❌ Missing	✅ Available
Bucket / site replication (S3 API)	❌ Missing	✅ Available
ACL / bucket policies	❌ (own permission model)	✅ (IAM / Policies)
Default S3 API port	3900	9000
Documented minimum RAM	1 GB	not published

Garage's "missing" rows are not bugs — they are its own S3-compatibility page stating the feature is not implemented. RustFS rows are its README Feature & Status table (re-checked 2026-09-21). We cite the docs, not benchmarks.

What is the real difference between Garage and RustFS?

Garage is an S3-compatible distributed object store designed for self-hosting at a small-to-medium scale, built by the French non-profit Deuxfleurs. Its own docs describe it as a service that "replicates data 3x over distant servers" and is "not as latency sensitive" because it avoids consensus algorithms like Paxos or Raft. You run one garage binary per node, point nodes at each other, and assign a cluster layout. The design center is geo-distributed clusters of cheap, mismatched machines across physical locations.

RustFS is narrower by design: a purpose-built, MinIO-compatible S3 object store written in Rust under the Apache 2.0 license. Its README Feature & Status table marks S3 Core, Versioning, Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode as available, with S3 Tables (Iceberg REST) and MinIO On-Disk Compatibility in preview. There is no POSIX layer and no FUSE mount. Both are single Rust binaries, but Garage optimizes for resilient distribution across locations; RustFS optimizes for S3 correctness and the smallest operational surface.

How do the licenses actually differ?

Garage's source code is released under the AGPL v3 license. The project states it plainly: "if you patch Garage and then use it to provide any service over a network, you must share your code." That is network copyleft — stronger than the ordinary GPL. If you modify Garage and expose it as a service, you must publish your modified source. For a homelab or an internal-only deployment that never serves third parties, AGPL is usually a non-event. The moment you ship a modified Garage as a product or a hosted service, the obligation attaches.

RustFS is released under the permissive Apache 2.0 license. The README notes it is "released under the permissible Apache 2.0 license," with no copyleft. You can run, modify, and embed it in a closed-source product or a commercial service without a source-disclosure obligation. For most companies evaluating storage, this is the practical differentiator: RustFS imposes no network-copyleft tax, while Garage's AGPL-3.0 does. If license friction is a concern, that single line settles the comparison before any feature table does.

What S3 features does Garage leave out?

Garage implements a deliberate subset of the S3 API, and its own compatibility page lists what is missing. Bucket versioning is "Missing." Object Lock — both legal hold and retention — is "Missing" across every lock endpoint. Server-side encryption (PutBucketEncryption and friends) is "Missing"; the project's stance is that you should encrypt at the partition or client side instead. Bucket replication via the S3 API is "Missing." ACLs and bucket policies are "Missing" — Garage has its own per-access-key, per-bucket permission model rather than AWS's ACL-or-policy system.

Lifecycle is only partially implemented: Garage supports AbortIncompleteMultipartUpload and Expiration and nothing that depends on versioning or storage classes. Multipart upload, presigned URLs, CORS, and static-website hosting are implemented. The honest read: Garage is a clean S3-compatible put/get store with replication built in at the cluster level, not a drop-in for apps that assume versioning, WORM immutability, SSE, or IAM policies. If your client needs those, test first or look at RustFS, which lists every one of them as available.

How does each one run on a single node?

Garage's documented single-node path is one command after you generate a config. The quick start generates a garage.toml (metadata_dir, data_dir, replication_factor = 1, ports 3901/3900/3902/3903) and then starts the server:

garage server --single-node --default-bucket

That flag auto-configures a single-node cluster with no replication and creates a default access key and bucket from environment variables. For Docker, the official quick start pins dxflrs/garage:v2.3.0 (newer stable releases exist) and maps the four ports:

docker run \
  -d \
  --name garage-container \
  -p 3900:3900 -p 3901:3901 -p 3902:3902 -p 3903:3903 \
  -v $(pwd)/garage.toml:/etc/garage.toml \
  -e GARAGE_DEFAULT_ACCESS_KEY \
  -e GARAGE_DEFAULT_SECRET_KEY \
  -e GARAGE_DEFAULT_BUCKET \
  dxflrs/garage:v2.3.0 \
  /garage server --single-node --default-bucket

RustFS starts with one docker run (verbatim from its README):

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

That serves the S3 API on 9000 and a console on 9001. Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any real exposure. RustFS also ships a single binary you can drop on a host.

When should you pick Garage over RustFS?

Pick Garage when the requirement is resilient storage across locations on hardware you already own, and you are comfortable with AGPL-3.0. Its sweet spot is exactly what Deuxfleurs built it for: a few nodes in two or three physical locations replicating data so it stays available through failures and high-latency links. The operational surface is small — one binary, a TOML file, a layout command — and the memory footprint is tiny (the project cites 1 GB RAM and 16 GB disk as minimums). For homelab backups, media, or app assets, self-hosting wins on price the moment the hardware is sunk cost.

Garage also wins if you specifically want its model: CRDT-based federation instead of Raft/Paxos, so it tolerates slow or flaky inter-node links that would stall a consensus-based cluster. If your app only needs put/get/list plus presigned URLs and a static-site endpoint, Garage is lighter than running a full S3 engine. The catch is the missing feature set above — if your app leans on versioning, Object Lock, or IAM, Garage is the wrong tool and you should not try to bolt those on.

Where does RustFS win?

RustFS wins on the axis most S3 teams actually care about: a complete, MinIO-compatible S3 endpoint with the least operational surface. One docker run, no cluster-layout choreography, a web console on 9001. The feature set is broad and marked available in the README — Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode are all shipped and covered by CI gates. If your tools already speak MinIO's S3 dialect, RustFS lists MinIO On-Disk Compatibility as a preview feature and targets that dialect as its primary surface.

The Apache 2.0 license is the other win: no network-copyleft obligation, so you can wrap RustFS in a product or a hosted service without source-disclosure strings. For a team whose only contract with storage is the S3 API and whose app assumes versioning, immutability, or IAM policies, RustFS is the closer fit and the shorter path to a working, compliant endpoint. RustFS is younger as an ecosystem, and S3 Tables (Iceberg) plus MinIO on-disk compatibility are still preview — own that if a lakehouse or byte-exact MinIO migration is the requirement.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of September 2026. Garage rows come from garagehq.deuxfleurs.fr (quick start and the S3-compatibility page); RustFS rows come from its README Feature & Status table (re-checked 2026-09-21). No benchmarks, no marketing.

Capability	Garage	RustFS
S3-compatible API	✅ (subset)	✅ (S3 Core)
Object versioning	❌ Missing	✅ Available
Object Lock (WORM)	❌ Missing	✅ Available
Server-side encryption	❌ Missing	✅ Available
Bucket / site replication (S3 API)	❌ Missing	✅ Available
Lifecycle (ILM)	⚠️ Partial (Abort MPU + Expiration)	✅ Available
ACL / bucket policies	❌ (own model)	✅ (IAM / Policies)
Web console / admin API	✅ (admin API)	✅ (Web Console)
Geo-distribution model	CRDT federation, no Raft/Paxos	Distributed Mode (✅ Available)
Single binary	✅ (`garage`)	✅
License	AGPL-3.0	Apache 2.0
Implementation	Rust	Rust

Two rows deserve a call-out. RustFS lists S3 Tables (Iceberg REST) as preview, so a mature lakehouse over RustFS data is not a promise the project makes yet. And Garage's missing versioning, Object Lock, SSE, and ACL rows are by design — its strength is resilient distribution, not S3-feature parity.

How do you choose between them?

Use this rule of thumb: if the ticket says "backups across sites on hardware we own, AGPL is fine," start with Garage. If the ticket says "S3 endpoint for an app that needs versioning, immutability, or IAM, and we don't want copyleft," start with RustFS.

A split is legitimate and common. Run Garage for the geo-distributed, secondhand-hardware layer where its CRDT model and tiny footprint pay off, and run RustFS in front of the S3-facing applications where versioning, Object Lock, and a MinIO-compatible dialect are the whole ask. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job — Garage dragged into an app that assumes Object Lock becomes a missing-feature ticket, and RustFS pushed into a no-copyleft-averse, flaky-link geo cluster carries obligations it was not built to carry.

FAQ

Is Garage a drop-in MinIO replacement?

No. Garage implements a subset of the S3 API — its own compatibility page lists versioning, Object Lock, replication, SSE, and ACL/policy as missing — so most S3 clients work for put/get/list but apps that assume MinIO's full feature set will break. RustFS targets the MinIO S3 dialect as its primary surface and lists MinIO On-Disk Compatibility as a preview feature. If a MinIO-compatible endpoint is the requirement, RustFS is the closer fit.

Does Garage support object versioning?

No. Garage's S3-compatibility page marks bucket versioning as "Missing" and notes it "does not (yet) support object versioning." GetBucketVersioning returns a stub that always reports versioning disabled. If your workload needs versioned objects or point-in-time recovery through S3, Garage cannot provide it today; RustFS lists Versioning as available.

Is RustFS Apache 2.0 like Garage is AGPL?

No. RustFS is released under the permissive Apache 2.0 license with no copyleft, while Garage is AGPL-3.0. The practical difference: if you modify Garage and expose it as a network service, you must publish your modified source; RustFS imposes no such obligation. For a hosted or embedded product, that single license line is often the deciding factor.

Can Garage run on a single node?

Yes. Garage's quick start runs a single node with garage server --single-node --default-bucket, which auto-configures a no-replication cluster and creates a default key and bucket. The docs warn this provides no data redundancy and "should not be used in production." Garage is designed for three nodes across zones; single-node is for evaluation and dev, not for durable storage.

Which one should I run for a homelab backup?

If your backup needs versioning, immutability (Object Lock), or IAM-scoped keys, run RustFS — it ships all three and is Apache 2.0. If you want geo-distributed copies across secondhand machines, accept AGPL-3.0, and only need put/get plus presigned URLs, Garage is lighter and built exactly for that. Match the missing-feature list to your client before committing.

Sources

Garage quick start (commands, ports, single-node): https://garagehq.deuxfleurs.fr/documentation/quick-start/
Garage S3 compatibility status (versioning, Object Lock, SSE, replication, ACL missing): https://garagehq.deuxfleurs.fr/documentation/reference-manual/s3-compatibility/
Garage license (AGPL v3) and design goals (1 GB RAM, 16 GB disk, geo-distribution): https://garagehq.deuxfleurs.fr/
Garage source repository and language (Rust, AGPL-3.0): https://git.deuxfleurs.fr/Deuxfleurs/garage
RustFS README (Feature & Status, docker run, default credentials, Apache 2.0): https://raw.githubusercontent.com/rustfs/rustfs/main/README.md

SeaweedFS vs RustFS: When a Tiny Footprint Wins

Ethan Carter — Sat, 19 Sep 2026 01:52:00 +0000

SeaweedFS and RustFS are both Apache 2.0, single-binary stores, but they solve different problems. SeaweedFS is a distributed file system that also speaks S3, with a 16-byte per-object index that lets one master track billions of files. RustFS is a purpose-built, MinIO-compatible S3 object store with a web console. Pick SeaweedFS when you need a POSIX/FUSE mount, an Iceberg lakehouse, or billions of small files; pick RustFS when S3 is your only interface and a drop-in MinIO replacement in one docker run is the goal.

Key stats

	SeaweedFS	RustFS
License	Apache 2.0	Apache 2.0
What one binary serves	S3 + POSIX file system + Iceberg lakehouse	S3-compatible object storage
In-memory index per object	16-byte entry per blob	not published
On-disk metadata per file	40 bytes	not published
1KB-file throughput (1M files, conc 16, single SSD)	15,708 write/s, 47,019 read/s	not published

The SeaweedFS throughput figure is its own README's unscientific single-machine number: 1 million 1KB files at concurrency 16 on a MacBook with an SSD. We cite it as a scale signal, not a hardware review.

What is the real difference between SeaweedFS and RustFS?

SeaweedFS is a simple and highly scalable distributed file system. Its own README says one weed binary "serves an S3 object store, a POSIX file system, and a lakehouse with S3 Tables, all over the same data." Under the hood, a master server tracks volumes (not individual files), volume servers store blobs in append-only volume files, and a filer exposes the data as a POSIX file system, an S3 endpoint, and an Iceberg catalog.

RustFS is narrower by design. It is S3-compatible object storage written in Rust and released under the Apache 2.0 license. Its README's Feature & Status table marks S3 Core, Versioning, Object Lock (WORM), Lifecycle Management, Bucket Replication, Site Replication, IAM/Policies, Server-Side Encryption, and Distributed Mode as available today, with S3 Tables (Iceberg REST) in preview. There is no POSIX layer and no FUSE mount. The difference is the whole story: SeaweedFS is a file system that also does S3; RustFS is an S3 engine that does one thing.

How small is the metadata footprint, really?

This is where "tiny footprint" stops being marketing. SeaweedFS packs small files into append-only volume files, so there is no per-file inode and no per-file metadata file. The volume server keeps a 16-byte index entry per blob in memory and reads it in a single seek, even for erasure-coded data; on disk it stores 40 bytes of metadata per file. Because the master tracks volumes rather than files, a cluster holding billions of files still has only a few thousand volumes, so the master stays small and never sits in the read path — clients cache the volume-to-server mapping and talk to volume servers directly.

RustFS is a single Rust binary or container. The documented start is one docker run; there is no monitor quorum, no placement groups, no daemon zoo to reason about. We do not publish a per-object index size, and the comparison is not apples-to-apples — RustFS optimizes for S3 semantics, not for billions of tiny files on a single file system. The footprint win is real on different axes: SeaweedFS wins raw metadata scale; RustFS wins operational simplicity for an S3-only team.

How does each one deploy?

SeaweedFS ships as a single weed binary. The fastest path is weed mini, which starts the master, a volume server, the filer, WebDAV, the Iceberg REST catalog, and the Admin UI in one process, and exposes the S3 endpoint at http://localhost:8333:

AWS_ACCESS_KEY_ID=admin \
AWS_SECRET_ACCESS_KEY=secret \
S3_BUCKET=my-bucket \
./weed mini -dir=./data

To grow capacity, point more volume servers at the master:

weed volume -dir=/data -master=<master_host>:9333

RustFS ships as a single binary or a container. The documented start command is:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

That one line serves the S3 API on port 9000 and a console on 9001. For bare metal there is a one-line installer:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any production exposure.

When should you pick SeaweedFS over RustFS?

Reach for SeaweedFS when the workload is not "objects" but "files at extreme scale." Its design target — store billions of files, serve them fast — shows up in the numbers: a 16-byte per-blob index, 40 bytes per file on disk, and a master that tracks volumes, not files. If you are building a content-addressable store, a photo or video library, or an ML feature store with tens of millions of tiny shards, that metadata model is the difference between a master that fits in RAM and one that does not.

SeaweedFS also wins the moment you need a file system or a lakehouse over the same bytes. One weed binary serves an S3 endpoint, a FUSE mount on Linux/macOS/Windows, WebDAV, SFTP, HDFS, and an Iceberg REST catalog, all addressing the same data. If a data scientist wants to ls the lakehouse while an app writes through S3, SeaweedFS does both without a sync job. RustFS does not offer a FUSE/POSIX mount, so that requirement alone steers you to SeaweedFS.

Where does RustFS win?

RustFS wins on the axis most S3 teams actually care about: a clean, MinIO-compatible S3 endpoint with the least operational surface. One docker run, no quorum, no volume-server choreography. The feature set is now broad — Object Lock (WORM), Lifecycle Management, Bucket and Site Replication, IAM/Policies, OIDC/SSO, Server-Side Encryption, and Distributed Mode are all marked available in the README, with MinIO On-Disk Compatibility in preview. If your tools already speak MinIO's S3 dialect, RustFS is a drop-in.

The Apache 2.0 license is the same as SeaweedFS's, so licensing is not the differentiator. The differentiator is focus: RustFS spends its complexity budget on S3 correctness and the web console, not on a file system layer. For a team whose only contract with storage is the S3 API, that focus means fewer knobs to misconfigure and a shorter path to a working endpoint.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of September 2026. SeaweedFS rows come from its GitHub README; RustFS rows come from its README Feature & Status table (2026-09-18). No benchmarks, no marketing.

Capability	SeaweedFS	RustFS
S3-compatible API	✅	✅ (S3 Core)
POSIX / FUSE file system	✅ (FUSE on Linux/macOS/Windows)	❌
Iceberg / lakehouse over same data	✅ (S3 Tables)	🧪 Preview (S3 Tables, Iceberg REST)
Erasure coding	✅ (warm data, background)	❌
Bucket / site replication	✅ (volume-level)	✅ (Bucket + Site Replication)
Web console / Admin UI	✅	✅ (Web Console)
Kubernetes	CSI driver + Operator	✅ (Helm Charts)
Single binary	✅ (weed)	✅
License	Apache 2.0	Apache 2.0
Implementation	Go	Rust

Two rows deserve a call-out. RustFS lists S3 Tables (Iceberg REST) as Preview, so if a mature lakehouse over the same bytes is the requirement, SeaweedFS is ahead today. And SeaweedFS's erasure coding runs on warm data in the background, so writes never pay the encoding cost up front — a model RustFS does not yet offer in its feature table.

What can RustFS not replace yet?

Be clear about the boundaries before you commit. RustFS's README has no FUSE/POSIX mount, so if you need a file system interface or a mounted lakehouse, SeaweedFS (or CephFS) is the answer today. RustFS's S3 Tables (Iceberg REST) is in preview, so a production lakehouse over RustFS data is not a promise the project makes yet. And while Distributed Mode is now marked available, a RustFS cluster at SeaweedFS-scale file counts is a different beast — SeaweedFS was built from the start to keep the master small at billions of files, and RustFS does not publish an equivalent per-object index claim.

None of this is a knock on RustFS. It is a scope statement: RustFS is an S3 engine, and it is good at being one. Pushing it into the file-system or exabyte-file-count role is the wrong tool for that job.

How do you choose between them?

Use this rule of thumb: if the words "file system," "FUSE," or "lakehouse" appear in the ticket, start with SeaweedFS. If only "S3" appears, and the real constraint is headcount, time, or MinIO compatibility, start with RustFS.

A split is legitimate and common. Run SeaweedFS for the data-lakehouse and file-serving layer where billions of files and a POSIX mount pay off, and run RustFS in front of the S3-facing applications where a MinIO-compatible endpoint and a web console are the whole ask. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job — SeaweedFS dragged into a pure-S3, small-team role carries a file system you will never mount, and RustFS pushed into a POSIX role it does not implement becomes a custom FUSE you have to build yourself.

FAQ

Is SeaweedFS a drop-in replacement for MinIO?

Not directly. SeaweedFS implements S3 as one of several interfaces on top of its distributed file system, so most S3 clients work, but it is not built to mirror MinIO's S3 dialect command for command. RustFS, by contrast, lists MinIO On-Disk Compatibility as a preview feature and targets the MinIO S3 dialect as its primary surface. If your only requirement is S3 and you want MinIO-compatible tooling, RustFS is the closer fit.

Does SeaweedFS support erasure coding?

Yes. SeaweedFS replicates hot data for speed and applies erasure coding to warm data in the background, so writes never pay the encoding cost up front. Volume servers replicate or erasure-code at the volume level, and the in-memory index stays 16 bytes per blob even for erasure-coded data. RustFS's feature table does not list erasure coding today.

Does RustFS have a FUSE mount?

No. RustFS's README Feature & Status table has no FUSE/POSIX mount, so there is no file system interface. If you need to mount storage as a local directory or expose a lakehouse over the same bytes, SeaweedFS provides a FUSE mount on Linux, macOS, and Windows. For pure S3 object access, the missing mount is irrelevant.

Which one scales to more files?

SeaweedFS is designed for it: the master tracks volumes rather than files, so a cluster with billions of files holds only a few thousand volumes and the master stays small. RustFS is a capable S3 object store, but it does not publish a comparable per-object index claim, and its strength is S3 semantics at operationally simple scale rather than billions of tiny files on one file system.

Are both projects Apache 2.0?

Yes. Both SeaweedFS and RustFS are released under the Apache License 2.0, a permissive license with no copyleft. If you run either as a service the practical difference is small; if you embed the engine in a shipped product, Apache 2.0 spares you the copyleft questions that a GPL/LGPL project would raise.

RustFS 1.0 Is GA After 2.5 Years: What's Actually Production-Ready, and What Isn't

Ethan Carter — Thu, 17 Sep 2026 14:08:01 +0000

RustFS 1.0.0 reached general availability on September 16, 2026. The core S3-compatible object storage engine is stable and supported for production. RustFS is a distributed, Apache 2.0-licensed object store written in Rust, now with 32,000+ GitHub stars and 2.7 million+ deployed instances worldwide.

Key stats

Metric	Value
GitHub stars	32,000+
Docker Hub pulls	10,000,000+
Instances deployed	2.7 million+
Contributors	160+
First commit to GA	Feb 2024 to Sep 2026 (2 yr 7 mo)
License	Apache 2.0

RustFS 1.0.0 GA: what actually shipped

RustFS 1.0.0 is the first generally available release of the project. "GA" here means one specific thing: the core object storage engine (PUT, GET, multipart upload, versioning, replication, and the S3 API surface) is stable and supported for production traffic. The team spent two years and seven months getting there, from the first commit in February 2024 to open sourcing in July 2025, then Beta in April 2026 and RC in August 2026.

What shipped is not a rewrite of someone else's storage engine. RustFS is built from scratch in Rust, which the team chose for memory safety and concurrency under heavy write loads. GA matters because it lets you stop evaluating RustFS and start running it where downtime costs money. The release also closes a promise the project made at open source: a self-hosted, S3-compatible store with no commercial license gate on the core engine.

What is RustFS, and why Rust?

RustFS is a distributed object storage system that speaks the S3 API. You point an existing S3 client (the AWS CLI, MinIO's mc, rclone, or your application's SDK) at a RustFS endpoint and it behaves like S3. Under the hood it is written in Rust, which is the part that actually matters to operators.

Rust brings two things to a storage engine. First, memory safety without a garbage collector, so there is no stop-the-world pause while you serve a hot bucket. Second, fearless concurrency: the borrow checker lets the team write parallel I/O paths that do not corrupt data under load. The practical result the project cites is steady throughput on PUT-heavy workloads. If you have run a Go-based store and fought GC tail latency, that is the class of problem RustFS targets. The trade-off is a smaller ecosystem and fewer prebuilt integrations than the incumbents.

Is RustFS production-ready today?

For the core object storage path, yes. RustFS 1.0.0 lists single-node and distributed deployment as available, along with versioning, object lock (WORM), bucket and site replication, durability through bitrot protection, and server-side encryption. Those are the features most teams actually turn on in production.

What I would not do on day one is assume RustFS matches MinIO or Ceph feature for feature. The RustFS feature matrix marks S3 Tables (Apache Iceberg) as preview, not GA, and the surrounding tooling (backup operators, third-party connectors, managed offerings) is thinner than the incumbents. My recommendation: run RustFS for a real workload you can measure, not just a hello-world bucket. Stand up a single node, point rclone at it, and watch p99 latency under your own traffic before you trust it with the primary copy of anything.

What can RustFS do that matters for storage teams?

RustFS covers the S3 surface most applications need: multipart upload for large objects, versioning, object lock for compliance hold, and S3 Select for server-side filtering. On access it is not S3-only. The GA release ships the Swift API, WebDAV, FTPS, and SFTP alongside the S3 endpoint, which matters if you are retiring an old Swift cluster or want file-style access without a separate gateway.

On the control plane, RustFS gives you IAM policies, OIDC single sign-on, a RustFS KMS, STS tokens, and audit logging. For operations there are event notifications, health inspection, multi-tenancy, and Helm charts for Kubernetes. Data movement is handled by bucket replication and site replication, plus ILM tiering to a remote S3 target. If your stack already speaks S3, the migration is mostly a DNS and credential change.

How do you run RustFS 1.0.0?

The fastest path is a single container. The official image is rustfs/rustfs:latest, and the README's verified command maps the API port 9000 and console port 9001, with volumes for data and logs:

docker run -d -p 9000:9000 -p 9001:9001 \
  -v $(pwd)/data:/data -v $(pwd)/logs:/logs \
  rustfs/rustfs:latest

[sourced from RustFS GitHub README, NOT EXECUTED IN CI]

Default credentials are rustfsadmin / rustfsadmin (change them before this touches a network). The web console is at http://localhost:9001. For a bare-metal box there is also a one-click script: curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh. Kubernetes users get Helm charts. For a quick distributed test, docker compose with the project's docker-compose-simple.yml brings up multiple nodes. Pick the method that matches where your workload already runs.

Where does RustFS still trail MinIO and Ceph?

Honesty first: RustFS is younger. Ceph has a decade of production deployments and MinIO has a far larger install base and connector ecosystem. On mixed and read-heavy workloads RustFS is still closing the gap. Our beta.10 benchmarks showed RustFS ahead on PUT-heavy patterns, but read-mixed tuning is where the incumbents remain ahead today.

S3 Tables (Iceberg) is in preview, so if your roadmap depends on in-storage analytics tables, that is not GA yet. The third gap is operational muscle memory: fewer community runbooks, fewer third-party backup tools with first-class RustFS support, and a smaller talent pool who have run it at scale. None of that is a reason to skip RustFS, but it is a reason to validate on your own workload and keep a rollback path. RustFS is betting that Rust's safety and write-path concurrency outweigh ecosystem lag over time.

What's next after 1.0.0 GA?

GA is a start line, not a finish. The RustFS team's stated next steps are two: harden what shipped and push further into AI infrastructure. On hardening, the plan is to keep absorbing feedback from production users and close rough edges in replication, tiering, and the console.

On the AI side, RustFS is adding S3 Vectors support and framing a RustFS 2.0 built around AI data centers, object storage that feeds training and inference pipelines directly. That direction fits the project's positioning as AI-native storage rather than a generic S3 bucket. For now, treat S3 Vectors and 2.0 as roadmap, not something you can deploy today. If vector storage or AI data lakes are on your 2027 plan, RustFS is worth watching, but build this year's system on what is GA.

Should you migrate from MinIO to RustFS now?

If you are greenfield or adding a new storage tier, RustFS 1.0.0 is reasonable to adopt for production workloads that are S3-centric and write-heavy. The S3 compatibility means your existing clients work unchanged, and the Apache 2.0 license means no commercial surprise.

If you already run MinIO or Ceph in production with mature tooling, do not rip it out. Migrate one workload, measure it, and expand only if RustFS earns it on your numbers. The lowest-risk first project is a secondary tier (backups, artifacts, or a staging bucket), where a write-path latency win shows up without betting the primary data path. RustFS is not asking you to trust a promise; it is asking you to run a benchmark on your own traffic. That is the right bar for a 1.0.

FAQ

Is RustFS compatible with the S3 API?

Yes. RustFS implements the S3 API, so existing S3 clients such as the AWS CLI, MinIO's mc, and rclone work against a RustFS endpoint without code changes. Multipart upload, versioning, and object lock are supported.

Can RustFS replace MinIO?

For S3-centric, write-heavy workloads, RustFS 1.0.0 is production-ready and a viable alternative. It does not have feature parity yet. MinIO's ecosystem and read-mixed performance lead. Migrate one workload first and measure.

Is RustFS free and open source?

Yes. RustFS is licensed under Apache 2.0, with the source on GitHub. There is no commercial license gate on the core object storage engine.

How do I install RustFS?

The fastest method is a single Docker container using the official rustfs/rustfs:latest image (API on 9000, console on 9001). A one-click script (install_rustfs.sh) covers bare metal, and Helm charts cover Kubernetes.

Does RustFS run distributed, and is it production-ready?

Distributed deployment is listed as available in RustFS 1.0.0. Single-node mode is the simplest production start. Durability comes from bitrot protection, versioning, and bucket or site replication. S3 Tables (Iceberg) remains in preview.

Originally published at the RustFS blog.

Ceph vs RustFS: When to Use What (2026)

Ethan Carter — Wed, 09 Sep 2026 12:15:00 +0000

Ceph vs RustFS: When to Use What (2026)

Ceph and RustFS both speak S3, but they solve different problems. Ceph is a unified object, block, and file platform that scales to exabytes and carries real operational weight. RustFS is a single-binary, S3-focused object store for teams that want MinIO-level simplicity without a storage team. Choose Ceph when one cluster must serve block, object, and file. Choose RustFS when the workload is S3 objects and you want it live today.

What is the real difference between Ceph and RustFS?

Ceph is a distributed object, block, and file storage platform. Its documentation states it uniquely delivers object, block, and file storage in one unified system, reaching from petabytes to exabytes of data across thousands of clients. Under the hood sits RADOS (Reliable Autonomic Distributed Object Store), with RBD for block, RGW for the S3/Swift gateway, and CephFS for POSIX file access layered on top.

RustFS is narrower by design. It is S3-compatible object storage written in Rust and released under the Apache 2.0 license. The project's README lists S3 Core Features, Versioning, Bucket Replication, Event Notifications, Bitrot Protection, Single Node Mode, Multi-Tenancy, Keystone Auth, and the Swift API as available today. There is no block layer and no file layer. That scope difference is the whole story: Ceph is the generalist that spans three storage paradigms, RustFS is the S3 specialist that does one thing with a small operational surface.

How does each one deploy?

Ceph is a cluster of cooperating daemons. The official cephadm tool bootstraps a tiny cluster on a single node with one Monitor and one Manager, then you expand it by adding hosts and provisioning OSDs (object storage daemons), MDS (metadata servers, needed for CephFS), and RGW (the S3 gateway). Each daemon type carries its own configuration, failure modes, and tuning surface, and a production deployment normally spans several nodes with redundant monitors. This is not a flaw; it is the cost of unifying three storage paradigms in one system.

RustFS ships as a single binary or a container. The documented start command is:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[sourced from https://github.com/rustfs/rustfs/blob/main/README.md, NOT EXECUTED IN CI]

That one line serves the S3 API on port 9000 and a console on 9001. For bare metal there is a one-line installer:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

[sourced from https://github.com/rustfs/rustfs/blob/main/README.md, NOT EXECUTED IN CI]

Default credentials are rustfsadmin / rustfsadmin, which you must rotate before any production exposure.

When should you pick Ceph over RustFS?

Reach for Ceph when a single storage layer has to do more than one job. If your Kubernetes cluster needs RBD block volumes for databases, an S3 endpoint for applications, and a CephFS mount for shared files, all from the same hardware, Ceph is one of the few open-source systems that covers all three. Its RADOS gateway implements a large subset of the Amazon S3 RESTful API, so most S3 clients work against it with only minor differences in canned ACLs.

Ceph also earns its keep at serious scale. The project documents petabyte-to-exabyte deployments and a mature ecosystem with multiple vendors offering paid support. If you already employ storage engineers who know CRUSH maps, placement groups, and OSD recovery, Ceph's operational model is a feature rather than a tax. Choose it when scope and scale outrun simplicity, and when block or file access is a hard requirement rather than a nice-to-have.

Where does RustFS win?

RustFS wins on the axis most small and mid-size teams actually care about: time to a working S3 endpoint. One binary, one docker run, no monitor quorum to reason about, no placement groups to size. The Feature & Status table is honest about what is ready: S3 Core, Versioning, Bucket Replication, Event Notifications, and Bitrot Protection are marked available, which covers the common object-storage patterns, backup targets, and application data lakes.

The Apache 2.0 license matters more than people admit. Ceph is mainly LGPL 2.1/3.0 with some GPL and BSD components; if you ever embed the engine rather than run it as a service, Apache 2.0 spares you copyleft questions. And because RustFS is written in Rust, there is no garbage collector in the request path, which keeps tail latency easier to reason about under mixed workloads. For teams that just need S3 without becoming storage experts, that combination is the point.

Feature-by-feature: what the docs say

The table below is a straight read of each project's own documentation as of August 2026. No benchmarks, no marketing. Ceph's RGW feature rows come from the official RADOS Gateway S3 API reference; RustFS's come from its GitHub README Feature & Status table.

Capability	Ceph (RGW)	RustFS
S3 buckets, objects, multipart	✅	✅ (S3 Core)
Object Versioning	✅	✅
Bucket Lifecycle	✅	🚧 Under Testing
Bucket Replication	⚠️ Partial (across zones only)	✅
Bucket / Object Notifications	✅	✅ (webhook)
Block storage (RBD)	✅	❌ (object only)
File storage (CephFS)	✅	❌
K8s deployment	Rook operator / cephadm	✅ Helm Charts
Console / UI	ceph-mgr dashboard	✅ web console
License	LGPL 2.1/3.0 (mixed)	Apache 2.0
Implementation	C++ daemons	Rust (single binary)

Two rows deserve a call-out. RustFS lists Lifecycle Management as Under Testing, so do not promise automated tiering yet. Ceph's Bucket Replication is marked partial and permitted only across zones in RGW. Both gaps are real and both are documented by the projects themselves, not by us.

What can RustFS not replace yet?

Be clear about the boundaries before you commit. RustFS does not provide block storage or a POSIX file system; the README's Feature & Status table has no RBD and no FUSE/POSIX mount, and no Object Lock. If your requirement is WORM retention or a mounted file system, Ceph or a purpose-built tool is the answer today.

Distributed Mode is also Under Testing in RustFS. Single-node is a supported, solid story; multi-node production at petabyte scale is not a promise the project makes yet. Its ecosystem is younger than Ceph's by more than a decade, so you will find fewer third-party integrations and fewer war-story blog posts. I would not pitch RustFS as a drop-in for an existing exabyte Ceph fabric. For S3 object workloads that fit a single node or a modest cluster, it is a different conversation and one worth having.

How do you choose between them?

Use this rule of thumb. If the words "block", "file system", and "object" all appear in the same ticket, start with Ceph. If only "S3" appears and the real constraint is headcount or time, start with RustFS.

A hybrid is legitimate and common. Run Ceph for the infrastructure layer (RBD for stateful workloads, CephFS for shared files) and RustFS in front of the S3-facing applications where operational simplicity pays off. They do not conflict; they occupy different layers. The mistake is forcing one to do the other's job. Ceph dragged into a pure-S3, small-team role becomes expensive overhead. RustFS pushed past its single-node comfort zone becomes a bet on an untested distributed path. Match the tool to the layer and both stay boring in the good way.

FAQ

Is RustFS a replacement for Ceph?

Partially, and only for the object-storage slice. RustFS can replace the RGW (S3 gateway) portion of a Ceph stack for S3-focused workloads, with far less operational overhead. It cannot replace RBD block storage or CephFS file storage. If your stack needs block or file from the same cluster, Ceph stays the right call.

Is Ceph harder to operate than RustFS?

Yes, materially. Ceph runs multiple daemon types (MON, OSD, MDS, MGR, RGW), each with its own config and failure modes, and a real cluster spans several nodes. RustFS is one binary or one container; the documented start is a single docker run. The trade-off is scope: Ceph covers three storage paradigms, RustFS covers object storage with less to manage.

Does Ceph support the S3 API?

Yes. Ceph's RADOS Gateway implements a large subset of the Amazon S3 RESTful API, plus a subset of the OpenStack Swift API, with its own user management. Most S3 clients work against it, though Ceph notes differences in canned ACLs and a partial Bucket Replication (across zones only). It is not a 100 percent S3 clone, but it covers the common operations.

Can RustFS run on a single node?

Yes. RustFS lists Single Node Mode as an available feature, and the documented quick start is a single docker run on one host serving S3 on port 9000 with a console on 9001. Multi-node distributed mode is Under Testing, so production scale-out is not a promise yet. For many S3 object workloads a single node is enough.

What license is each project under?

RustFS is Apache 2.0, a permissive license with no copyleft. Ceph is mainly dual-licensed under LGPL 2.1 or 3.0, with some GPL and BSD components in the tree. If you run either as a service the practical difference is small; if you embed the engine in a shipped product, Apache 2.0 is the more flexible choice.

Wasabi vs RustFS: S3-Compatible Cost Showdown

Ethan Carter — Mon, 07 Sep 2026 12:14:10 +0000

Disclosure: I work on RustFS. Every comparison number below is pulled from a primary source and dated 2026-08-21, so check the Sources table before you budget.

Wasabi wins on simplicity: one flat $7.99 per TB per month with no egress or API fees, ideal when you have no hardware and want 16 regions by this afternoon. RustFS wins on raw dollars per TB once you already own or colocate servers, because storage becomes disks you bought instead of a recurring bill. The real question is not "which is cheaper" but "who carries the hardware, power, and operations."

Key Stats

Metric	Wasabi	RustFS
Storage price	$7.99 / TB / mo (NA, EMEA, APAC)	$0 / TB / mo — you supply disks
Egress	Free under fair-use (≤ stored volume / mo)	Free — it is your own network
API requests	Free under fair-use	Free
Minimum term	90-day retention charge	No minimum
Minimum commitment	1 TB billed even if unused	None
Storage regions	16 regions	Wherever you place the node
Durability claim	11 nines (99.999999999%)	Your redundancy
Object Lock (WORM)	Yes	Not in README Feature table
License	Proprietary	Apache 2.0

What This Cost Showdown Actually Measures

A cost showdown between a cloud and a self-hosted object store is not a single price tag. It is the recurring opex of renting capacity versus the capital plus operations of owning it. Wasabi quotes a flat rate per terabyte per month and bills nothing extra for moving data in, out, or for API calls, within fair-use limits. RustFS is software you run; the bill is the servers, disks, power, and the engineer who gets paged. The comparison only makes sense once you know your dataset size, how long it lives, and whether you already pay for a rack somewhere. Everything below keeps those three variables explicit so the numbers stay honest.

How Does Wasabi Price Storage?

Wasabi's pay-as-you-go Hot Cloud Storage lists at $7.99 per TB per month across North America, EMEA, and APAC, with no separate charge for ingress, egress, or API requests (Wasabi pricing page, checked 2026-08-19). That flat rate is the headline and the reason backup vendors default to it. For 100 TB, the math is 100 × $7.99 = $799 per month, or $9,588 per year, before any minimums. Reserved-capacity terms in 1, 3, or 5-year blocks lower that further if you can commit capacity up front. What you do not see on the front page is the small print that decides whether your real bill matches the calculator, and that is where the next section lives.

Where Does the Wasabi Bill Actually Bite?

Three clauses in the pay-as-you-go FAQ change the number for real workloads. First, a 90-day minimum storage duration: delete an object before 90 days and you still pay "Timed Deleted Storage" for the remaining days, so high-churn data costs more than its live size suggests. Second, a 1 TB monthly minimum means storing 200 GB still bills a full terabyte. Third, objects under 4 KB are billed as 4 KB, and enabling versioning or bucket logging grows your stored bytes and your bill. Egress is free only while monthly downloads stay at or below your active stored volume; above that, Wasabi calls it "not a good fit" and can limit or suspend service.

What Does RustFS Actually Cost to Run?

RustFS charges nothing per gigabyte because it is software, not capacity. Your cost is hardware. A 24 TB enterprise HDD listed for roughly $400 to $600 in 2025 (market retail range), so raw capacity capital lands near $17 to $25 per TB one-time. Amortize that drive over a five-year life and disk capEx is about $3.40 to $5 per TB per year, before you account for mirrors or backups. Power, cooling, and rack space are extra, but your bandwidth is free because the data never leaves your network through a metered exit. The trade you accept is operations: patching, monitoring, and recovery are on you, not a support ticket.

RustFS Feature Reality Check: What You Get, What You Don't

RustFS's README Feature & Status table is the only honest source for what the software does today. Marked Available: S3 Core, Upload/Download, Versioning, Bucket Replication, Logging, Event Notifications, K8s Helm Charts, Keystone Auth, Multi-Tenancy, Swift API, Bitrot Protection, and Single Node Mode. Under Testing: Lifecycle Management, Distributed Mode, and RustFS KMS. The table does not list Object Lock, erasure coding, or a FUSE mount, so do not plan a WORM archive or a POSIX filesystem on RustFS yet. Read it plainly: single-node RustFS is ready now, multi-node redundancy is not a production promise. I work on RustFS, so weigh that against the README, not my enthusiasm.

How Do the Two Compare on Cost-Relevant Features?

Dimension	Wasabi	RustFS
Footprint	16 managed regions	Your hardware, your site
Durability	11 nines (vendor claim)	What you build (mirror or backup)
Immutable retention	Object Lock: WORM, legal hold	Not available today
Lifecycle / tiering	Yes	Under Testing
Egress model	Free under fair-use	Your network, no meter
Lock-in	Proprietary account	Apache 2.0, you own it

The features that move cost are durability and lifecycle. Wasabi hands you 11 nines and Object Lock out of the box; RustFS hands you the software and expects you to supply redundancy until Distributed Mode ships. If compliance retention matters, that gap is the whole decision today.

When Should You Pick Wasabi?

Pick Wasabi when you have no servers and no interest in running storage. It is the faster path to 16 regions, data-residency choices across North America, Europe, and Asia, and immutable backups through Object Lock without standing up anything. The flat $7.99 per TB per month is easy to defend in a budget review because there is no egress line item to explain. For backup, DR, and active archive where someone else should carry the pager, the recurring fee buys absence of operational load. If your team's core product is not storage, renting it is usually the cheaper career move.

When Does RustFS Win on Total Cost?

RustFS wins when the dataset is large, stable, and already sitting on hardware you pay for. At 100 TB, Wasabi recurs at $9,588 per year; the same capacity as disks you own amortizes to a few dollars per TB per year in capital, and your egress is free because the traffic never crosses a metered provider link. The break-even is earlier the more you download, since Wasabi's fair-use cap bites exactly where RustFS has no meter. The catch is durability: until Distributed Mode leaves testing, mirror to a second node or back up to another S3 target yourself. You trade a recurring fee for owned infrastructure and Apache-2.0 freedom from lock-in.

How Do You Point Your Tools at Each?

Both speak the S3 API, so the same SDKs and CLI habits apply. For Wasabi, the MinIO Client connects with the commands from Wasabi's official doc:

# Wasabi via MinIO Client (mc) — verbatim from docs.wasabi.com
# [sourced from https://docs.wasabi.com/how-do-i-use-minio-client-with-wasabi, NOT EXECUTED IN CI]
wget https://dl.minio.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc config host add https://s3.wasabisys.com   # prompts for access/secret key
./mc ls

And the AWS CLI, also verbatim from Wasabi's doc:

# [sourced from https://docs.wasabi.com/v1/docs/how-do-i-use-aws-cli-with-wasabi, NOT EXECUTED IN CI]
aws s3 ls --endpoint-url=https://s3.wasabisys.com

RustFS starts with the README's docker command:

# [sourced from https://github.com/rustfs/rustfs, NOT EXECUTED IN CI]
docker run -d -p 9000:9000 -p 9001:9001 \
  -v $(pwd)/data:/data \
  -v $(pwd)/logs:/logs \
  rustfs/rustfs:latest

Default credentials are rustfsadmin / rustfsadmin; change them before exposing port 9000.

FAQ

Is Wasabi really cheaper than AWS S3?

At typical access volumes, yes. Wasabi's own comparison example sizes 100 TB stored with 20 TB downloaded per month at about $818 versus roughly $4,097 for AWS S3, once egress is included. The gap comes almost entirely from Wasabi's missing egress and API line items. Whether that beats self-hosting depends on whether you already own the hardware; the cloud comparison only holds against other clouds.

Does RustFS charge per gigabyte?

No. RustFS is open-source software under Apache 2.0; you run it on your own disks and pay no per-GB fee and no egress fee, because the data stays on your network. Your cost is the servers, drives, power, and the operations work to keep them healthy. Treat it as capital and labor, not a metered bill.

Can RustFS replace Wasabi for immutable backups?

Not today for compliance retention. RustFS's README Feature & Status table does not list Object Lock, while Wasabi and Backblaze B2 both offer S3 Object Lock with WORM and legal hold. If you need ransomware-immutable, audit-ready retention, use Wasabi, B2, or Ceph now and keep RustFS for analytical and batch object workloads where immutability is not the requirement.

What is Wasabi's 90-day minimum storage policy?

Objects deleted before they have been stored 90 days incur a "Timed Deleted Storage" charge for the remaining days, comparable to minimum-duration policies on some hyperscaler tiers. It mainly hurts high-churn workloads where data lives for days or weeks. For long-lived archives it is invisible, because the data stays past the threshold anyway.

Is RustFS production-ready?

Single-node RustFS is marked Available and is a reasonable place to start today. Distributed Mode, Lifecycle Management, and RustFS KMS are still Under Testing, so multi-node redundancy and automated tiering are not production promises yet. Run a single node and mirror or back it up to a second S3 target until Distributed Mode ships; do not bet a WORM archive on it yet.

Sources

All claims checked against these primary sources on 2026-08-21.

Claim	Source
$7.99/TB/mo, no egress/API fees, 1 TB monthly min, 90-day retention, 4 KB min object, fair-use egress cap	https://wasabi.com/pricing , https://wasabi.com/pricing/faq
16 storage regions, S3 + IAM API, Object Lock, 11 nines durability	https://wasabi.com/cloud-object-storage
Wasabi mc commands	https://docs.wasabi.com/how-do-i-use-minio-client-with-wasabi
Wasabi AWS CLI commands	https://docs.wasabi.com/v1/docs/how-do-i-use-aws-cli-with-wasabi
RustFS Feature & Status, docker run, default creds, Apache 2.0, `1.0.0-rc.3`	https://github.com/rustfs/rustfs
Enterprise 20TB+ HDD retail range $400-$600 (2025), used as raw-capacity cost anchor	https://www.accio.com/business/hdd-failure-rate (market summary, "Enterprise High-Capacity HDDs (20TB+)" section)

Self-Hosting S3-Compatible Storage on Bare Metal

Ethan Carter — Thu, 27 Aug 2026 12:52:00 +0000

You self-host S3-compatible storage on bare metal by installing a single Rust binary on a Linux server and pointing any S3 client at it. RustFS installs with one script, listens on port 9000 (S3 API) and 9001 (console), and is Apache 2.0 licensed. Single-node mode is production-ready today; multi-node clustering is still under testing.

Every command below is copied verbatim from the official source cited beside it. This sandbox has no Docker daemon, so none of the commands were executed here; they are marked accordingly.

Key Stats

Fact	Source
RustFS installs with one command and runs as a systemd service on x86_64 or aarch64 Linux	RustFS docs (Linux quick-start)
Default S3 API port is 9000; console port is 9001	RustFS GitHub README
Default credentials are `rustfsadmin` / `rustfsadmin` and must be changed	RustFS README + docs
RustFS is Apache 2.0 licensed and S3-compatible	RustFS GitHub README
Single-node mode is production-ready; distributed mode is still under testing	RustFS README Feature & Status

What is self-hosted S3-compatible storage?

A self-hosted S3-compatible storage server is a program you run on your own hardware that speaks the Amazon S3 API. Applications using AWS SDKs, the aws CLI, or MinIO's mc can talk to it without code changes, because the bucket, object, and credential model matches S3. The difference from a cloud bucket is ownership: the disks, the network path, and the uptime are yours.

RustFS is one such server, written in Rust and licensed under Apache 2.0. It exposes the S3 API on port 9000 and a web console on 9001, and it stores objects on the local filesystem. Because it is S3-compatible, the same client code that targets AWS S3 also targets a RustFS node. That compatibility is the whole point of self-hosting here: you get an S3 endpoint without renting one.

Why run object storage on bare metal?

Running object storage on bare metal means installing the server directly on a Linux machine instead of in a container or a managed cloud. The appeal is control. Your data stays on disks you own, in a network you define, and the cost is the hardware plus your time rather than a per-gigabyte cloud bill that grows with every upload.

The trade-off is real and worth stating plainly. You own the failure modes too. A single disk dies, the server reboots, the certs expire, and that is now your incident, not a vendor's. Bare metal suits teams that already run Linux servers, care about data residency, or have steady high-volume storage that makes a cloud bill painful. It is a poor fit if you want zero operational responsibility. Self-hosting trades a recurring fee for recurring attention.

What you need before you start

Minimum: a Linux server with systemd on x86_64 or aarch64, root or sudo access, the unzip utility, outbound network access to pull the package, and open ports 9000 (S3 API) and 9001 (console) so clients and the browser console can reach the service.

Plan the data disk before you install. The quick-start runs RustFS in single-node, single-disk (SNSD) mode, which stores everything under /data/rustfs0 with no redundancy. That is fine for evaluation and dev, but a single disk has no spare. For anything you would hate to lose, attach a second disk, schedule filesystem or object-level backups, or wait for distributed mode to leave testing. The installer lets you change the data path and ports at install time, so decide the layout now rather than after data accumulates.

How do you install RustFS on bare metal?

Run the official one-line installer. It downloads the binary, installs it to /usr/local/bin/rustfs, registers a systemd service, starts it, and stores data under /data/rustfs0 by default:

curl -O https://rustfs.com/install_rustfs.sh && bash install_rustfs.sh

[sourced from RustFS docs (Linux quick-start), NOT EXECUTED IN CI]

After it finishes you get a summary with the service port (9000), console port (9001), and data directory, plus a security warning to replace the default credentials. Open http://<server-ip>:9001 in a browser to reach the console, then log in with the access and secret keys you set. The same server answers S3 API calls on 9000. From here you can create a bucket in the console or from the command line, which the next sections cover.

How do you run RustFS with Docker instead?

If you prefer a container, the Docker image gives you the same server with no systemd dependency. The official command maps both ports, persists data and logs to local volumes, and detaches:

docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest

[sourced from RustFS GitHub README, NOT EXECUTED IN CI]

Do not drop the -p or -v flags. Without -p 9000:9000 the S3 API is unreachable from the host, and without the volume mounts a container restart loses everything under /data and /logs. The image tag :latest tracks the newest build; pin a specific release like 1.0.0-rc.2 if you need reproducibility. Podman users run the equivalent with :Z,U on the volume mounts for SELinux. Either path lands you at the same console on 9001 and the same S3 endpoint on 9000.

How do you point an S3 client at your server?

RustFS ships its own CLI, rc, which handles both object operations and admin checks. After installing it, register an alias that stores the endpoint and keys, then verify the server is alive:

rc alias set local http://localhost:9000 <your-access-key> <your-secret-key>
rc ping local
rc ready local

[sourced from RustFS docs (rc CLI), NOT EXECUTED IN CI]

Create a bucket and push a file the same way any S3 client would:

rc bucket create local/my-bucket
rc object copy /path/to/hello.txt local/my-bucket/hello.txt
rc object list local/my-bucket

[sourced from RustFS docs (rc CLI), NOT EXECUTED IN CI]

Because the API is S3-compatible, AWS SDKs, the aws CLI with --endpoint-url, and MinIO's mc all work against port 9000 with the same keys. You are not locked into rc; it is just the native client. Point your application at http://<server-ip>:9000 and the buckets behave like any other S3 endpoint.

What RustFS does today, and what it doesn't

RustFS today is a solid single-node S3-compatible server. The features marked available in the project's own status table include S3 core operations, upload/download, versioning, logging, event notifications, bucket replication, bitrot protection, single-node mode, multi-tenancy, Keystone auth, and the Swift API. If your workload fits one node, those are real and usable now.

Be honest about the gaps. Distributed mode, lifecycle management, and RustFS KMS are still under testing, so multi-node erasure-coded clusters and automated tiering or expiry are not production promises yet. There is no Object Lock, no FUSE or POSIX mount, and no built-in erasure coding in the current release. RustFS does not ship a turnkey multi-region setup. For a single bare-metal node serving S3 to your apps, that is plenty. If you need cross-node redundancy or compliance retention today, plan around those limits rather than assuming they exist.

How do you keep a bare-metal server healthy?

First, change the default credentials. The installer leaves placeholder keys in /etc/default/rustfs; set your own access and secret keys there, then restart:

RUSTFS_ACCESS_KEY=<your-access-key>
RUSTFS_SECRET_KEY=<your-secret-key>   # e.g. output of: openssl rand -base64 24

sudo systemctl restart rustfs
sudo systemctl status rustfs --no-pager

[sourced from RustFS docs (Linux quick-start), NOT EXECUTED IN CI]

Keep the service under systemd so it restarts on boot, and watch systemctl status rustfs for the active (running) line. Because single-node mode has no redundancy, back up the data directory or replicate the bucket to a second site on a schedule you trust. Check disk space before it is gone, since a full volume stops writes. None of this is exotic, but it is yours to do. The console on 9001 shows bucket and object state when you want a visual check.

FAQ

Is RustFS free to self-host?

Yes. RustFS is licensed under Apache 2.0, so you can run it on your own hardware at no license cost. You pay only for the server, disks, and your time to operate it.

What are the default RustFS credentials?

The default access key and secret key are both rustfsadmin. The installer warns you to replace them, and if you leave them unset the server falls back to those same defaults. Set RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY in /etc/default/rustfs and restart the service before exposing the port.

Can RustFS replace MinIO on a single Linux server?

For single-node S3-compatible needs, yes. Both speak the S3 API, so existing client code and SDKs work against either. RustFS is written in Rust, licensed Apache 2.0, and ships its own rc CLI alongside the standard S3 interface.

Does RustFS support multi-node distributed storage?

Distributed mode is listed as under testing in the project's Feature & Status table, so it is not a production promise yet. Single-node mode is available and production-ready today. If you need cross-node erasure coding now, that gap should factor into your decision.

What port does RustFS use?

The S3 API listens on port 9000 and the web console on 9001. Both must be reachable: 9000 for client and application traffic, 9001 for the browser console.