An S3 throughput benchmark is only as honest as its setup. Most published numbers hide three variables that move results by 5x or more: where the client runs, how big the objects are, and which second of the run got reported. This article shows a workflow with warp that keeps those variables visible, on a real single-node server.
| Metric (first-hand run) | Value |
|---|---|
| GET throughput, 1 MiB objects, 8 workers | 663 MiB/s avg; 1s splits ranged 530 to 749 MiB/s |
| PUT throughput, same setup | 102 MiB/s avg; 1s splits ranged 63 to 128 MiB/s |
| GET throughput, 10 KiB objects, 8 workers | 136 MiB/s, 13,965 obj/s (27 s run); repeat run: 107 obj/s, 10,708 obj/s |
| PUT throughput, 10 KiB objects, 8 workers | 3.5 MiB/s, 360 obj/s; median latency 21 ms |
| Mixed workload total (GET/STAT/PUT/DELETE) | 266 MiB/s, 443 obj/s |
| GET TTFB median | 10 ms (1 min run), 5 ms (concurrency 4) |
| Environment | RustFS 1.0.1, single node, laptop (Ryzen 7 5800H, 16 GB RAM, NVMe SSD), warp 1.3.1, loopback |
Treat every number in that table as a relative reference point, not a capacity claim. The server ran on loopback, on a developer machine, with a client on the same host. That limitation is the first thing this article teaches you to disclose.
What a throughput number actually measures
A throughput number is the output of three components in series: the benchmark client, the network path, and the storage server. Whichever one is slowest sets the ceiling, and the benchmark report will not tell you which one it was. When I ran warp get against RustFS 1.0.1 on loopback, the client and the server shared one machine, so the network component was effectively zero and the disk did most of the talking.
That setup is fine for comparing versions of the same server. It is useless for predicting what a remote client will see. In my run, GET averaged 663 MiB/s with 8 concurrent workers on 1 MiB objects. A client on another host, behind one 1 GbE link, would cap near 125 MiB/s and blame the server for the difference. Name the path in every result you publish, or the number means nothing.
PUT is not GET
The fastest way to spot an untrustworthy storage benchmark is one that reports a single throughput number. On the same server, same run length, same concurrency, my PUT result was 101.6 MiB/s while GET was 663.3 MiB/s, a 6.5x gap on one node. The write path persists data before it acknowledges; the read path serves from page cache after the first fetch. No amount of tuning closes that gap entirely, and pretending it does not exist misleads every reader.
The mixed benchmark shows the interaction effect too. Under a combined GET/STAT/PUT/DELETE workload, GET dropped from 663 MiB/s standalone to 199.6 MiB/s, and PUT fell from 101.6 to 66.3 MiB/s. Writes steal IOPS and cache from reads. If your real workload writes while others read, the mixed number, not the standalone one, is your planning baseline. Publish both or publish neither.
How long should a benchmark run?
Long enough that warm-up stops mattering. The first seconds of a run are cold: empty caches, open connections, and the server allocating space. warp handles this explicitly. Per its documentation, the analysis window starts when every thread has completed one request and ends when the last thread starts its final one, which is why the analyzed time is slightly below the requested duration.
Run length also drives variance. My 1 minute GET run split into 57 one-second buckets: fastest 749 MiB/s, median 672.7, slowest 530.5. That is a 1.4x spread between best and worst second, on loopback, with nothing else running. A 10 second run would have waved that spread around as noise, and a 5 minute run narrows the confidence interval further. Short runs also disagree with each other: two independent 10 KiB GET runs, one 30 seconds and one 20 seconds, averaged 13,965 and 10,708 obj/s, a 23 percent gap between runs measured minutes apart on an idle machine. For anything you plan to publish or reuse in capacity planning, run at least a minute per operation type, run it more than once, and report the spread, not a single second.
Why object size changes the answer
Throughput in MiB/s and objects per second are different claims, and object size decides which one you can honestly make. At 1 MiB objects, my GET run delivered 663 obj/s. At 10 KiB objects on the same server, the same 8-worker setup delivered 13,965 obj/s but only 136 MiB/s: 21 times more objects, a fifth of the bandwidth. Per-request overhead dominates at small sizes, and which of the two numbers matters depends entirely on what your traffic looks like.
The write side told an even sharper story. PUT at 1 MiB managed 101.6 obj/s; PUT at 10 KiB managed 360 obj/s. Objects a hundred times smaller bought only 3.5 times more objects per second, and median latency sat at 21 ms per request. Whatever the write path is doing per request, it does not shrink in proportion to object size, so small-object write workloads hit that per-request floor long before they stress bandwidth. If your production traffic is millions of tiny uploads, benchmark at that size, not at 1 MiB, because the 1 MiB number will flatter the server.
warp defaults each benchmark type to its own object size and lets you override it with --obj.size. It also supports randomized sizes (--obj.randsize), which the documentation describes as log2-distributed: equal object counts per doubling of size, so the average lands well below the maximum you pass. Pick the size your production traffic actually has. If you do not know it yet, measure that first, because a benchmark with the wrong object size is an answer to a question nobody asked.
Report the distribution, not the best second
Every benchmark produces a distribution, and the dishonest version of a report quotes its maximum. warp prints three 1-second aggregates by default: fastest, 50% median, and slowest. My GET run: 749, 672.7, 530.5 MiB/s. My PUT run: 128.2, 101.7, 63.4 MiB/s. The honest summary line is the median with the range next to it, because your production p50 will look like the median, and your worst tenancy-neighbor day will look like the slowest second.
Latency deserves the same treatment. In the GET run, TTFB median was 10 ms, the 99th percentile 16 ms, the worst single request 36 ms. In the mixed run, STAT requests answered in a median 2.5 ms while DELETE sat at 39.1 ms median, because deletes wait on real I/O. Quote percentiles with the operation name attached. An average latency across mixed operations is a number that describes nothing.
Where should the benchmark client run?
As close to the server as honesty allows, which is rarely on the same box. Loopback removes the network, so it measures the server plus the client kernel, and results only transfer to deployments with a similar path. My numbers came from loopback on purpose: the goal was comparing server versions, not predicting a datacenter. When you need a network-realistic number, run the client on a separate machine on the same segment, and run it from several at once.
warp supports both modes. Multiple hosts can be passed comma-separated (--host=10.0.0.1:9000,10.0.0.2:9000), and for larger setups a warp server can drive remote warp client processes with --warp-client=client-{1...10}, each applying the same concurrency. The documentation's example of a distributed GET run uses 10 clients against 16 servers. Distributed runs also change interpretation: parameters apply per client, so total load multiplies. Write down the client topology next to the result.
Watch the server while the client runs
The warp report tells you what the client saw. It never tells you why. While a 20-second GET run against 10 KiB objects was in flight, I sampled the server machine every two to three seconds: CPU sat between 94 and 100 percent for the entire window, and physical disk reads stayed between 0 and 1.1 MB/s while warp reported about 105 MiB/s being served. Every one of those bytes came from the operating system's page cache, not from the disk. On a loopback setup the benchmark client shares those same saturated cores with the server, so the number measures the whole machine, not just storage.
Sampling during the run is what turns a suspicious number into an explained one. If CPU is pegged, you are client- or processor-bound; if the disk is pegged, the storage path is the ceiling; if neither is busy while throughput looks high, you are benchmarking the cache. Any process-level monitor works: CPU percentage, disk read/write bytes per second, free memory (watch it fall while the cache fills). One caveat from my own setup: my CIM-based sampler needed two to three seconds per sample, so treat those numbers as coarse, and put the sampler on the machine the server runs on.
Your GET number is probably a cache number
Cold reads are what your cache-miss traffic will actually see, and my runs did not measure them. The GET figures in this article are warm-cache numbers: the prefill wrote every object moments before the reads, and the page cache absorbed them. Getting closer to a cold number takes one of three things: a working set larger than physical memory so the cache cannot hold it, dropping the OS cache between runs (Linux has /proc/sys/vm/drop_caches; Windows exposes no supported equivalent, which is why my Windows-hosted numbers stay warm), or --obj.randsize to spread reads across a much wider object set. Say which one you did, because a GET figure quoted without its cache state is a number with an asterisk nobody wrote down.
Benchmarks need capacity too
This one cost me a failed run. The 1 minute PUT benchmark at 101.6 MiB/s wrote roughly 5.8 GB of test objects; the earlier GET run had already filled most of a bucket. When I raised the object size to 10 MiB, uploads failed with a generic InternalError from the server, and the benchmark aborted before it started. The server log showed the real cause: os error 112, storage full, on the data volume.
Two lessons worth the tuition. First, size your test volume before the run: benchmark data is real data, warp writes it at full speed, and a bucket can grow by gigabytes per minute. Second, a client-side error message is not a diagnosis. The S3 error XML said "please try again"; only the server log said "disk full". When a benchmark fails, read the server log before you conclude anything about the server.
A warp workflow you can copy
warp is MinIO's open-source S3 benchmark tool, and the workflow below uses only flags and syntax documented in its README. It creates and completely cleans a bucket called warp-benchmark-bucket before and after each run, so never point it at a bucket holding data. Three runs cover the planning basics:
warp get --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8
warp put --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8
warp mixed --host=127.0.0.1:9000 --access-key=... --secret-key=... --duration=1m --obj.size=1MiB --concurrent=8
Each run prints the per-operation report with fastest/median/slowest 1-second splits, then saves raw request data for later analysis with warp analyze --analyze.op=GET --analyze.v <file>.csv.zst, which reproduces segmented statistics per request-size band. One caveat from my run: warp 1.3.1 on Windows wrote the .json.zst run metadata but not the raw CSV, so treat the analyze step as version- and platform-dependent and capture the console report as your primary record.
Frequently Asked Questions
What is warp?
warp is an open-source S3 benchmarking tool from MinIO that measures GET, PUT, DELETE, STAT, and mixed workloads, saves raw request data, and reports throughput, request latency, and TTFB distributions with warm-up excluded from the analysis window.
Why is PUT so much slower than GET on the same server?
The write path persists objects before acknowledging while the read path is served after a single fetch, often from cache. In my single-node loopback run the gap was 6.5x (102 MiB/s PUT vs 663 MiB/s GET at 1 MiB objects); the exact ratio depends on hardware, but a large gap is normal, and a benchmark showing only GET throughput is incomplete.
What concurrency should I use for an S3 benchmark?
Start where your application actually operates, then sweep. warp's --concurrent flag sets workers per client; too few workers leave the server idle and too many measure your client's queuing instead of the server. My 8-worker loopback runs saturated a single node comfortably; the honest method is reporting throughput at several concurrency levels.
How do I benchmark more than one S3 server?
Pass hosts comma-separated (--host=10.0.0.1:9000,10.0.0.2:9000) so warp alternates between them, or use distributed mode with remote warp client processes driven by --warp-client=client-{1...10}. Note that flags apply to each client individually, so total load scales with client count.
Why are my GET numbers suspiciously high?
Check whether the operating system's page cache is serving the reads. In my runs, warp reported 105 MiB/s of GET traffic while physical disk reads sat near zero; the objects had been written moments earlier and never left memory. Options for a more realistic figure: a dataset larger than RAM, dropping the cache between runs on Linux, or --obj.randsize to spread reads across a wider object set.
What number should I publish from a benchmark?
The median with its range and the full environment: server version, node count, network path, object size, concurrency, and run duration. For example: 663 MiB/s median-bucket 673 MiB/s (530 to 749 range), RustFS 1.0.1 single node, loopback, 1 MiB objects, 8 workers, 1 minute. That sentence is falsifiable, which is exactly what makes it worth publishing.
A note on how this was verified
Article type: this is a hands-on benchmark methodology report based on runs I executed, plus documentation-sourced tool behavior.
The benchmark runs in this article were executed first-hand on 2026-10-12 against a single-node RustFS 1.0.1 (Windows x86_64 binary, GitHub release build dated 2026-10-03) on loopback, using warp 1.3.1 (143bf4f): GET 1 MiB / 8 workers / 1 min, PUT 1 MiB / 8 workers / 1 min, GET 1 MiB / 4 workers / 20 s, mixed 1 MiB / 8 workers / 40 s, GET 10 KiB / 8 workers / 30 s, PUT 10 KiB / 8 workers / 30 s, and a repeat GET 10 KiB / 20 s, all quoted with real console output. Server-side resource samples (CPU, free memory, disk read/write bytes per second) were collected on the same machine during the repeat GET run with a CIM-based sampler polling every two to three seconds; they are coarse and cover one machine. The tool-behavior claims (bucket cleanup, warm-up exclusion, randomized size distribution, multi-host and warp-client flags, the mixed-workload operation mix of GET 45% / STAT 30% / PUT 15% / DELETE 10%) come from the warp v1.3.1 README, not from my runs. Not exercised: any network path (all runs were loopback), distributed warp-client runs, cold-read or cache-bypassed measurements (every GET figure here is warm-cache; the cache section explains the options), object sizes above 1 MiB (the 10 MiB attempt aborted on a full test volume), runs longer than one minute, and the warp analyze step, which could not run because warp 1.3.1 did not persist the raw CSV on Windows in my environment; that limitation is quoted above rather than worked around. Nothing here predicts a production cluster. Re-run the same commands against your own hardware before you plan capacity on any of it.
Sources
- warp v1.3.1 README (commands, defaults, warm-up behavior, mixed-workload mix): https://github.com/minio/warp/blob/v1.3.1/README.md
- warp releases (v1.3.1 Windows binary used for the runs): https://github.com/minio/warp/releases/tag/v1.3.1
- RustFS GitHub repository and releases (1.0.1 Windows binary used for the runs): https://github.com/rustfs/rustfs/releases/tag/1.0.1
- RustFS documentation (deployment and compatibility): https://docs.rustfs.com
Top comments (0)