Every Droplet comes with its own SSD, and DigitalOcean also sells Volumes, network block storage you attach to a Droplet and can move to another one. I'd only ever used volumes for "more space", and never checked what they cost in speed or what moving one actually looks like. So I put a volume and a Droplet's own disk through the same tests, then moved the volume between two Droplets, once properly and once by detaching it while Postgres was writing to it.
The setup
Two s-2vcpu-4gb Droplets in fra1, which is the shared-CPU plan at $24 a month, running Ubuntu 24.04, and a 100 GiB volume at $10 a month. I let DigitalOcean format the volume as ext4 on creation. It doesn't add an fstab line; it writes a systemd mount unit, mnt-vt.mount, with defaults,nofail,discard,noatime.
DigitalOcean documents the limits for volumes, and they depend on the Droplet. On a shared-CPU Droplet a volume gets 7,500 IOPS and 300 MB/s, and can burst to 10,000 IOPS and 450 MB/s "for up to 60 seconds". The docs say nothing about latency.
I ran fio 3.36 with direct I/O on an 8 GiB file on each disk, local first, then the volume, with a 120 second pause before every volume run so its burst could reset. Then pgbench against two Postgres 16 instances with identical settings, one with its data directory on each disk.
The burst is exactly as documented
Sequential reads in 1 MiB blocks ran at 450 MiB/s for 62 seconds and then dropped to a flat 300. Sequential writes did the same, within a second. Random 4 KiB writes with 32 in flight held 10,000 IOPS for about 80 seconds and then sat on 7,500. These are hard limits, not rough ones. The line is almost perfectly straight once it drops.
The Droplet's own disk did 4,102 MB/s sequential reads and 1,206 MB/s writes over the same runs, and 39,703 random write IOPS. It wasn't close on throughput, and nobody should expect it to be.
| Test | Droplet's disk | Volume |
|---|---|---|
| Sequential read, 1 MiB, 150 s | 4,102 MB/s | 378 MB/s |
| Sequential write, 1 MiB, 150 s | 1,206 MB/s | 378 MB/s |
| Random read, 4 KiB, 32 in flight | 75,274 IOPS | 7,162 IOPS |
| Random write, 4 KiB, 32 in flight | 39,703 IOPS | 8,833 IOPS |
| Random read, 4 KiB, one at a time | 4,391 IOPS | 1,006 IOPS |
| Random write, 4 KiB, one at a time | 2,575 IOPS | 3,432 IOPS |
| 4 KiB write + fdatasync, one at a time | 2,133/s | 838/s |
The volume's 378 MB/s averages are the 450 burst and the 300 after it, blended over 150 seconds.
Latency is where it shows up
With one request at a time, which is how a lot of code reads, the limits don't matter. Latency does. A random 4 KiB read took a median of 146 µs on the local disk and 700 µs on the volume, and at the 99th percentile, 0.95 ms against 5.7 ms. Every read is a trip over the network.
Writes went the other way. A 4 KiB write took 169 µs on the volume and 272 µs locally. The kernel reports a write cache on the volume's disk, so the write may be acknowledged before it's stored anywhere permanent. The flush shows the real cost: fdatasync after a write took a median of 922 µs on the volume and 358 µs locally.
Random reads with 32 in flight never settled on the volume. Second by second they swung between 2,552 and 10,809 IOPS, averaging 7,162, under the limit rather than pinned to it.
Postgres
pgbench at scale 100, about 1.5 GB, with 8 clients for 120 seconds, three runs on each disk, alternating:
| Run | Droplet's disk | Volume |
|---|---|---|
| 1 | 1,324 tps | 908 tps |
| 2 | 1,585 tps | 866 tps |
| 3 | 1,671 tps | 1,070 tps |
The volume was about a third slower, with average latency of 7.5 to 9.2 ms against 4.8 to 6.0 ms. The database fits in memory here, so I think this is mostly the cost of flushing the WAL on every commit, the same fdatasync from the table above. Loading the data and building the indexes took 37.0 seconds on the volume and 22.9 locally.
Moving the volume properly
This is the part a local disk can't do. I ran two writers on the volume on the first Droplet: one appending a numbered line to a file and fsyncing after each, one inserting rows into Postgres. Each logged what it had been told was saved, on the Droplet's own disk. Then I stopped them, stopped Postgres, unmounted, and moved the volume.
The detach took 8.2 seconds from API request to completed, and the attach 7.2. The disk appeared on the second Droplet 3.4 seconds after the attach request, almost four seconds before the API said it was done. Mounting took 0.6 seconds, and Postgres started with "database system was shut down", no recovery needed. From the detach request to Postgres accepting connections on the other Droplet took 16.8 seconds.
Nothing was missing. The second Droplet had 8,424 rows and 5,790 lines, one more of each than the writers had logged as saved. That last write landed, but the writer was killed before it could log it.
Detaching it while Postgres was writing
Next I skipped the unmount. With both writers running and Postgres up, I asked the API to detach the volume. I expected it to refuse, since the volume was mounted and busy. It accepted.
About three seconds after the request (2.7 and 3.3 seconds in two runs), the writes started failing with Input/output error. The kernel logged "device offline error", ext4 shut the filesystem down, and Postgres stopped with:
PANIC: could not fdatasync file "000000010000000100000008": Input/output error
The filesystem still showed as mounted. The API reported the detach complete four to five seconds after the errors began.
Then I attached the volume to the other Droplet. ext4 replayed its journal on mount ("recovery complete"), and Postgres ran crash recovery, which took 0.08 seconds of redo. Postgres had confirmed 16,800 inserts before the error, and all 16,800 rows were there. The file writer had fsynced 11,686 lines, and all 11,686 were there.
In the first forced run, the volume had one line more than the writer had confirmed. The last write was on disk even though its fsync had returned an error, which is the uncomfortable direction for an application that retries on errors.
What I got wrong
I expected the volume to lose every test. It was faster for single 4 KiB writes, and I can only point at the write cache as the likely reason. I didn't prove it.
I expected the API to refuse to detach a mounted volume that was being written to. It doesn't check, and from the Droplet's side the disk simply disappears.
I expected the 32-in-flight random reads to sit on the documented 10,000 IOPS burst like the writes did. They never got there for long.
My Postgres writer broke on the first forced run. The table already existed, CREATE TABLE IF NOT EXISTS printed a notice, and my script read the notice as the answer to the next query and stopped after one row. I fixed it and ran the forced test again in the other direction, which is where the 16,800 rows come from.
And I stopped the writers with pkill -f writer.py, which matched the SSH command that ran it and killed my own session. It's an old trap, and I still walked into it.
Caveats
One Droplet size with a shared CPU, one region, one volume. Each fio test ran once, pgbench three times. The local disk's 4.1 GB/s reads may come partly from a cache on the host, which I can't see from inside the Droplet. The volume limits are higher on dedicated-CPU Droplets, 10,000 IOPS and 450 MB/s before any burst, and I didn't test those. The forced detach ran twice. The whole experiment ran for 70 minutes and cost about 7 cents.
What I'd take from it
- Put data on a volume because it has to outlive the Droplet or move between Droplets, not for speed. On a shared-CPU Droplet the volume is capped at 300 MB/s and 7,500 IOPS, and a burst lasts about a minute.
- Expect single reads around 0.7 ms instead of 0.15, and flushes around 0.9 ms. A database that commits often will feel the flush.
- Moving a volume properly took 17 seconds end to end, most of it the API's own detach and attach.
- Unmount before you detach. DigitalOcean won't stop you, and the Droplet gets I/O errors three seconds later. In my runs nothing that had been confirmed was lost, but the application on that Droplet crashes.


Top comments (0)