DEV Community

Remdore
Remdore

Posted on

Uploading straight from the browser to DigitalOcean Spaces

Spaces Gallery is a small Flask app I maintain that uploads images to DigitalOcean Spaces and shows them in a gallery. Right now every upload goes through the app. The browser sends the file to Flask, Flask checks it, makes a thumbnail and writes both to Spaces.

The usual advice is to skip that middle step. The app hands the browser a presigned POST, a short-lived signed form, and the browser sends the file straight to Spaces. I built that into the gallery and measured what it changes, using the app's own code and deployment setup.

The setup

The app ran under gunicorn exactly as its Procfile starts it, which means one sync worker, on a 1 GB Droplet in Frankfurt. The Spaces bucket was in Frankfurt too. My laptop played the browser, uploading noise PNGs of 1, 4 and 8 MB that can't be compressed. On the Droplet, a sampler recorded the app's CPU time, memory and network traffic four times a second.

The direct path has three steps:

  1. The page asks the app for a presigned POST. The policy pins the object key, the content type and the ACL, and caps the size at 8 MB.
  2. The browser posts the file to Spaces.
  3. The page tells the app it's done. The app downloads the file, checks that it really is an image, makes the thumbnail and sets the cache header.

That last step exists because a policy can limit a file's size and declared type, but it can't look inside it.

Spaces enforces the policy

Before any timing, I checked that Spaces actually holds the line:

  • A file sent with Content-Type: text/html against a policy that says image/png got a 403.
  • A 9 MB file against the 8 MB limit got a 400.
  • A five-byte file called fake.png was accepted, since the policy can't tell. The app's check then found it wasn't an image and deleted it.

So the size and type limits move from the app to Spaces, and the "is this really an image" check stays in the app.

Cache-Control is checked and then thrown away

The gallery gives every upload Cache-Control: public, max-age=31536000, immutable, so the CDN and browsers keep images for a year. I put that header in the presigned policy like the others. The upload worked, and the stored object had no Cache-Control at all.

Spaces does read the condition. Leaving the field out or sending a different value got a 403. But with the right value, the object was stored without the header. The lowercase field name behaved the same. The thumbnail, written by the app through the normal API, had the header as expected.

The fix is a copy of the object onto itself, with the metadata replaced, in the finish step. That's one request, and no file data passes through the app for it. After it, the public URL returned the one-year cache header.

For one upload, it wasn't faster

I expected the direct path to win easily. Over five runs of each size, the medians were:

File Through the app Direct Direct, sending to Spaces only
1 MB 0.73 s 1.15 s 0.08 s
4 MB 0.86 s 0.87 s 0.16 s
8 MB 1.04 s 1.05 s 0.26 s

The transfer to Spaces was quick. My connection is fast, and 8 MB took a quarter of a second. But the direct path adds two calls to the app around it. For the 8 MB file, the presign took 0.21 s and the finish step 0.57 s. Those two together cost as much as the transfer saved, and for small files they cost more.

The app still touched every byte

The sampler first said that the proxied upload received 8 MB and sent almost nothing to Spaces. That couldn't be right. The traffic to Spaces in the same region was going over the Droplet's second network interface, eth1, which I hadn't been counting. With both interfaces counted, an 8 MB upload looked like this:

  • Through the app: 8.3 MB in from the client and 8.4 MB out to Spaces, the file plus its thumbnail.
  • Direct: nothing from the client, but 8.0 MB in from Spaces during the finish step, which downloads the file to check it, and 0.4 MB out for the thumbnail.

CPU time was about the same either way, 0.53 to 0.6 seconds per upload, almost all of it Pillow decoding the image and making the thumbnail. The direct path moved the upload off the app, but the finish step brings the file straight back.

Ten at once

With ten 8 MB uploads started together at full speed, the single worker handled them one at a time either way. Through the app, they finished about 0.9 seconds apart, the last after 9.0 seconds. A health check sent during the batch waited 8.8 seconds for an answer.

Direct, all ten finished in 7.6 seconds and the health check waited up to 4.3. Better, because the transfers themselves ran in parallel, but the finish steps still queued behind each other on the one worker.

Slow clients are where it matters

A laptop on a fast connection is the best case for proxying. So I throttled three clients to 1 MB per second and had each upload 8 MB, while a health check ran every half second.

Every upload took between 8.6 and 10.4 seconds either way, since the connection set the pace. The difference was everyone else. Through the app, one health check took 10.1 seconds, the whole length of the batch. The worker was busy reading slow request bodies and nothing else got in. Direct, the slowest of 25 health checks took 0.87 seconds. The app only saw the quick presign and finish calls, and the slow part happened between the browser and Spaces.

That's the real argument for direct uploads with a single small worker. They don't make one upload faster, they stop slow uploads from freezing the app.

Without CORS, uploads fail and succeed at the same time

A browser will only let the page post to Spaces if the bucket allows the app's origin. I tried it first without a CORS rule. The page's fetch threw "Failed to fetch", so it showed an error and never called the finish step.

The file was in the bucket anyway. A plain form POST doesn't need the browser to ask permission first. It sends the request and then refuses to let the page read the response. So Spaces stored the upload, and the page believed it had failed. Two test images ended up in the gallery with no thumbnail, no cache header and no image check.

Adding a CORS rule for the app's origin, allowing POST, fixed the browser side. I also changed the page so that a network error still goes on to the finish step, which either finds the file and completes it or reports it missing. With the rule removed again, an upload from Chrome then finished normally, thumbnail and all.

What I got wrong

I expected the direct path to be faster per upload. On a fast connection it wasn't, and for a 1 MB file it was slower, because two extra calls to the app cost more than the transfer.

My first traffic numbers were wrong, because the sampler only counted eth0. They made it look as though the proxied upload never reached Spaces.

I assumed a Cache-Control field in the policy would set the header, since Spaces checks it. It doesn't, and I'd have shipped uploads without their year-long cache header if I hadn't looked at the stored object.

And I took "Failed to fetch" to mean the upload had failed. It meant the browser wouldn't show the page the answer.

Caveats

This is one Droplet size, with the single worker the Procfile gives, and one region, with my laptop as the client. The slow clients were throttled in software. The single-upload timings are five runs each, and the concurrent and slow tests ran once. I ran the app on a Droplet so I could watch its processes. On App Platform, the load balancer in front of the app might buffer request bodies, which would change the slow-client result, and I didn't test that. The bucket and the Droplet existed for about 15 minutes, which cost well under a cent.

What I'd take from it

  • Use presigned POSTs to protect the app from slow uploads, not to make uploads faster. On a fast connection they were no quicker.
  • Put the size and type limits in the policy, because Spaces enforces them, and keep a server-side check for what a policy can't see.
  • Don't rely on the policy for Cache-Control. Set it afterwards with a copy onto the same key.
  • Add the CORS rule before switching the page over, and treat a network error from the upload as "unknown", not "failed".
  • If the server still downloads every file to check it, the bytes come back anyway. The thumbnail step is the next thing to move.

Top comments (0)