If you moved private files off presigned URLs and started serving them through
your own route, you have probably hit this: images are fine, PDFs are mostly
fine, and video is broken. The player loads, the timeline is dead, and
scrubbing does nothing. There is no error in your logs because as far as your
server is concerned every request was a 200.
The cause is almost always Range, and specifically what you do with a range
you cannot serve. I got this wrong, then I read RFC 9110 properly and measured
what a correct implementation actually answers. Here is the table I wish I had
had.
Why the object store was doing more than you thought
When you redirect to a presigned URL, S3 or R2 is answering the request. That
includes range requests, and it is very good at them. The moment you proxy
instead, that becomes your code, including the parts that have nothing to do
with your application.
It is not optional work. A <video> element sends a range request before it
will let the user scrub at all. A PDF reader asks for the last few bytes first,
because the cross reference table lives at the end of the file. Answer those
wrongly and the file looks corrupt rather than forbidden, which is why this
shows up as a player bug and not as a permissions bug.
The part everyone gets wrong
416 Range Not Satisfiable is what you reach for when a range cannot be served
as asked. It is almost always the wrong answer.
RFC 9110 says an invalid range must be ignored. Not refused, ignored: you
answer 200 with the whole representation, as if the header had not been there.
416 is reserved for a range that parsed cleanly and cannot be satisfied by this
particular object, which is a much narrower case than it sounds.
Here is what that means in practice. Twenty six bytes, one per letter of the
alphabet, so a wrong offset reads as the wrong letters instead of as plausible
binary:
| Range header | status | why |
|---|---|---|
bytes=9-4 |
200 | last position before first position, so the range set is invalid |
bytes=-0 |
200 | a suffix of zero asks for nothing |
bytes=abc |
200 | unparseable |
items=0-4 |
200 | a unit that is not bytes
|
bytes=0-4,10-14 |
200 | multiple ranges, see below |
bytes=99-120 |
416 | parsed cleanly, past the end of a 26 byte object |
Four of the five that look like a 416 are not one. And the one that is carries
something easy to forget:
HTTP/1.1 416 Range Not Satisfiable
Content-Range: bytes */26
That */26 tells the client the actual size, which is the information it needs
to ask a better question. A 416 without it is a dead end.
Multiple ranges deserve their own paragraph
bytes=0-4,10-14 is two questions in one header. You have three options and
only two of them are safe.
You can answer all of them as multipart/byteranges. You can ignore the header
and serve the whole object under a 200. Or you can answer the first one under a
206, which is what a lot of hand written code does because it falls out of the
parser naturally.
That third one is a silent data corruption bug. The client asked two questions.
Nothing in a 206 says only one of them was answered. So it takes your reply,
assumes it covers the first range, and stitches the second range in at an
offset that is now wrong. No error anywhere, and a file that is subtly
different from the one you stored.
Ignoring is the only option that cannot mislead, and it is one line.
The rest of the table, for completeness
The cases that do work, same 26 byte object:
| Range header | status | Content-Range | body |
|---|---|---|---|
| none | 200 | abcdefghijklmnopqrstuvwxyz |
|
bytes=0- |
206 | bytes 0-25/26 |
abcdefghijklmnopqrstuvwxyz |
bytes=0-4 |
206 | bytes 0-4/26 |
abcde |
bytes=-4 |
206 | bytes 22-25/26 |
wxyz |
bytes=20-999 |
206 | bytes 20-25/26 |
uvwxyz |
Two things in there are worth pointing out.
bytes=-4 is a suffix, not a negative offset. It means the last four bytes.
If you parse it as "start at -4" you will either throw or serve from the
beginning, and the PDF reader that sent it will decide your file is damaged.
bytes=20-999 is not an error. A last position past the end of the object is
satisfiable per the spec, and the response covers to the end. Returning 416
here is the second most common mistake after the ones in the first table.
Accept-Ranges goes on every response, including the 200
This is the one I had backwards for a while.
Accept-Ranges: bytes is how a client learns it is allowed to seek. So it has
to be on the plain unranged 200, which is the response the client reads before
it has ever sent a range. Sending it only on 206 responses tells the client
something it already knew, at the only moment it cannot act on it.
A parser that gets the above right
About thirty lines. Returns null for anything that must be ignored, and
null never means error:
type RangeSpec = { start: number; end?: number } | { suffix: number };
function parseRange(raw: string | undefined): RangeSpec | null {
if (typeof raw !== 'string') return null;
const m = /^bytes=(.*)$/.exec(raw.trim());
if (!m) return null; // another unit, or not a range
const specs = m[1].split(',');
if (specs.length !== 1) return null; // multiple ranges: ignore, serve whole
const spec = specs[0].trim();
const suffix = /^-(\d+)$/.exec(spec);
if (suffix) {
const n = Number(suffix[1]);
if (!Number.isSafeInteger(n) || n === 0) return null;
return { suffix: n }; // the LAST n bytes
}
const explicit = /^(\d+)-(\d*)$/.exec(spec);
if (!explicit) return null;
const start = Number(explicit[1]);
if (!Number.isSafeInteger(start)) return null;
if (explicit[2] === '') return { start };
const end = Number(explicit[2]);
if (!Number.isSafeInteger(end)) return { start }; // absurd last-pos means "to the end"
if (end < start) return null; // invalid, so ignored, NOT a 416
return { start, end };
}
Then, when you resolve it against the real object size:
// `null` from the parser: no range was asked for, or it must be ignored.
if (!range) return respond(200, whole, { 'accept-ranges': 'bytes' });
const start = 'suffix' in range
? Math.max(0, size - range.suffix)
: range.start;
// The only 416: it parsed, and this object cannot satisfy it.
if (start >= size) {
return respond(416, null, {
'content-range': `bytes */${size}`,
'accept-ranges': 'bytes',
});
}
const end = 'suffix' in range ? size - 1 : Math.min(range.end ?? size - 1, size - 1);
return respond(206, slice(start, end), {
'content-range': `bytes ${start}-${end}/${size}`,
'accept-ranges': 'bytes',
});
One thing to be careful about in the 206 path: derive the status from whether
you are sending a partial body, do not let the caller pass it in. A partial
body sent under a 200 is undetectable by every HTTP client on earth. It stores,
caches, hashes and hands on a truncated file without a single error anywhere.
The status code is the only thing that makes it a range.
How to check your own route in five minutes
Put a file of known length behind your route and run these. You do not need a
player to find out whether seeking will work:
curl -s -o /dev/null -D - -H 'Range: bytes=0-4' "$URL" # expect 206
curl -s -o /dev/null -D - -H 'Range: bytes=-4' "$URL" # expect 206, last 4 bytes
curl -s -o /dev/null -D - -H 'Range: bytes=9-4' "$URL" # expect 200, whole file
curl -s -o /dev/null -D - -H 'Range: bytes=0-4,9-12' "$URL" # expect 200, whole file
curl -s -o /dev/null -D - -H 'Range: bytes=999999-' "$URL" # expect 416 + Content-Range
curl -s -o /dev/null -D - "$URL" # expect Accept-Ranges: bytes
If the third or fourth one comes back 416, that is the bug your video player is
reacting to. If the last one has no Accept-Ranges, the player will not even
try to seek.
Disclosure
I work on an open source library that serves private files, so I wrote this
parser and then wrote the tests that pin every row of those tables. The tables
above are the output of a script, not my recollection. The script is in the
repo, so you can point it at your own implementation and see what yours does.
Top comments (0)