You hit play on a video that's still downloading, and the player immediately wants bytes you don't have. First the start of the file. Then, quite often, the end, because that's where an MP4 keeps its index. Then wherever you drag the seek bar. BitTorrent doesn't care about any of that: it grabs pieces in whatever order the swarm hands them out, rarest first.
Getting those two to cooperate turned out to be most of the work.
I needed this for Torsher, the self-hosted media center I'm building, and I gave myself one rule: on a normal home connection, playback has to start in under 15 seconds. Below is what I tried, what failed, and the numbers.
Every test used Blender's open movies (Sintel, Big Buck Bunny, Tears of Steel). They're free to share, which makes them perfect for this.
How I measured
I didn't want to click around a browser all evening, so a small script pretends to be the player:
- start: grab the first 2 MB and the last 1 MB of the file at the same time, and time how long until both are in;
- seek: grab 2 MB from the middle of the file, then 2 MB from 85% in.
Each run starts from scratch: torrent removed, files deleted. Two rounds per engine, swapping the order between them. Fetching the torrent's metadata isn't counted.
First try: a ready-made library
I started with an existing Rust library that already had streaming built in. Median start: 41 seconds. Worse, it was random. The same file started in 2 seconds on one run and 57 on the next. Seeking into a part that wasn't downloaded yet took 43 seconds.
So I read its source. The stream window, 32 MB, went out to peers all at once, and the piece sitting right under the playback position was no more urgent than anything else in that window. If a slow peer happened to grab that piece, the player just sat there until the request got handed to someone else, and that only kicked in once the request was ten times slower than average.
What fixed it: a responsive reader
I swapped it for a small Go service on top of anacrolix/torrent. The library handles the hard stuff (peers, DHT, picking pieces), and my code is just the HTTP API, the stream, file selection and storage.
The part that matters is the library's responsive reader. It gives urgent priority to the pieces at the read position, so whatever the player is waiting for right now jumps the queue. Each HTTP request gets its own reader:
var streamReadahead int64 = 16 << 20
func newStreamReader(ctx context.Context, file *torrent.File) torrent.Reader {
reader := file.NewReader()
reader.SetContext(ctx) // the player disconnects, the read is cancelled
reader.SetResponsive()
reader.SetReadahead(streamReadahead)
return reader
}
For Range requests I let http.ServeContent do the work: 206, 416, Content-Range, Content-Length, all handled. One thing to watch out for: set Content-Type yourself before you call it.
w.Header().Set("Content-Type", contentType)
http.ServeContent(w, r, name, time.Time{}, reader)
If you don't, ServeContent reads the beginning of the file to guess the type. With a torrent, that means it pulls the first pieces even when the player asked for the middle.
Download a window, not the whole file
In a torrent client, picking a file usually means "download all of it". For watching, that's wrong in two ways. The rest of the file fights with the piece under the playback position for the same peers. And a 20 GB file you watched for one minute ends up sitting on your disk.
So while you watch, only a window ahead of the position gets downloaded: roughly three minutes of video, worked out from the file's average bitrate and kept between 64 MB and 1 GiB. The whole file only comes down if you explicitly hit "download".
Marking a full 1 GiB window as wanted in one go would bring back the original problem, so the window goes out in 64 MB batches (at least four pieces), closest to the position first. The batch gets recalculated whenever the piece under the position changes, and once a second on top of that, because a paused player doesn't move but the window should keep filling.
Close the stream, and the pieces outside any open window drop back to "not needed". Downloading just stops.
One thing made this a lot easier than I expected. In anacrolix, a piece's priority is the highest of three: the file's, the readers', and the piece's own. So a viewing window never clashes with a file someone chose to download in full, and the other way around.
What made it worse: prewarming
This one looked obvious. As soon as a torrent is added, read the start and the end of the file, so they're already there when the player asks. I measured it, and the median start went from 8.3 to 20.3 seconds. The prewarm requests were fighting the player's own requests for the same peers. I took it out, and it stays out until a benchmark says otherwise.
Results
| Start, median (range) | Seek, median (range) | |
|---|---|---|
| Rust library | 41 s (2–59) | 43 s (6–70) |
| Go + responsive reader | 8.3 s (3.4–19.5) | 3.8 s (1.3–6.9) |
| Go + prewarm | 20.3 s (5.4–33.0) | 10.9 s (4.2–18.2) |
For reference, a popular streaming server built on the same library did 6.0 s to start and 3.0 s to seek on the same files. So I'm still a couple of seconds behind. What bugs me more is how much the start time jumps around. My guess is that other pieces compete with the first ones early on, and that's what I'm measuring next.
Usual caveats: one machine, one home connection, one evening, two runs per file. Swarms change over time.
Two things that bit me
Library callbacks run under its lock. anacrolix calls some callbacks while it holds the client lock. Touch the client from inside one, and the engine deadlocks.
A panic in an HTTP handler can leave that lock held. net/http recovers panics in handlers, so the process keeps running, /health still says OK, and everything that needs the lock hangs forever. Now any panic in a route takes the whole process down, a watchdog checks the lock every 5 seconds and exits if it's been stuck for a minute, and the parent server restarts the engine.
Where this lives
This is the engine inside Torsher, an open-source media center that runs in one Docker container and starts from the TV. The engine is in engine/, the window logic in engine/window.go.
If you've beaten the start-time spread some other way, I'd really like to hear how.
Top comments (0)