Hello, I'm Maneshwar. I'm building git-lrc, a Micro AI code reviewer that runs on every commit. It is free and source-available on Github. Star git-lrc to help devs discover the project. Do give it a try and share your feedback.
A couple of posts back we spent a while inside XOR distance, then used it to build Kademlia, the DHT algorithm that lets a network find anything without a directory.
I promised that algorithm shows up "under BitTorrent, IPFS, Ethereum."
Today we cash that check.
We're taking BitTorrent apart, piece by piece, and Kademlia is going to walk right back in through the side door.
Also, fun fact before we start: a suspicious number of people on Reddit think Bram Cohen, the guy who wrote BitTorrent alone in Python in 2001, is secretly Satoshi Nakamoto.
I'm not saying it's true.
I'm saying that by the end of this post you'll understand why people keep saying it.
The number that should not have been possible
In 2004, a measurement firm called CacheLogic reported that BitTorrent alone was responsible for roughly 35% of all internet traffic.
More than every other peer to peer network combined. More than the entire web.
One protocol. Written by one guy. No company. No datacenter. No servers anywhere with "BitTorrent Inc" on the rack.
That last part is the whole story.
Every "normal" system you've ever worked on scales by throwing money at it: bigger box, more replicas, a CDN in front.
BitTorrent had nobody to throw money at anything, so every hard problem, capacity, trust, scheduling, incentives, discovery, had to get solved inside the protocol itself.
Problem 1: the client-server ceiling has a name
Distributing a file in 2001 meant one server, one uplink, and every download eating a slice of that uplink.
Capacity is fixed and provisioned in advance. Demand is not.
Get linked from somewhere popular and you get what people at the time literally called the Slashdot effect: your server saturates at the exact moment the most people want the thing it's serving.
You're stuck picking a poison: over-provision and burn money on capacity nobody uses most days, or under-provision and eat an outage on the one day it actually mattered.
The inversion: your download is somebody else's upload
Here's BitTorrent's actual trick, and it's almost insultingly simple once you say it out loud: every peer that downloads a file also becomes a source for it.
Not optionally. Not "if you feel like sharing." It's baked into the protocol.
So the file gets sliced into pieces (typically 256KB to a few MB each).
The origin only has to push each byte into the swarm once.
After that, peers replicate to each other.
Total capacity becomes the sum of every peer's uplink, which means:
A flash crowd that would take down a normal server literally makes a BitTorrent swarm faster.
More downloaders, more uploaders, more capacity.
It's the one system I know of where the traffic spike is the fix, not the incident.
Slicing the file also buys you parallelism for free: you're pulling different pieces from different peers concurrently instead of waiting on one slow pipe, and the moment a piece is verified you can start re-serving it.
You become a source within seconds of joining, not after the whole download finishes.
But inverting client-server this hard immediately opens three new problems, and this is where it gets genuinely clever.
Problem 2: how do you trust a stranger's bytes
If any anonymous peer can hand you a piece of the file, what stops someone malicious (or just buggy) from handing you garbage? You can't authenticate the sender, peers are supposed to be interchangeable and anonymous, that's the entire point of decentralization.
So BitTorrent doesn't bother authenticating the sender at all.
It verifies the content.
Every piece has a SHA-1 hash listed in the .torrent file, computed by whoever originally shared the file.
You hash what you received locally and compare:
Nobody needs a reputation, an account, or a "verified peer" checkmark, because the data proves itself.
This idea alone is worth stealing for your own systems: once content can verify itself, you can fetch it from literally anywhere, an untrusted mirror, a sketchy cache, a random peer, doesn't matter.
Problem 3: which piece do you download next
Naive answer: piece 1, then 2, then 3, in order.
Feels obvious. Turns out to be a disaster for three separate reasons.
First, if everyone downloads in the same order, everyone ends up holding the same early pieces and nobody has anything the others need.
There's no trading happening, just a room full of people holding identical playing cards.
Second, requests for the later pieces all pile up on whoever seeded first, because they're the only one who has them.
Congratulations, you've recreated a bottleneck inside the thing you built to avoid bottlenecks.
Third, and this one's fatal: if that original seeder disappears before anyone downloads the later pieces, those pieces now exist in zero copies anywhere.
Permanently incomplete file.
If you've ever seen a torrent stuck at 99% forever, this is exactly what happened to it.
BitTorrent's fix is rarest-first: every client tracks how common each piece is across the peers it knows, and always grabs the rarest one next.
The one exception is the very first piece, which is picked at random, because a brand new peer needs something to trade immediately and rare pieces are by definition slow to get.
The elegant part: every peer is technically optimizing for the swarm's health, not its own, and it turns out over the length of a download those two goals converge into the same thing.
Selfish and altruistic behavior collapse into one strategy. Rare.
There's a neat cleanup mechanism at the very end too, called endgame mode: once only a handful of blocks remain, your client requests them from every peer that has them, simultaneously.
First one back wins, the rest get cancelled. A little duplicate traffic is cheap.
Getting held hostage by one slow peer at 99% is not.
Problem 4: nobody's forcing anyone to upload
Uploading costs bandwidth. On a metered connection it costs actual money.
The rational move for a purely selfish peer is: download everything, upload nothing.
This is called free riding, and BitTorrent's predecessor, Gnutella, got absolutely wrecked by it.
A 2000 study found 70% of Gnutella users shared zero files, while the top 1% of hosts carried close to half of all responses.
Gnutella's "decentralized" network quietly turned into a client-server topology run by volunteers, minus every guarantee an actual client-server setup gives you.
Worst of both worlds.
BitTorrent has zero accounts, zero reputation systems, zero central enforcement of any kind.
So it makes uploading the only path to fast downloads, via a mechanism it calls tit-for-tat, straight out of game theory (cooperate first, then mirror the other side, the strategy that famously won Axelrod's prisoner's dilemma tournaments).
Every client has a handful of upload slots (4 by default).
Every 10 seconds, it re-ranks connected peers by how fast they've recently uploaded to it, and unchokes the top 4 while choking everyone else.
Upload fast, get reciprocated.
Upload nothing, get frozen out, one neighbor at a time.
| Peer rate (KB/s) | Status |
|---|---|
| 950 | unchoked |
| 720 | unchoked |
| 540 | unchoked |
| 310 | unchoked |
| 120 | choked |
| 60 | choked |
| 0 | choked |
There's an obvious hole in this though: a brand new peer has zero pieces, so it can upload nothing, so everyone chokes it, so it can never get a first piece to start trading with.
Pure tit-for-tat can't bootstrap anyone.
The fix is the optimistic unchoke: every 30 seconds, each client hands one upload slot to a completely random peer, rates ignored entirely.
That random slot is how new peers get their first piece and enter the reciprocation loop.
It's also a sneaky exploration mechanism, sometimes that random stranger turns out faster than your current top 4, and earns a permanent slot on merit.
Explore/exploit, straight out of reinforcement learning, quietly running inside a protocol from 2001.
Discovery: killing the last thing anyone was in charge of
Up to this point BitTorrent still leaned on one centralized piece: the tracker, a server whose entire job was handing new peers a list of about 40 addresses to try.
Small job, but a single point of failure both technically (server goes down, swarm can't find new members) and legally (trackers have an operator, an address, a hosting bill, and got taken down constantly).
So in 2005, BitTorrent shipped Mainline DHT, built on Kademlia, the exact algorithm from my last post.
Every node picks a random 160-bit ID.
Every torrent already has a 160-bit ID too, the SHA-1 hash of its metadata, called the infohash.
Kademlia's rule kicks in immediately: whichever nodes have IDs closest to the infohash become responsible for storing that torrent's peer list.
No coordinator assigns this, it just falls out of the XOR distance math we already covered.
The last centralized artifact left standing was the .torrent file itself, still usually downloaded from some website.
A later extension called metadata exchange let peers pass that metadata around chunk by chunk too, verified against the infohash exactly like any other piece.
Which means the only thing you actually need to join a swarm is the 40-character infohash. That's a magnet link.
magnet:?xt=urn:btih:a94f6c3b2e8d1a9f0c7e4b8d3a1f6c9e0b2d7f83
That one string is content addressed all the way down.
It doesn't point at a server, it points at the data, and any node in the world holding it can serve it back to you.
This is exactly why an AI lab can ship a multi-hundred-gigabyte open weights model release by tweeting a 40-character string, Mistral does this routinely.
Watch the whole dependency chain disappear over the years:
timeline
title From a website to a hash
2001 : site + .torrent + tracker
2005 : DHT replaces the tracker
Later : metadata exchange replaces the .torrent file
Now : a 40-character magnet link is all you needCongestion: don't be the reason someone's Zoom call froze
Early BitTorrent had a side effect nobody predicted: it wrecked video calls and games running on the same connection, and it wasn't actually about running out of raw bandwidth.
TCP figures out how much capacity a link has experimentally, by sending faster and faster until packets start dropping.
Problem: home routers buffer packets before they ever drop one.
A BitTorrent client holds dozens of TCP connections open simultaneously, so those buffers fill up completely and stay full, and every packet on the connection, video call included, sits in that queue.
This is called bufferbloat, and the queueing delay it causes can balloon to multiple seconds even though the link technically isn't "full."
The fix, shipped around 2009, is a transport protocol called uTP, paired with a congestion control algorithm called LEDBAT.
Instead of waiting for packet loss, every packet carries a timestamp, and the moment one way queueing delay creeps past roughly 100ms, the sender backs off, before the buffer actually fills and before your call stutters.
rate += gain × (100ms − measured_delay)
The neat trick is that regular TCP is more aggressive by design, so uTP loses on purpose whenever real traffic shows up and just fills whatever's left over.
It behaves like a proper background job instead of a bandwidth bully.
This idea got standardized by the IETF as LEDBAT (RFC 6817), and it now ships inside both Windows Update and Apple's software update pipeline. Your torrent client's manners became an OS feature.
Where this actually runs today
The exact same architecture is quietly running in places you've probably used:
It's not free though
Nothing this clever comes without trade-offs, and BitTorrent has some real ones worth knowing before you go stealing its ideas wholesale.
No privacy. To download, you have to announce yourself, so any participant can log every IP address it sees in the swarm.
Throughput over latency. Rarest-first is the opposite of sequential, which makes the protocol a bad fit for streaming, the piece a viewer needs right now is rarely the piece the swarm is currently prioritizing.
Availability follows demand. Once a flash crowd fades, the file's survival depends entirely on whether volunteers keep seeding it.
Open membership cuts both ways. The DHT accepts any node, including an attacker running thousands of fake identities to bias lookups or poison records, the classic Sybil attack.
Wrapping up
Cohen's original write-up is literally called "Incentives Build Robustness in BitTorrent", and once you've walked through all seven pieces, that title stops sounding like a paper abstract and starts sounding like a thesis statement for the whole thing.
If you want to go straight to the source material, the protocol specs are all public as BEPs: the original spec, Mainline DHT, metadata exchange, and uTP.
A protocol with no company behind it carried a third of the internet at its peak, and its individual pieces are still quietly running inside Windows, inside Uber's deploy pipeline, inside how the next big open weights model reaches your laptop.
Great architectures don't maximize features. They maximize consequences.
Go forth and seed responsibly.
AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs, without telling you. You often find out in production.
git-lrc fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.
Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.
⭐ Star it on GitHub:
HexmosTech
/
git-lrc
Free, Micro AI Code Reviews That Run on Git Commit
| 🇩🇰 Dansk | 🇪🇸 Español | 🇮🇷 Farsi | 🇫🇮 Suomi | 🇯🇵 日本語 | 🇳🇴 Norsk | 🇵🇹 Português | 🇷🇺 Русский | 🇦🇱 Shqip | 🇨🇳 中文 | 🇮🇳 हिन्दी |
git-lrc
Free, Micro AI Code Reviews That Run on Commit
GenAI today is a race car without brakes. It accelerates fast -- you describe something, and large blocks of code appear instantly. But AI agents silently break things: they remove logic, relax constraints, introduce expensive cloud calls, leak credentials, and change behavior -- without telling you. You often find out in production.
git-lrc is your braking system. It hooks into git commit and runs an AI review on every diff before it lands. 60-second setup. Completely free.
In short, git-lrc helps Prevent Outages, Breaches, and Technical Debt Before They Happen
At a glance: 10 risk categories · 100+ failure patterns tracked · every commit…
Top comments (0)