DEV Community: Sospeter Kinyanjui

How BitTorrent Actually Works (a p2p network protocol)

Sospeter Kinyanjui — Wed, 24 Jun 2026 08:38:37 +0000

You've probably used it. You almost certainly don't know what happened under the hood.

The Problem With One Server

Imagine a band releases their debut album for free. By morning, a million people are trying to download it. Their server buckles, downloads crawl, and the hosting bill becomes a catastrophe.

This is what the internet looked like before BitTorrent. If you wanted to distribute a large file, you needed powerful, expensive servers. The bigger your audience, the more it cost you. Popularity was a punishment.

BitTorrent looked at this problem and asked a different question entirely.

What If the Downloaders Were Also the Distributors?

That one idea is the whole thing.

When you download a file using BitTorrent, you simultaneously upload pieces of it to others who need them. The more people downloading, the faster everyone downloads. Popularity becomes a feature, not a problem.

The file is split into thousands of small pieces pulled from different people at the same time, then reassembled into the complete file. This is a peer-to-peer network — no single server in charge, everyone both a customer and a supplier.

Step 1: The Torrent File (or Magnet Link)

A .torrent file is not the content itself — it's a map describing the content: the file name, its size, how many pieces it's split into, a checksum fingerprint for each piece, and where to find people who have it. It doesn't contain the treasure. It tells you where to look and how to verify you found the real thing.

Magnet links do the same job with just a string of text — no file needed at all.

Step 2: The Tracker

Your client contacts a tracker — a server that keeps a list of everyone downloading or sharing a torrent. The tracker doesn't have the file. It's a matchmaker. You give it your address, it gives you a list of peers. It also registers you so future downloaders can find you.

Step 3: Talking to Peers

Your client connects to peers and they exchange piece lists. Your client is smart about this — it deliberately requests the rarest pieces first, ensuring scarce pieces spread through the network before someone goes offline.

You talk to many peers simultaneously, constantly dropping slow ones and finding faster ones. The full collection of everyone sharing a torrent is called the swarm.

Step 4: Pieces and Checksums

As pieces arrive, each one is verified against its checksum. The client runs a mathematical function on the received data and compares the result to what the torrent file says it should be. A match means the piece is genuine. A mismatch means it's silently discarded and requested again. You cannot be fooled into saving corrupted data.

Step 5: Seeding

Once your download completes, keeping your client open means you're now uploading pieces to others — this is called seeding. The system depends on it. If everyone stopped the moment they finished, the swarm would collapse.

BitTorrent even has a tit-for-tat mechanism: peers who upload more get faster downloads in return. Generosity is rewarded. Freeloading is punished. It's elegant social engineering baked directly into the protocol.

The Beautiful Part: What Happens Without a Tracker?

Trackers are a single point of failure. If one goes down, new peers can't find each other.

So the question became: what if we didn't need a tracker at all?

The answer is a DHT — Distributed Hash Table. Instead of one central directory, knowledge of who has what is spread across thousands of regular users' computers. Your client doesn't ask one server — it asks the network itself, hopping from node to node, each one pointing slightly closer to the answer.

The result: no single entity can be shut down, taken offline, or pressured. The network is the server. It lives as long as people participate. This is not just engineering — it's a philosophy. Information can be free not because someone decided it should be, but because the architecture makes it structurally true.

Why This Matters Beyond Piracy

BitTorrent's ideas are everywhere: Linux distributions ship via torrent, IPFS is built on the same principles, blockchain networks use peer-to-peer distribution, and game studios use it for update delivery. The insight that receivers can be distributors turned out to be one of the most durable ideas in networking.

Decentralization Didn't Stop at File Sharing

BitTorrent proved something the industry hadn't accepted: you don't need a server to build a reliable system. You need a protocol smart enough to turn participants into infrastructure.

Today we call this serverless computing — functions distributed across hundreds of edge locations with no single machine in charge. Amazon Lambda, Cloudflare Workers, Vercel Edge Functions are, at their philosophical core, children of the same thinking. The server is no longer a place. It's a role, and any node in the network can play it. Systems become faster, cheaper, and harder to destroy — because there is no single throat to choke.

Blockchain: BitTorrent's Philosophical Child

If BitTorrent decentralized file distribution, blockchain decentralized something far more ambitious — truth itself.

The connection is architectural, not metaphorical. A blockchain is a ledger held simultaneously by thousands of computers. When a transaction happens, nodes reach agreement through consensus — a mathematical vote — before writing it permanently. No single node can alter the record because it lives everywhere at once.

The integrity mechanism mirrors BitTorrent's checksums directly. Every block contains a cryptographic fingerprint of the block before it. Change anything and every fingerprint after it breaks — the tampering becomes mathematically obvious to every node in the network.

Bitcoin, Ethereum, every chain that followed — all fundamentally peer-to-peer networks with a shared ledger instead of a shared file. Satoshi Nakamoto described Bitcoin explicitly as a peer-to-peer electronic cash system. The lineage from BitTorrent is direct.

The Future Belongs to the Network

Here's the uncomfortable truth: you don't control your data. Your messages, photos, and identity live on servers owned by corporations who can suspend or sell your presence at any moment. Centralization was the price of convenience when most people couldn't run their own server.

That tradeoff is becoming harder to justify.

The next generation of the web is an attempt to rebuild core infrastructure on peer-to-peer principles — cryptographic identities no company controls, files stored on networks where you hold the keys, transactions validated by the network rather than a bank. This is what decentralization actually promises: the right to participate without asking permission from an owner, because there is no owner.

BitTorrent was the first large-scale proof this was possible — not theoretically, but in practice, at scale, used by millions of ordinary people who had no idea they were participating in one of the most resilient distributed systems ever built. Everything that followed is the same idea applied to larger domains.

The architecture is the guarantee. Not a promise, not a policy — the structure of the network itself.

We are still very early.

I'm Building One

emrent started with a simple obsession: I wanted to understand BitTorrent from the inside. Not use it — understand it. The only way I know how to do that is to build it. It is not a production client. It will not replace qBittorrent.

What it is — is PoC. Proof that the protocol is understandable, that the concepts in this article are not magic.

Architecturally, emrent is organized as a Cargo workspace — a collection of focused crates each responsible for one layer of the protocol: tracker communication over both HTTP and UDP, the peer handshake and wire protocol, piece selection and download logic, checksum verification, and finally disk I/O. On top of that sits a desktop interface built with Tauri and React, making it a fully usable application, not just a library.

If you're curious about how any of this translates into code, take a look at the repo: github.com/emrent-rs/emrent

Thanks for reading. If something was unclear or you want to go deeper on any section, find me on linkedin or my github account: github or by email: sospeterkinyanjui57@gmail.com

until next time, peace, focus

Utilizing Oceans to Power Data Centers

Sospeter Kinyanjui — Tue, 12 May 2026 15:14:14 +0000

There are some global problems you don't want to think about. When it comes to processing artificial intelligence computing tasks, you don't wanna think about it too. Such energy consumption are truly demanding. I want us to share with you about Panthalassa, a data center that generates it's energy from ocean waves. As you'd expect, it's architecture is quite different from traditional, on land structures. This is it:

and the part that's immersed in water looks like:

I want us to look at why it's promising, how it impacts AI computing tasks, how it's a clean alternative to other non-environmental friendly ways to curb these demands.

My inspiration for this article came when I listened to Panthalassa's CEO and co-founder, Sheldon Coulson. Here are his inspirations:

Everyone wants to find the things that make it so that people have schools, and roads, and hospitals, food. All of the things that humans need, are all driven by energy. The question is can you find things that are much kinder to the planet or just try to find the things that work for humanity. You look out of the ocean and you see these millions of square miles and you start to calculate how much energy you can go and get and find it's a lot more than humans need.

But, think about it. We don't need energy in the middle of the ocean. So what does the machine do right or different.

"As it goes up and down with the waves, it causes water that's in that tube to be forced up into the top. Once it's in the ball, the water is forced through a turbine. The turbine spins, and that's what makes the electricity."

The computation happens on the spot as the energy is being generated. And then sends the output through the satellites. This eliminates the need to plat cooling systems because obviously the ocean is super cool. Again, it's significantly cheaper to construct compared to data centers on land.

Ocean-3 is planned to be deployed this year. But it also arises the re-liabilities issues for several reasons: the waves can turn to storms and destroy it, salty water is corrosive to the machine or even gradual accumulation of microorganisms on it's surface. There are measures that have been taken in place to mitigate the effects.

Myself I think it's a good move. Mainly because it's a way of generating unlimited, clean and locally sourced power directly from ocean waves to run computing AI at sea, by-passing land-based power shortages, cooling constraints and infrastructure limitations.

You can find out more about Panthalassa and it's Ocean-2 on their website: https://panthalassa.com/ and watch this video for more insights: https://youtu.be/Q7Pmgq2JKbI?si=Lrj6bUFGpzMhWqDc.

Until next time, peace, focus.
Happy coding.

Understanding TCP: The Backbone of Reliable Network Communication

Sospeter Kinyanjui — Fri, 20 Mar 2026 09:03:41 +0000

We’re in 2026 and just how fast and convenient it is to share computer resources and information across computer networks often goes unnoticed. It’s what inspired the first wide area network, ARPANET. The desire to share expensive and limited computing resources. The idea of allowing multiple users to access there resources remotely, sharing the processing power and data storage was revolutionary. I want us to look at the one thing that made all these possible, the TCP/IP protocol suite.

Definition of Terms

Network: two or more computers connected for the purpose of communication and sharing resources.
Protocol: agreed upon set of rules that make communication between two computers on the same network possible.
TCP: a connection-oriented transport layer protocol that ensures reliable, ordered, and error-checked delivery of data packets between applications on an IP network.
IP: the foundational set of rules governing how data packets are addressed, routed, and transmitted across networks, enabling global communication.
Internet Protocol Suite: is a framework for organizing the communication protocols used in the Internet and similar computer networks according to functional criteria.

IP establishes the ground on how information is carried across the network. TCP wraps around IP traffic to ensure the data transmitted reached the destination, intact and in the correct order. As a developer it should come without saying that every application must convey this behavior. In the coming paragraphs I want us to analyze the format of a TCP packet, so we can see how it has become reliable to send data, how the data is divided into packets and the packets reach still ordered and reassembled correctly, and finally how to cope with a situation where some bits of data inside a packets disappeared along the way.

Theoretically a single TCP packet can carry, up to 65KB of data. But in practice due to fragmentation the Maximum Transmission Unit is usually 1.5KB. Just an overview, each packet contains, source port and address, sequence and acknowledgement number and data to be transmitted.
Write on Medium

In order to a single connection to be established there’s what we call a three way handshake. Let’s face it, two computers, a client and a server want to communicate. Let’s say the client by the help of a browser wants to request some web files. The protocol says, the client initializes the connection. A client sends a synchronization TCP packet with the SYN reserved word set. The server responds with a TCP packet with the ACK reserved word set. Eventually the client responds with SYN/ACK flags set. and the sequence numbers too. Think of the handshake as an introduction to a meeting:

-> Client: “Hi, I’m here.”

-> Server: “Hi, I’m here, nice to meet you.”

-> Client: “Nice to meet you too!”

Now that we’re sure the connection is stable, the server starts sending HTML, CSS, JS, image, videos, music, PDF, and other formatted data.

In order to ensure all the data reached the destination IP, the sequence number (a unique number) is assigned to each bit of data. This way both sides keep track of what data has been sent and received. If anything goes wrong the packet is re-transmitted while the others wait in the queue for assembling. If the receiver sees a gap between two sequence numbers for two packets, for example 134 and 136, the sender’s timer for 135 expires so it re-sends.

Before TCP came along, early networks were chaotic: data could arrive out of order, get lost, or be duplicated, leaving users with garbled or incomplete information. There was no built-in way to detect errors, re-transmit missing pieces, or ensure that messages flowed in a steady, reliable stream. TCP changed all of that by introducing sequence numbers, acknowledgments, and retransmissions, which smooth out the bumps in the network. It reorders packets, fills in missing data automatically, and ensures that information reaches its destination accurately and in the right order, making network communication predictable and dependable instead of a messy guessing game.

My team and I at https://www.zone01kisumu.ke/ had been working on a command line Chat Application Server. Sort of like netcat, apart from it neatly organizes chatters into chat rooms, persists chats for later retrieval, chatters can switch rooms and change their names, see whose in the room and much more. You can check it on github: https://github.com/sospeter-57/net-cat-server. When creating the server we used the go’s standard package, net. We created a listener through net.Listener. We also specified that we wanted our server to strictly listen for incoming TCP connections.

listener, err := net.Listen("tcp", ":"+configs.Port)
 if err != nil {
  return nil, err
 }

It worked because we were confident that every bit of message that the chatters received was in the correct order and complete. Even with interpolation of other language’s features such as goroutines. But let’s be real, the normal user will never care about the technicals of how he/she receives the message. What matters is probably the most convenient way.

Speaking of such, TCP might be irreplaceable in today’s world, but it has it’s downsides too. To begin with, it’s slow but sure, I know it sounds good, but the bad side still exists, it’s slow. Another protocol designed specifically to solve this was created, User Datagram Protocol. But UDP doesn’t promise you reliability. But wait, we also have QUIC does what TCP and UDP does all at a go. But more on those ones in the coming articles. All in all, there’s a good reason why TCP over IP has has dominated over 75% of the entire traffic on the internet. Right now Transmission Control Protocol is 53 years old, 53 years from how will you and me still be using my beloved TCP. Let me picture that, I’ll be in my 70s or 80s, probably implanted a Neuralink chip now receiving data packets using a century old network protocol. What do you think? You know what? I’d like very much to be part of that future. Sure, it might not be TCP, the the love for keeping everyone connected, and securely, with unlimited access to information, is not stopping soon. In the last article I promised TCP, here is it. Next well look at IP on it’s own.

You can find me on:
Github: https://github.com/Sospeter-M-Kinyanjui
LinkedIn: https://www.linkedin.com/in/sospeter-kinyanjui-6a091b2a5/

Until next time, peace, focus.

Happy coding.

HTTP: the web's protocol

Sospeter Kinyanjui — Mon, 23 Feb 2026 23:55:08 +0000

HTTP (Hypertext Transfer Protocol) is an application-layer protocol for distributed, collaborative, hypermedia information systems. It is the foundational, stateless protocol used to transfer data (such as HTML documents, images, and videos) on the World Wide Web.

How HTTP Works
HTTP is a request-response protocol:

Client Request - A browser or app sends a request to a server, specifying a method(GET, POST, etc.) and a source path.
Server Response - The server processes the request and returns a response containing a status code, headers, and an optical body. Requests and responses are structured as messages, making them human-readable and extensible.

HTTP Methods
GET – Retrieve a resource. This method is safe (does not modify the resource) and idempotent (repeating it has the same effect). Typical use: loading a web page or fetching data from an API.
POST – Submit data to a server to create a new resource or trigger processing. Not idempotent. Example: submitting a form or posting a comment.
PUT – Replace a resource entirely with the request payload. Idempotent. Example: updating a file on a server.
PATCH – Apply a partial modification to a resource. Not necessarily idempotent. Example: updating a single field in a database record.
DELETE – Remove a resource. Idempotent. Example: deleting a user account.
HEAD – Same as GET but without the body. Used to fetch metadata (headers) about a resource, like checking if it exists or its size.
OPTIONS – Ask the server which methods are supported on a resource. Commonly used in CORS preflight requests in web applications.

Headers and Metadata
HTTP messages are more than just the request or response body — they carry headers, which provide metadata about the message or instructions on how to handle it. Headers make HTTP flexible and extensible.

Common Request Headers
Host – Specifies the domain name of the server. Required in HTTP/1.1 to handle virtual hosting.
User-Agent – Identifies the client software (browser, app, or bot). Useful for analytics or server behavior adjustments.
Accept – Indicates the content types the client can process, e.g., text/html or application/json.
Authorization – Carries credentials or tokens for authentication.

Cookie – Sends previously set cookies to the server.

Common Response Headers
Content-Type – The MIME type of the response body, e.g., text/html, application/json.
Content-Length – The size of the response body in bytes.
Set-Cookie – Instructs the client to store a cookie for future requests.
Cache-Control – Defines caching policies (e.g., no-cache, max-age=3600).
Location – Used in redirects to indicate the new URL.

Special Header Categories
Caching Headers – ETag, Last-Modified, Expires. Help clients avoid downloading unchanged resources.
Security Headers – Strict-Transport-Security, Content-Security-Policy, X-Frame-Options. Protect against attacks like XSS and clickjacking.
Custom / Application Headers – Many APIs define headers like X-Request-ID for tracing requests or X-Rate-Limit for throttling.

Why Headers Matter
Headers are what allow HTTP to be protocol-agnostic and extensible. Without headers:
The client and server wouldn’t know content type, length, or caching policies.
Authentication and session management would be impossible.
Advanced features like partial content delivery, compression, and CORS would not work.
In short, headers transform HTTP from a simple message-passing protocol into a fully-featured web communication framework.

HTTP Versions
HTTP has evolved over time:
HTTP/1.1 – Persistent connections, chunked transfers.
HTTP/2 – Multiplexed streams, binary framing.
HTTP/3 – Runs over QUIC for faster, more reliable connections.
Despite differences, all versions follow the same basic request-response model.

Summary
HTTP is the language that makes the web possible. It defines how a client asks for something and how a server responds. Every time you open a website, load an image, submit a form, or use an API, HTTP is working underneath. Through its methods, it defines what action you want to perform. Through its headers, it carries instructions and metadata about how that data should be handled. Through its evolving versions, it improves speed, efficiency, and security — without changing its fundamental model. At its core, HTTP is simple: request and response.
But from that simplicity, the entire modern web is built.

Now you know what happens behind those backend systems. I'm going to be releasing articles for other network protocols. It's going to be a series one per week. Next week we'll look at TCP. Follow me on: LinkedIn and Github.
Until next time, peace, focus.

Leveraging io_uring for performant asynchronous linux applications.

Sospeter Kinyanjui — Sun, 08 Feb 2026 23:52:24 +0000

author: Sospeter Kinyanjui

intro

For the longest time, in linux there just existed epoll, a I/O notification facility that enables applications to make read/write system calls to the linux kernel. epoll first emerged in the Linux Kernel 2.5.44 back in 2002 which later gained mainstream adoption at 2.6 on 2003. epoll uses the readiness-model to make system calls to the kernel, epoll_create, epoll_ctl and epoll_wait. The kernel notifies applications when resources are available in order for apps to submit tasks. This makes it O(1) complexity, which means it's speed stays the same whether you're watching 10 connections or 10,000 connections. And since making system calls are the only way for applications to communicate with the kernel to access computer resources, we might consider this behavior inefficient. And that presents the syscall tax problem, making expensive context switching system calls for every connections while switching between user mode and kernel mode. Not until 2019 was io_uring Linux Kernel System call for I/O asynchronous operations came.

First let's define asynchronous execution: it's the ability of an application to start a long-running task and continue responsive execution rather than waiting for that task to finish. This makes better utilization of computer resources. Considering epoll was event-driven, mimicking such behavior in programs can be considered an illusion. io_uring aimed at making less system calls by batching multiple read/write requests then making one. as a result, multiple read and write operations do no have to wait for each other.

definition and implementation

Technically, io_uring exposes three calls: io_uring_setup(2), io_uring_enter(2) and io_uring_register(2).

A call to io_uring_setup initializes the io_uring context by creating the submission queue (SQ) and completion queue (CQ), returning a file descriptor to the application. It takes parameters to configure it's primary data structures, rings, their sizes, the head, tail, ring_mask and ring_entries. As you can imagine a ring follows the FIFO thing. The first task on the sequence queue will be the first to be passed to the Completion Queue Entry. The CQEs are also structured similarly. A call to io_uring_setup leads to a creation of a shared memory with the size specified comprising of SEQs and CQEs sections. For SEQs the user/application space only have writing permissios while kernel permissions are only granted for reading. For CQEs the kernel only has writing permissions while the user/application has just reading permissions. Apparently to keep things performant, they use the Single Producer / Single Consumer logic.

A call to io_uring_enter is the engine starter for the whole operation. It tells the kernel: "I've put some SQEs in the ring; go process them." This is how it looks like in the C structure:

#include <linux/io_uring.h>
    int io_uring_enter(unsigned int fd, unsigned int to_submit,
                   unsigned int min_complete, unsigned int flags,
                   sigset_t *sig);

If io_uring_enter is the engine starter, io_uring_register is the "VIP pass" for your data. It is a system call used to pre-register resources (like memory buffers or files) with the kernel. By doing this, you remove the overhead of the kernel having to "look up" or "map" these resources every time you perform an I/O operation. With those three pieces, asynchronous programming in linux has never been the same again.

There are existing C wrappers around these kernel level calls, liburing and libc does a great job. But there is another hand in play for leveraging io_uring's capabilities in building asynchronous applications, Rust. Just imagine that, it solves the fundamental conflict: Completion Based I/O (the whole concept of io_uring) but with memory safety. In io_uring, the kernel takes control of your memory buffers to perform I/O asynchronously. This creates a "danger zone" where both the kernel and your application could try to access the same memory at once. The Rust compiler ensures your code cannot even touch that buffer until the kernel returns it in a Completion Queue Entry (CQE). Existing Rust wrappers for this Linux Kernel I/O API are some including tokio's io-uring, tokio-uring and glammio among others.

there's more

Implementing a Completion-Based mechanism for I/O operations at the kernel level can be more useful in today's real world not just for linux. Yeah, Windows has had such technology, it's called I/O completion ports (IOCP). Even though iocp is asynchronous it still requires a system call for every request which makes the 'tax' a little higher compared to io_uring's. The standard way for doing this in MacOS is by using the kqueue API. It is still readiness-based. You have to call kevent to find out what is ready, then make separate system calls to actually perform the I/O. It suffers from the same syscall overhead that io_uring was designed to kill. Which makes io_uring a true asynchronous programming linux system calls API.

But here's the thing about finding technological solutions, you don't have to solve everything. Someone I once looked up to once told me, "Be the best cog, but keep in mind you're not the only one in the machine." From my own point of view, I think the real problem lies in creating an asynchronous runtime that is cross-platform and completion-based. Such a technology exist, we have compio, a Rust framework for asynchronous I/O operations. What makes it less perfect are two things: it's arguably doesn't have zero-cost abstraction and it uses fixed buffers(part of io_urings's design) which are immutable references. Most of the Rust ecosystem is built on top of std::io::Read and std::io::Write traits which take this mutable references. Compio? It primarily emphasizes the ownership of buffers rather than borrowing them. Which as we've seen aligns with io_uring completion-based model, but then it poses a real problem with integration with the rest of the ecosystem.

But again, like I said, we just have to be here, implementing one solution at a time. By believing in ourselves even when it seems impossible. Until next time, peace, focus, desire.

You can checkout others of my blogs.
Follow me on:
On github: https://github.com/Sospeter-M-Kinyanjui
On LinkedIn: https://www.linkedin.com/in/sospeter-kinyanjui-6a091b2a5/