DEV Community: Erkin Khidirov

Everything You Need to Know About IPv4, IPv6, Dual Stack, DNS64, and NAT64

Erkin Khidirov — Wed, 21 Oct 2026 14:01:41 +0000

When you start diving into server and network configuration, you constantly run into terms like IPv4, IPv6, NAT, CGNAT, Dual Stack, DNS64, NAT64, and 464XLAT. All too often, they are explained in a way that makes zero sense without a degree in network engineering. Let’s break down from scratch and in plain English how internet addresses work, why we ran out of the old ones, how carrier-grade translators operate under the hood, and the exact places where your backend code might crash with an error.

Why We Ran Out of Old IPv4 Addresses

For computers, phones, and servers to find each other on the internet and exchange data, every device needs its own digital identifier an IP address.

In 1981, the fourth version of the Internet Protocol, called IPv4, was adopted. This protocol allocated 32 bits of memory for each device’s address. That limit allows for exactly 4,294,967,296 unique addresses (just over four billion). Back in the 1980s, computers only existed in universities and military labs, so four billion numbers seemed like they would last forever.

Today, however, there are eight billion people on the planet. Almost everyone has a smartphone, homes have routers, laptops, and smart TVs, and data centers run hundreds of millions of servers. As a result, the world physically ran out of unallocated IPv4 addresses back in 2011.

Public vs. Private Addresses and How NAT Works

When free addresses started running out, Internet Service Providers (ISPs) could no longer hand out a dedicated, globally routable IP address to every home computer and phone. To solve this problem, addresses were split into two types that developers refer to as public (external, or "white" in Russian networking slang) and private (internal/local, or "gray").

Imagine a large office building. The building itself has an official street address say, 5 Lenin Street. Any courier in the city can find this building on a map and deliver a package there. In computer networking, a real, globally unique address on the internet like this is called a public IP address.

Inside this office building, there are a thousand rooms numbered 101 to 1100. If you order food delivery and only put "Room 105" as the address without specifying the street and building number, the courier will never reach you because every other building in town has a Room 105. In networking, these internal room numbers are called private (or local) IP addresses. They are free, you can hand out as many of them as you want, and they only work inside a single building (within a single local network) the wider internet has no idea they exist.

So how do people sitting in those offices order goods from the internet if they only have internal private numbers?

To make this work, the building places a receptionist with a notepad at the front entrance. In networking, the software that acts as this receptionist on a router is called NAT (Network Address Translation). It works like this:

An employee in Room 105 wants to open Google and sends a request.
The receptionist at the entrance (the router running NAT) takes this request, erases the return address "Room 105" from the envelope, and replaces it with the street address of the entire building: "5 Lenin Street."
At the same time, the receptionist jots down a note in their ledger: "Request #1 to Google was sent by the person in Room 105."
Google’s server receives the request, sees the building's street address, and sends the response back to "5 Lenin Street."
The receptionist at the entrance catches the response from Google, checks their notepad, sees that Room 105 is waiting for this specific reply, and forwards the data inside to the right room.

This exact same miniature NAT receptionist is running right now inside your home router so that your phone, laptop, and TV can all access the internet through a single cable.

When ISPs themselves started running out of addresses, they placed one giant receptionist at the level of an entire neighborhood or city. This massive ISP-level translator is called CGNAT (Carrier-Grade NAT). An ISP takes just a single real public IPv4 address and puts anywhere from 1,000 to 5,000 different apartments behind it. All of these thousands of people sit on internal private addresses and access the internet through one shared external ISP address.

Three Problems Caused by ISP CGNAT

Putting thousands of different users behind a single shared ISP address solved the address shortage, but it created three major headaches for users and web developers:

Getting blocked because of your ISP neighbors. Because a thousand different apartments access the internet under the exact same public IPv4 address, to external websites all these people look like a single person. If just one of those thousand subscribers runs an aggressive web-scraping script, starts sending spam, or catches malware, security systems like Cloudflare or Google will block that external IP address. As a result, the other 999 regular people on the same ISP get an access denied error or get stuck solving endless CAPTCHAs whenever they try to open a website.
Inability to connect two devices directly (for video calls and gaming). When you simply open a website, your computer messages the server first, so the NAT receptionist writes an entry in their notepad and knows where to send the reply. However, some tasks require two client computers to connect directly to each other (peer-to-peer) without a central server in the middle.
The first example is browser-based video calls powered by WebRTC. When two people are on a video call on a website, routing a heavy video stream through the website owner's server is far too expensive. Instead, browsers use built-in WebRTC technology to stream audio and video directly from one caller's phone to the other's.
The second example is co-op multiplayer games. In many games, developers don't rent dedicated servers for matches. Instead, one player's computer becomes the host for the current game lobby, and the other players' computers try to connect directly to that host PC.

Now look at what happens when both people are sitting behind ISP CGNAT. The first person's computer tries to knock on the second person's computer and sends a packet to the shared external address of their ISP. The packet arrives at the ISP's CGNAT receptionist. The receptionist opens their notepad and sees that nobody inside the building ordered this package. With five thousand different apartments behind them, the receptionist has no idea which of the five thousand subscribers this incoming call is meant for so they simply drop the packet. The direct connection fails. For the video call or game to work at all, the developer has to pay out of pocket to rent an intermediate relay server (called a TURN server in WebRTC) that both clients connect to outbound, routing all traffic through it.
Paid IP addresses at hosting providers. Because there are no free IPv4 addresses left, cloud and hosting providers buy them from one another at prices ranging from $35 to $50 per address. That is why, when you rent a cloud server today, hosting companies charge developers a separate monthly fee just for having a single public IPv4 address attached to the server.

Why IPv6 Was Created and Why Everyone Hasn't Switched Yet

The sixth version of the Internet Protocol (IPv6) was designed to get rid of ISP CGNAT translators entirely and once again give every smartphone, router, and server in the world its own dedicated public IP address.

Instead of 32 bits per address, IPv6 allocates 128 bits. That yields exactly 340,282,366,920,938,463,463,374,607,431,768,211,456 unique addresses. With a supply that massive, there is no longer any need to conserve numbers, which is why hosting providers hand out massive blocks of IPv6 addresses for free.

So why hasn't the entire world shut off the old IPv4 protocol yet?

The main reason is that the creators of IPv6 made it completely backward-incompatible with the old IPv4 protocol. It is like two people where one only speaks Russian and the other only speaks Chinese. Without a translator, they won't understand a single word. Similarly, a computer configured only with the old IPv4 protocol physically cannot send data to a server that only has a new IPv6 address, and vice versa.

This incompatibility caused the rollout delay. If you disable old IPv4 on your website right now and leave only IPv6, not a single user whose home ISP still runs exclusively on the old protocol will be able to reach your site.

And many residential ISPs (especially in the CIS and Central Asia) are in no rush to enable IPv6 for a simple financial reason. An ISP's network is built out of expensive enterprise-grade core routers that pump traffic between districts and cities. On older hardware, IPv6 support often performs poorly or requires costly paid licenses and a complete reconfiguration of the company's network. It makes no financial sense for an ISP to spend money on this when they have already hidden thousands of subscribers behind a single address for free using CGNAT, and all popular websites still load just fine over legacy IPv4.

The companies that switched to the new protocol first were those where the old setup simply buckled under the load. These include major cloud hosting providers and massive mobile carriers (for example, in Germany, France, the US, and India), where tens of millions of smartphones were literally overflowing the state tables and memory of CGNAT gateways. As a result, more than half of global mobile traffic today already runs over IPv6, while fixed-line home internet in many countries continues to run on IPv4.

What Is Dual Stack?

Because half the internet sits on the old protocol and half of all mobile networks have already moved to the new one, website and server owners have had to support both protocols at the same time. This operating mode is called Dual Stack.

There is no translation from one protocol to another inside Dual Stack. Continuing our language analogy, a Dual Stack server is simply a bilingual person who speaks both Russian and Chinese fluently.

When you configure Dual Stack on your server, it looks like this:

Your server’s network interface is assigned two public addresses at once (one legacy IPv4 address and one new IPv6 address).
In the internet’s phonebook (the DNS system, which translates a human-readable domain name like example.com into a numeric IP address), you create two records for your domain. An A record stores your old IPv4 address, and an AAAA record stores your new IPv6 address.
Your web server (such as Nginx or a Go application) listens for incoming connections on both addresses simultaneously. If a user connects from a home internet connection in Tashkent that only has IPv4, the server talks to them over IPv4. If a user connects from a mobile network in Germany running IPv6, that exact same server replies to them directly over IPv6.

Meanwhile, modern browsers and smartphones run a built-in fallback mechanism called Happy Eyeballs. When a user opens your site, the browser requests both addresses (A and AAAA) from DNS and tries connecting over the newer IPv6 protocol first. If for some reason the ISP’s IPv6 routing is broken and the server doesn't respond within 150 to 250 milliseconds, the browser seamlessly falls back to the old IPv4 address without the user ever noticing.

What Are DNS64 and NAT64, and Why Are They Needed?

Maintaining Dual Stack on a single server is easy. However, inside a massive mobile carrier (like T-Mobile in Germany) with tens of millions of smartphones connected simultaneously, running two parallel networks at once is far too complex and expensive.

So mobile carriers took a different path: they completely turned off the old IPv4 protocol on their subscribers' phones and left smartphones with IPv6-only addresses.

At this point, any web developer will ask a practical question: Suppose I created a subdomain for my site, ws.example.com, that handles WebSocket connections, and I only configured an old IPv4 address (A record) for it in DNS without setting up an IPv6 address. Will a user in Germany on an IPv6-only phone be able to connect to my WebSocket?

Yes, they will 100% of the time.

First, if you have Cloudflare protection enabled in front of your server (the orange cloud icon is turned on in your DNS dashboard), Cloudflare automatically acts as a Dual Stack proxy on your behalf. It gives the user in Germany its own IPv6 address, accepts their connection, and then forwards the data to your server over your old IPv4 address.

Second, even if you disabled Cloudflare proxying and pointed traffic directly to your server with an old IPv4 address, the German user will still connect. Their connection is automatically bridged by a pair of translators DNS64 and NAT64 installed inside their mobile carrier's network.

The number 64 in these names has nothing to do with 64-bit processors. In networking, it stands for "from version 6 to version 4" meaning traffic translation from the IPv6 protocol to the IPv4 protocol.

Why are two separate programs (both DNS64 and NAT64) needed here? Because the IPv4 protocol is completely disabled on the user's smartphone. If you simply handed that phone your server's old IPv4 address (say, 185.10.20.30), the phone's operating system wouldn't understand the format and would immediately throw a network error without even attempting to send a packet anywhere.

So the mobile carrier tricks the smartphone in two steps:

Step one is handled by the DNS64 server (address synthesis in the phonebook). When the browser on the smartphone tries to connect to ws.example.com, it asks the carrier’s DNS64 server for a new-style IPv6 address (AAAA record). The DNS64 server checks your domain, sees that you don't have an IPv6 address, and finds only your old IPv4 address (185.10.20.30). To prevent the phone from throwing an error, DNS64 takes a special standardized IPv6 prefix (the translator prefix, typically 64:ff9b::/96) and glues your old 185.10.20.30 address onto the very end of it. This produces a synthetic, yet validly formatted IPv6 address, which DNS64 returns to the smartphone.
Step two is handled by the NAT64 router (packet translation at the network edge). The smartphone sees a normal-looking IPv6 address and happily sends packets to it. Inside the mobile carrier's network, a routing rule directs all packets starting with that synthetic prefix to a dedicated border router called NAT64. This router catches the packet from the phone, strips off the glued-on IPv6 header, extracts your real 185.10.20.30 address from the tail end, repacks the payload into a standard legacy IPv4 packet, and sends it out to your server. When your server replies, the NAT64 router wraps the response back into an IPv6 envelope and sends it back to the phone.

Here it is crucial to understand the golden rule: The DNS64 and NAT64 translators sit inside the client’s mobile carrier network and only work one way when a device inside an IPv6 network initiates an outbound call to an old IPv4 server.

Standard cloud hosts (like Hetzner or DigitalOcean) do not provide an inbound translator for you. If you decide to save a couple of dollars when renting a server by unchecking the public IPv4 address option and buying an IPv6-only server, users on legacy home internet connections will not be able to reach your server directly (you would have to put a Cloudflare proxy in front of it). Even worse, from an IPv6-only server, you won't even be able to run git clone from GitHub, because GitHub's servers still run exclusively on legacy IPv4. For an IPv6-only server to download packages from legacy IPv4 sites, you have to manually configure a free public DNS64 resolver (such as the free one provided by Cloudflare) in your server's DNS settings.

What Are 464XLAT and CLAT?

The DNS64 + NAT64 combo saves the day whenever a browser or app connects to your server using a domain name (like ws.example.com). After all, it is during that DNS lookup that the DNS64 server gets the chance to quietly slip the phone a synthetic IPv6 address.

But what happens if an app on the phone skips DNS entirely and tries to connect to a server directly using a raw numeric IPv4 address (like 185.10.20.30)?

In software development, this happens in two main scenarios:

First, legacy apps and games where programmers hardcoded the server’s raw IPv4 address directly into the source code.
Second, our familiar friend WebRTC for video calls, where during connection setup (ICE candidate exchange), browsers and servers swap raw numeric IP addresses with each other without using domain names at all.

When an app on an IPv6-only smartphone tries to send a packet directly to the raw numbers 185.10.20.30, the DNS64 server never even hears about it because nobody queried the DNS phonebook. The phone’s operating system sees an attempt to send a legacy-format packet and drops the connection immediately.

To keep video calls and games from breaking on mobile networks, engineers invented 464XLAT. The numbers 4, 6, and 4 stand for translating every packet twice along the path "from IPv4 to IPv6, and back to IPv4" while XLAT is short for translation.

Here is how it works:

Built directly into the operating system of every modern smartphone (both Android and iOS) is a tiny system-level translator daemon called CLAT (Customer-side Translator).
When CLAT detects that the mobile carrier only gave the phone an IPv6 connection, it spins up a virtual (fake) legacy network interface right inside the phone's memory and assigns it a local private IPv4 address.
The app or WebRTC video engine sees what looks like a working IPv4 internet connection on the phone and sends a packet to your server’s raw address: 185.10.20.30.
Right on the phone's CPU, the CLAT program intercepts this packet, repackages it into an IPv6 packet, and sends it out over the carrier’s mobile network (this is the first translation: from version 4 to version 6).
At the edge of the mobile carrier's network, this packet is caught by our familiar carrier border router, NAT64 (also called PLAT, or Provider-side Translator, in 464XLAT documentation), which converts the packet back into the old IPv4 format and delivers it to your server (this is the second translation: from version 6 to version 4).

What Is the Practical Takeaway for a Developer?

Thanks to the safety nets of DNS64, NAT64, and 464XLAT, a mobile user will reach your server and connect to your socket even if you only have legacy IPv4 configured.

However, going through translators always comes with an overhead cost:

With 464XLAT, every single voice or video packet is reassembled twice (first by the phone's CPU, then by the carrier's router), which adds latency.
Long-lived WebSocket connections running through a carrier's NAT64 router can drop out of nowhere. Because a mobile carrier’s NAT64 router stores a state table of all active subscriber connections in its RAM, it aggressively purges idle connections to keep its memory from overflowing often killing any connection that hasn't transferred data for 30 to 60 seconds. If nobody types anything in your chat room for half a minute, the carrier's translator will silently sever the connection. To prevent this, you must either send empty keepalive Ping and Pong frames over the WebSocket every 15 to 20 seconds, or enable native IPv6 on your server so mobile clients connect directly, bypassing all translators.

Why a Hetzner IPv6 Address Won't Paste into Cloudflare

Let’s look at a classic mistake developers make when setting up IPv6 for the first time, using a real-world example with Hetzner Cloud and the Cloudflare DNS dashboard.

When you rent a server on Hetzner, the host gives you one single legacy IPv4 address and an entire block of new IPv6 addresses. In the Hetzner control panel, this block is displayed as four groups of characters separated by colons, ending with a double colon and /64 (for example, 2a01:4f8:c010:d56::/64).

If you copy that entire string and try to paste it into an AAAA record in Cloudflare, you will get an error saying that the IPv6 address is invalid.

This happens because a string ending in /64 is not the address of a single specific server it is the notation for your entire allocated subnet (essentially an entire street containing 18 quintillion individual addresses).

Every 128-bit IPv6 address is always split cleanly in half:

The first 64 bits (the first four groups of hex characters) represent the identifier of your network on Hetzner's routing equipment.
The second 64 bits (everything that comes after the double colon ::) represent the specific device (interface) identifier inside your network. Two colons in a row (::) simply compress a long string of zeros to make the address shorter to write.

Cloudflare’s DNS settings won't accept the address of an entire street with a /64 suffix; they require the exact house number of a single server. When Hetzner automatically provisions your server, it always assigns the very first address from your allocated block to your server's network interface. Therefore, to add an AAAA record in Cloudflare, all you need to do is delete the /64 at the end of the string and put the number 1 right after the double colons (for example, 2a01:4f8:c010:d56::1).

Four Places in Code and Server Configs Where Developers Make Mistakes

As a web developer, you don't need to configure network translators yourself ISPs and services like Cloudflare handle that. However, the moment you enable Cloudflare for your site or add an AAAA record in DNS, your backend will start receiving requests from users with IPv6 addresses. Here are four specific places you need to audit in your project:

Database column length for IP addresses. If you store user IP addresses in your database (for instance, during registration, checkout, or in login audit logs), check the column width. In the past, developers often used VARCHAR(15) because an old IPv4 address is never longer than 15 characters. However, a full IPv6 address can be up to 45 characters long. If your database column is capped at 15 characters, the very first mobile user who logs into your site will trigger a database truncation/overflow error and won't be able to sign in. Always set your column length to VARCHAR(45) or use the dedicated INET data type in PostgreSQL.
Parsing IP addresses and ports in code. In the old IPv4 protocol, the IP address was separated from the port number by a colon (e.g., 185.10.20.30:8080), so many programmers simply split the string on the colon character using a standard string split(":") function. In IPv6, the address itself contains multiple colons. If you run a naive split(":") on an IPv6 address, your code will break and grab a chunk of the IP address instead of the port number. To avoid this bug entirely, never parse network addresses manually with string functions; always use the built-in functions from your programming language's standard networking library (such as net.SplitHostPort and net.JoinHostPort in Go's standard net package).
Rate limiting and bot blocking. Many websites protect themselves with a rate limiter that allows no more than, say, 10 requests per second from a single IP address. With legacy IPv4, we simply take the entire address and use it as the key in our rate-limiting counter. In the IPv6 world, however, blocking a single specific IP address is useless. As we saw with Hetzner, IPv6 networks don't hand out a single address to a server or home router they hand out an entire /64 subnet containing 18 quintillion free addresses. A spammer can easily send every new request from the next sequential address inside their free subnet just by rotating the last few digits. For your rate limiter to work properly, whenever you receive an incoming IPv6 address, you must programmatically mask/strip off the second half (the last 64 bits) and count all requests coming from the first 64 bits (/64 prefix) as belonging to the exact same user.

Open ports in the second firewall. When your server runs in Dual Stack mode, it effectively has two separate front doors. On Linux, the classic iptables firewall utility only closes ports for the old IPv4 protocol, while a completely separate utilityip6tablesgoverns the IPv6 protocol. This frequently leads to a dangerous security blind spot: a developer spins up a database or Redis instance on a server, locks down the port from the outside world using familiar iptables, scans the server externally via its old IPv4 address, sees that the port is closed, and assumes the server is secure. Meanwhile, that database is sitting wide open to the entire internet through the second front door via its public IPv6 address because nobody configured any rules in ip6tables. If you use a modern firewall wrapper like ufw, it automatically generates rules for both protocols at once but you should always test your closed ports from the outside using both your legacy IPv4 address and your new IPv6 address.

Conclusion

Let's get back to my project, shomla, and wrap up this entire story with the practical takeaways. Once I dug into the mechanics of how ISP translators work, it became clear why users on mobile internet in Germany were able to join my voice chat without any issues even when my server only had the old IPv4 protocol configured. Behind the scenes, Cloudflare proxy servers and carrier NAT64 gateways paired with the built-in CLAT daemon inside the smartphones themselves were quietly doing all the heavy lifting of translating packets for me.

However, understanding how this hidden machinery actually works allowed me to eliminate critical bottlenecks in my chat architecture. It is one thing when you run a standard content website with articles, where an extra packet reassembly step on a mobile carrier's router makes no noticeable difference. It is a completely different story when you are running a real-time voice chat built on WebSockets and WebRTC.

When a mobile user's traffic goes through NAT64 and 464XLAT translators, their smartphone and the carrier's hardware are forced to reassemble every single voice packet twice on the fly, while the carrier's router constantly tries to forcibly kill the WebSocket connection during any brief pause in data transmission. As soon as I properly added my server's IPv6 address (ending in ::1) to the AAAA DNS records for my domains, configured my server to listen on both protocols, and verified that keepalive Ping and Pong frames were firing regularly over the WebSockets, mobile clients were able to connect to the server directly over pure, native IPv6 with zero middlemen.

What I Recommend Doing on Your Project Right Now

Don't drop everything and panic if your server is currently running only on a legacy IPv4 address. Your website is already accessible to users worldwide because mobile carriers and Cloudflare have taken care of backward compatibility for you.
If your hosting provider (such as Hetzner or DigitalOcean) has already given your server a free IPv6 subnet, spend ten minutes enabling full Dual Stack. Replace the /64 suffix at the end of your allocated address with 1, add an AAAA record in your DNS dashboard, and make sure your Nginx proxy or backend application is listening on both network interfaces. Doing so will spare your mobile users from double packet translation and random connection drops on carrier routers.
Audit your backend code and server configuration against the four pitfalls we covered above. Expand your IP address database column to at least 45 characters (or INET), remove manual colon-based string splitting when parsing addresses in code, update your spam blockers and rate limiters to group IPv6 users by their /64 prefix (the first 64 bits), and check your server's ip6tables firewall rules today so your database isn't accidentally left wide open to the entire internet.

Installation and configuration of HermitMQ

Erkin Khidirov — Wed, 19 Aug 2026 14:05:41 +0000

I will show you how to deploy HermitMQ and integrate clients. The broker comes as a ready package for Debian and Ubuntu systems. Download the latest release and install it using the package manager. The service will start automatically.

cd /tmp/
wget https://github.com/ekhidirov/hermitmq/releases/latest/download/hermitmq_amd64.deb
sudo apt install -y ./hermitmq_amd64.deb
sudo systemctl status hermitmq

Client installation

To work with the broker download the SDK into your project. The import paths are now configured correctly for public access.

go get github.com/ekhidirov/hermitmq/pkg/client

Project structure

The Go compiler forbids keeping two entry points in one folder. I placed the files in different directories inside the cmd folder. This solves all compilation problems. Prepare the following file structure before writing code.

my_project/
├── cmd/
│   ├── consumer/
│   │   └── main.go
│   └── producer/
│       └── main.go
├── go.mod
└── go.sum

Interactive sender

First we initialize the producer. When connecting we specify the server address topic and partition number. Instead of a placeholder be sure to specify the real IP address where your broker is deployed. I added a loop to read keyboard input. The program will wait for your text and send it to the broker.

package main

import (
    "bufio"
    "fmt"
    "log"
    "os"

    "github.com/ekhidirov/hermitmq/pkg/client"
)

func main() {
    // Connect to the broker replacing YOUR_SERVER_IP with actual IP
    producer, err := client.NewProducer("YOUR_SERVER_IP:9092", "test_topic", 0)
    if err != nil {
        log.Fatalf("Connection error: %v", err)
    }
    defer producer.Close()

    fmt.Println("Connected to the cloud! Type your messages and press Enter:")

    // Start the command line scanner
    scanner := bufio.NewScanner(os.Stdin)
    for scanner.Scan() {
        text := scanner.Text()
        if text == "" {
            continue // Ignore empty Enter key presses
        }

        offset, err := producer.Send([]byte(text))
        if err != nil {
            log.Printf("Send error: %v\n", err)
            continue
        }

        fmt.Printf("✔ Delivered (Offset: %d)\n", offset)
    }

    if err := scanner.Err(); err != nil {
        log.Printf("Console read error: %v", err)
    }
}

Reading and committing

Listener configuration is done similarly. I create a consumer and pass the group name. The broker itself will remember the reading position. The receive method works by blocking the goroutine. I trigger a manual commit only after the text is successfully printed to the screen to guarantee delivery.

package main

import (
    "fmt"
    "log"

    "github.com/ekhidirov/hermitmq/pkg/client"
)

func main() {
    // Connect to the broker replacing YOUR_SERVER_IP with actual IP
    consumer, err := client.NewConsumer("YOUR_SERVER_IP:9092", "test_group", "test_topic", 0)
    if err != nil {
        log.Fatalf("Connection error: %v", err)
    }
    defer consumer.Close()

    fmt.Println("Listening to the network. Waiting for new messages...")
    for {
        msg, err := consumer.Receive()
        if err != nil {
            log.Fatalf("Network error or disconnection: %v", err)
        }

        // Print the received payload
        fmt.Printf("[Incoming | Offset: %d] %s\n", msg.Offset, string(msg.Payload))

        // Commit the read offset
        consumer.Commit(msg.Offset)
    }
}

Launching the integration

Open two terminal windows. In the first one start the listener. In the second one start the sender and write messages.

go run cmd/consumer/main.go
go run cmd/producer/main.go

How I wrote a Go message broker with a throughput of a million messages per second

Erkin Khidirov — Tue, 18 Aug 2026 21:15:58 +0000

I built HermitMQ entirely in Go. The main feature is ditching heavy wrappers like JSON in favor of a custom 29 byte binary protocol. Additionally, data transmission over the network uses a direct file to socket copy mechanism. I will go into detail about the architecture, data storage approaches, benchmark numbers, and show how it is implemented in code.

The full source code for the HermitMQ project is available on GitHub: https://github.com/ekhidirov/hermitmq

The problem with standard brokers and the cost of serialization

When the message counter exceeds hundreds of thousands per second, the main problem for a Go developer is the garbage collector. If every message is parsed via standard JSON, the application starts allocating a massive number of small objects in memory. The GC wakes up too frequently, eating up CPU time and causing network latency spikes.

To avoid triggering the garbage collector at every turn, I completely abandoned standard serialization libraries. Every message is packed into a custom header of exactly 29 bytes. In code, the message structure looks extremely simple:

type Message struct {
    Magic       byte
    Timestamp   uint64
    Offset      uint64
    KeySize     uint32
    PayloadSize uint32
    RecordCount uint32
    Key         []byte
    Payload     []byte
}

The first byte is a magic number for version checking and instantly discarding bad packets. Next come 8 bytes for the timestamp in nanoseconds and 8 bytes for the offset, which the broker fills in itself to maintain message order. Then come the key and payload sizes, 4 bytes each. Finally, 4 bytes are reserved for the record count to support batching. The broker reads the stream using the binary package and reuses buffers via sync.Pool. As a result, under standard loads, we achieve practically zero memory allocation.

Being honest about allocations and plans for zero serialization

To be completely honest: although the broker is incredibly frugal under standard loads, a memory management compromise still remains.

An absolute victory over allocations and the GC has been achieved on the side of serving data to consumers, thanks to io.CopyN (the very zero copy approach I will discuss below). However, on the receiving end from producers, the Decode method still dynamically allocates slices for the payload size (make([]byte, m.PayloadSize)).

In future releases, I plan to completely eliminate this structure. We will move towards the concept of zero serialization (like in flatbuffers) and micro optimizations with branchless logic in hot loops. The broker will access byte offsets directly in the network buffer, without unpacking them into Go structs at all. This will finally eliminate the GC from the message processing path.

Direct data transfer at the kernel level

Copying data from the OS kernel into user space and back to the network interface is very expensive. It is double the work for the CPU and RAM. In HermitMQ, consumers receive data directly.

When a client requests data, the broker looks up the offset in the log file. Instead of reading the file into the program memory, it uses the built in io.CopyN function.

bytesToSend := int64(HeaderSize) + int64(keySize) + int64(payloadSize)
reader := io.NewSectionReader(walFile, position, bytesToSend)
_, err = io.CopyN(conn, reader, bytesToSend)
Data flows straight from the physical file on disk into the consumer network socket. Latencies remain minimal, and the CPU does not strain itself moving bytes around.

Data storage, mmap, and compaction

All data is written to a write ahead log (WAL). The log is divided into segments. Each segment has a .wal file with raw data and an .idx file to map offsets to physical bytes on the hard drive.

For fast searching, indices are mapped directly into RAM via mmap. The broker performs a binary search in memory and instantly finds the required byte. Background log compaction is also implemented here. It deduplicates messages by key, leaving only the most relevant data.

Surviving crashes: the torn page recovery mechanism

Any database developer knows: a server can crash at any millisecond due to a power failure or a hard kill 9 signal. If this happens during a log write, a half written chunk of data (a torn page) will be left at the end of the file.

HermitMQ implements an automatic recovery mechanism. On startup, the broker sequentially scans the WAL files. If it sees that the expected message size in the header exceeds the physical size of the file on disk, it understands that the write was interrupted. Instead of panicking and crashing the entire segment, the broker safely truncates this corrupted tail at the hardware level using os.Truncate.

if currentPos+msgSize > totalFileSize {
    log.Printf("PARTIAL MESSAGE DETECTED. Truncating tail.")
    break
}

// ...
if err := os.Truncate(walPath, currentPos); err != nil {
    return err
}

This guarantees that the broker will boot up with a completely consistent state, even after the most severe operating system crash.

Lock sharding and scaling

Storing consumer group offsets under a single global mutex is a bad idea when dealing with thousands of connections. Threads will start queuing up and kill all performance. Therefore, the offset store is split into 256 independent shards.

The broker takes a string key, runs it through a hashing algorithm, and routes it to the correct shard. Here is the implementation of this hash:

func getShardIndex(key string) uint8 {
    var hash uint32 = 2166136261
    for i := 0; i < len(key); i++ {
        hash ^= uint32(key[i])
        hash *= 16777619
    }
    return uint8(hash & (OffsetShardCount - 1))
}

Incoming requests are evenly spread across the allocated memory. Producers and consumers do not block each other, and the broker scales perfectly across CPU cores.

Developer experience: on demand routing

Working with hardcore systems is often painful due to excessive infrastructure bureaucracy, you usually have to hit an API in advance to configure topics and partitions. I solved this problem through on demand routing.

If a producer or consumer knocks on the broker and requests a non existent topic, the broker does not reject the connection with an error. Instead, it transparently creates the required folder and file structure on the fly under an RWMutex lock.

b.mu.RLock()
_, exists := b.topics[topicName]
b.mu.RUnlock()

if !exists {
    slog.Info("auto-creating new topic on demand", "topic", topicName)
    b.CreateTopic(topicName, 1)
}

The client simply needs to start writing data, the entire physical structure under the hood will spin up on its own.

Smart batching on the client

Throughput depends not only on the server but also on the clients. In the project, the test producer uses Go channels and tickers to group messages.

const (
    ProducersCount    = 10
    MessagesPerWorker = 100000
    BatchSize         = 100
    LingerTime        = 100 * time.Millisecond
)

Workers buffer messages and flush them to the TCP socket only when two conditions are met: a batch of 100 records has accumulated, or a 100 millisecond wait timer expires. This glues a hundred logical records into a single dense header and a single system write call.

Benchmark numbers and stress test results

To measure the real throughput of the pipeline, I set up a cluster of 10 parallel producers and a separate consumer. The test scenario pumps 1,000,000 messages through the broker.

Producers generate 10 threads of 100,000 messages each. The consumer fetches the incoming stream, parses the 29 byte headers, and calculates the net throughput over time.

totalMB := float64(logicalCount * 1812) / 1024 / 1024
fmt.Printf("Throughput: %.0f msgs/sec (Approx %.0f MB/sec)\n", 
    float64(logicalCount)/elapsed.Seconds(), 
    totalMB/elapsed.Seconds())

With a batch of 1,000,000 messages, the entire volume of data is pumped through in a matter of seconds. Thanks to zero copy and the complete absence of intermediate memory allocations, the broker delivers hundreds of thousands of messages per second and utilizes tens of megabytes of traffic per second on a single instance, bottlenecked solely by the physical speed of the disk bus and the local TCP stack.

Overload protection and security

The broker is protected against memory exhaustion attacks by strict limits. The payload size is strictly validated before memory is allocated; by default, the limit is set to 50 megabytes.

maxBytes := uint32(maxPayloadMB * 1024 * 1024)
if m.PayloadSize > maxBytes {
    return fmt.Errorf("security policy violation: payload too large")
}

If the size exceeds the limit or a packet arrives with a bad starting byte, the broker does not crash with an out of memory error; it simply drops the malicious connection.

Graceful shutdown

High delivery speed is meaningless if data stuck in OS buffers is lost during a restart. That is why the broker knows how to respond correctly to system interrupt signals.

Interception of SIGINT and SIGTERM signals is implemented in main.go. Upon receiving a signal, the broker stops accepting new connections, closes the TCP listener, and neatly iterates through all topics and partitions. For each open segment, Sync() and Close() are forcibly called, flushing all file descriptors to disk.

sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
<-sigChan
slog.Info("received shutdown signal, shutting down...")

listener.Close()
b.Close()

Conclusion

Writing your own message broker is excellent practice for understanding how high load systems work at a low level. The architecture allows implementing direct memory management, data streaming, and concurrency without unnecessary headaches. Ditching fat frameworks gives full control over every allocated byte and provides predictable performance under any load.

How Messengers Actually Encrypt Messages (End-to-End)

Erkin Khidirov — Thu, 29 May 2025 22:34:20 +0000

Hi everyone!

I've written an article and I'm publishing it prematurely. Initially, I planned to write it after the project was completed, but since there are still a couple of months left until the end, I decided not to waste time and write the article while the information is still fresh in my mind. Besides, I'm mostly writing this for myself. :) In one of my latest projects, which I'm developing as open source, I implemented end-to-end encryption—similar to how WhatsApp or Telegram do it, for example.

In this article, we'll dive into the implementation of client-side message encryption using JavaScript and the Web Crypto API, breaking down a practical example that will be at the very end of the article.

Let's start by saying that if you're a complete beginner in cryptography, understanding what's written here might not be easy. Even with 10 years of development experience, I had to scratch my head a bit—everything happening here is pure mathematics, which we won't be discussing in this article :) The easily impressed might think it's magic :)

If I were to briefly explain the essence of end-to-end encryption without complex words and terms:

The Magic of Encryption in Three Keys

Three keys are the foundation upon which end-to-end encryption is built. Pay attention to the key in the center – this is important.

The foundation on which everything rests is these three keys. Refer back to this section if something is unclear.

Private Key: Stored (in encrypted form).
Public Key: Accessible to everyone.
Shared Secret / Symmetric Key: Generated based on your private key + your contact's public key. This is the key used for the direct encryption and decryption of messages.

The combination of your private key + your contact's public key allows you to obtain the shared secret key (in our example below, this will be an AES key). Thanks to this shared secret key, you can encrypt and decrypt messages.

Private and public keys can be stored in a database, but there's a nuance with the private key. The private key itself is not recommended to be stored in plain text; it needs to be additionally encrypted with the user's password or any other keyword (within a messenger, this is typically the user's password). We store the public key in plain text.

The shared secret key (the one referred to as this.aesKey in the code below) is not stored in the database. It is generated (calculated) each time a chat with a specific contact is initialized. This might cause confusion: how will we decrypt messages if this key is not stored but generated anew? This is where the "magic" of asymmetric encryption and key exchange protocols lies.

The shared key is "Your private key" + "Contact's public key."

When you open a chat with a contact, your client recalculates this shared secret key, as you already know, like this: (Your private key + Your contact's public key = Shared secret key). With this key, you encrypt new messages and decrypt all previous messages in this chat, as they were encrypted with the same shared secret key. You can wrack your brains for a long time to no avail until you understand asymmetric encryption and key exchange protocols.

Asymmetric Encryption and ECDH

Asymmetric encryption uses a pair of keys: public and private. The public key can be freely distributed, while the private key must be kept secret by its owner, meaning in an encrypted form.

ECDH (Elliptic Curve Diffie-Hellman) is a key exchange protocol based on elliptic curve mathematics. It allows two parties, each possessing their own ECDH key pair (private and public), to establish a shared secret key over an insecure channel. Importantly, a third party, even if they intercept their public keys, cannot compute this shared secret. Our example uses the P-256 curve – a popular and reliable standard.

I think few understood what they just read. All you need to understand at this stage is that the technology works :) Later, the puzzle will come together, perhaps after a reread. And now, a bit about built-in browser technologies.

Web Crypto API

The Web Crypto API is a JavaScript interface built into browsers that provides access to low-level cryptographic primitives. It allows performing operations such as hashing, signature generation, encryption, and decryption. Using the Web Crypto API is preferable to third-party libraries for basic cryptographic operations, as it is often hardware-accelerated and thoroughly vetted for security. All Web Crypto API operations are asynchronous and return a Promise.

Now let's move on to the practical analysis. I've created a ChatCrypto class, which we'll examine in more detail:

class ChatCrypto {

  constructor(myPrivateKeyBase64, theirPublicKeyBase64) {
    this.myPrivateKeyBase64 = myPrivateKeyBase64;
    this.theirPublicKeyBase64 = theirPublicKeyBase64;
    this.aesKey = null; // The shared symmetric AES key will be stored here
  }

  static base64ToArrayBuffer(base64) {
    const binary = atob(base64);
    const bytes = new Uint8Array(binary.length);
    for (let i = 0; i < binary.length; i++) {
      bytes[i] = binary.charCodeAt(i);
    }
    return bytes.buffer;
  }

  static arrayBufferToBase64(buffer) {
    const bytes = new Uint8Array(buffer);
    let binary = '';
    for (let b of bytes) {
      binary += String.fromCharCode(b);
    }
    return btoa(binary);
  }

  init() {

    // Convert keys from Base64 to ArrayBuffer
    const privateRaw = ChatCrypto.base64ToArrayBuffer(this.myPrivateKeyBase64);
    const publicRaw = ChatCrypto.base64ToArrayBuffer(this.theirPublicKeyBase64);

    // Note: The following lines for parsing publicRaw into x and y coordinates,
    // and assembling uncompressedPoint might be specific to a particular format
    // for representing a "raw" public key. If publicRaw is already in SPKI format,
    // they might not be necessary, as crypto.subtle.importKey("spki", ...)
    // expects a standard structure.
    // const x = publicRaw.slice(0, publicRaw.byteLength / 2);
    // const y = publicRaw.slice(publicRaw.byteLength / 2);
    // const uncompressedPoint = new Uint8Array([0x04, ...new Uint8Array(x), ...new Uint8Array(y)]);

    // Import our private key
    return crypto.subtle.importKey(
      "pkcs8", // Private key format (standard)
      privateRaw,
      { name: "ECDH", namedCurve: "P-256" }, // Algorithm and parameters
      false, // Non-exportable
      ["deriveBits"] // Allowed usage: for deriving bits (shared secret)
    ).then(privateKey => {
      // Import the contact's public key
      return crypto.subtle.importKey(
        "spki", // Public key format (standard)
        publicRaw,
        { name: "ECDH", namedCurve: "P-256" },
        false, // Non-exportable
        [] // Specific uses are not needed here for the public key in ECDH
      ).then(publicKey => {
        // 4. Compute the shared secret (deriveBits)
        return crypto.subtle.deriveBits(
          { name: "ECDH", public: publicKey }, // Specify the contact's public key
          privateKey, // Our private key
          256 // Length of the derived secret in bits
        );
      });
    }).then(sharedBits => {
      // Hash the shared secret to obtain an AES key (using SHA-256 as KDF)
      return crypto.subtle.digest("SHA-256", sharedBits);
    }).then(hashed => {
      // Import the hashed secret as an AES-GCM key
      return crypto.subtle.importKey(
        "raw", // "Raw" byte format
        hashed, // Hashed secret
        { name: "AES-GCM" }, // Symmetric encryption algorithm
        false, // Non-exportable
        ["encrypt", "decrypt"] // Allowed uses: encryption and decryption
      );
    }).then(aesKey => {
      this.aesKey = aesKey; // Save the obtained AES key
      return true;         // Signal successful initialization
    });
  }

  encrypt(plaintext) {
    if (!this.aesKey) return Promise.reject("ChatCrypto not initialized");

    // Generate a unique initialization vector (IV)
    const iv = crypto.getRandomValues(new Uint8Array(12)); // 12 bytes (96 bits) is recommended for AES-GCM

    // Convert the text message to bytes (UTF-8)
    const encoded = new TextEncoder().encode(plaintext);

    // Encrypt the data
    return crypto.subtle.encrypt(
      { name: "AES-GCM", iv: iv }, // Algorithm and IV
      this.aesKey, // Our shared AES key
      encoded // Data to encrypt
    ).then(encrypted => {
      // 4. Return IV and encrypted data (in Base64 for convenient transmission)
      return {
        iv: ChatCrypto.arrayBufferToBase64(iv),
        data: ChatCrypto.arrayBufferToBase64(encrypted)
      };
    });
  }

  decrypt(cipherBase64, ivBase64) {
    if (!this.aesKey) return Promise.reject("ChatCrypto not initialized");

    // Convert ciphertext and IV from Base64 to ArrayBuffer
    const encrypted = ChatCrypto.base64ToArrayBuffer(cipherBase64);
    const ivBuffer = ChatCrypto.base64ToArrayBuffer(ivBase64);

    // Decrypt the data
    return crypto.subtle.decrypt(
      { name: "AES-GCM", iv: new Uint8Array(ivBuffer) }, // Algorithm and IV (must be a TypedArray)
      this.aesKey, // The same shared AES key
      encrypted // Encrypted data
    ).then(decrypted => {
      // Convert decrypted bytes back to a string
      return new TextDecoder().decode(decrypted);
    });
  }
}

constructor() and base64ToArrayBuffer() method:
The constructor accepts your private key and the contact's public key in Base64 format. Base64 is a way of encoding binary data into a text string, convenient for transmission or storage.

this.aesKey is initialized as null and will be populated after the init() method successfully completes.

The static methods base64ToArrayBuffer and arrayBufferToBase64 are used to convert data between Base64 strings and ArrayBuffer (the format Web Crypto API works with).

init() Method: Establishing the Shared AES Key
This is the heart of our class, where the "magic" of ECDH occurs and the shared key for symmetric encryption is created.

Breakdown of steps in init():

Key Conversion: Keys are converted from Base64 to ArrayBuffer.
Import Private Key: Your private key is imported in pkcs8 format. It's specified that this is an ECDH key on the P-256 curve and will be used for deriveBits (computing the shared secret).

Import Contact's Public Key: The contact's public key is imported in spki format.

Compute Shared Secret (deriveBits): This is the key ECDH step. Using your private key and the contact's public key, deriveBits computes a shared secret set of bits (sharedBits). This secret will be the same for you and your contact if they use their respective private keys and each other's public keys.

Hash Shared Secret (digest): sharedBits are hashed using SHA-256. This is a common practice to transform the output of deriveBits into a cryptographically strong key of the desired length for a symmetric cipher (AES in this case). This step also serves as a KDF (Key Derivation Function).

Import AES Key (importKey): The resulting hash (hashed) is imported as a "raw" key for the AES-GCM algorithm. This key (this.aesKey) is now ready to be used for encrypting and decrypting messages.
After successful execution, this.aesKey will contain a CryptoKey object, ready for use.

encrypt(plaintext) Method: Encrypting a Message
Breakdown of steps in encrypt():

Generate IV (Initialization Vector): A random 12-byte IV is created. As a reminder, it must be unique for each encryption with the same key.

Encode Text: The message is converted from a JavaScript string to a Uint8Array (a sequence of bytes in UTF-8 encoding) using TextEncoder.
Encryption: crypto.subtle.encrypt performs data encryption using AES-GCM, our this.aesKey, and the generated iv.

Return Result: The encrypted data and IV (both in Base64) are returned as an object. The IV must be transmitted to the recipient along with the ciphertext, as it will be required for decryption.
decrypt(cipherBase64, ivBase64) Method: Decrypting a Message
Breakdown of steps in decrypt():

Data Conversion: The received ciphertext and IV (in Base64) are converted back to ArrayBuffer. Note that for crypto.subtle.decrypt, the iv parameter must be a TypedArray (e.g., Uint8Array), so we pass new Uint8Array(ivBuffer).

Decryption: crypto.subtle.decrypt performs the decryption. Importantly, AES-GCM will not only decrypt the data but also verify its integrity and authenticity, using the same aesKey and iv that were used for encryption. If the data has been tampered with, the key is wrong, or the IV is wrong, the decrypt method will return an error (reject the Promise).

Decode Text: Successfully decrypted bytes are converted back into a readable string using TextDecoder.

How It Works Together: Conceptual Flow Key Pair Generation:

User A generates their ECDH key pair (public PkA and private SkA).
User B does the same (public PkB and private SkB). This step is not shown in the provided ChatCrypto code but is shown below in the examples section (it is performed once, for example, during user registration), and it precedes the use of the class. The Web Crypto API has a crypto.subtle.generateKey method for this. The private key Sk must be stored securely and encrypted with the user's password.
Public Key Exchange:

User A transmits their public key PkA to User B.
User B transmits their public key PkB to User A. This exchange must be secure to avoid Man-in-the-Middle (MitM) attacks. For example, via a secure server or by verifying key fingerprints.

ChatCrypto Initialization and Shared Secret Key Computation:

On User A's side: const cryptoA = new ChatCrypto(SkA_base64, PkB_base64); await cryptoA.init();
On User B's side: const cryptoB = new ChatCrypto(SkB_base64, PkA_base64); await cryptoB.init(); As a result, both (cryptoA.aesKey and cryptoB.aesKey) will have computed the same symmetric AES key.
Message Exchange:

User A encrypts a message for B: const { iv, data } = await chatCryptoA.encrypt("Hello, B!"); Then A sends the { iv, data } object to user B.

User B receives { iv, data } and decrypts: const message = await chatCryptoB.decrypt(data, iv); // message will be "Hello, B!"

Example: Encrypting Text encrypt()

// Initialize the class
let chatCrypto = new ChatCrypto( "My_private_key" , "Contact_public_key" );

// Run it
chatCrypto.init().then(() => {
   chatCrypto.encrypt("Text").then(result => {

     // The encrypted text will be output
      console.log(result);

  });
});

Example: Decrypting Text decrypt()

// Initialize the class
let chatCrypto = new ChatCrypto( "My_private_key" , "Contact_public_key" );

// Run it
chatCrypto.init()
  .then( () => chatCrypto.decrypt("Encrypted_text", "Vector_key_iv") ) // Note: "Vector_key" likely means IV
  .then(result => {

    // The decrypted text will be output
    console.log(result);

  })

Conclusion

We've looked at how to implement robust end-to-end message encryption in JavaScript using the Web Crypto API. The combination of ECDH for secure key exchange and AES-GCM for efficient and authenticated data encryption is a powerful and modern approach. The ChatCrypto class serves as a good starting example of such an implementation. Remember the importance of secure generation, storage of private keys, and reliable exchange of public keys to build a secure system.