DEV Community

Cover image for MPTCP: Enabled in the Kernel, Used by Nobody
Mustafa ERBAY
Mustafa ERBAY

Posted on Originally published at mustafaerbay.com.tr

MPTCP: Enabled in the Kernel, Used by Nobody

The first time you hear about Multipath TCP it sounds too good: let a single TCP connection flow over several paths, don't drop the session when a link dies, and if you have two links, use both. Anyone with fibre and an LTE backup has had that thought. And the feature isn't in some distant future — it is in your kernel right now, enabled.

That is exactly what I found when I checked my own server. And right after it, the thesis of this article: being enabled does not mean being used. In MPTCP, enabled=1 isn't a starting line, it's just the first door standing ajar. Behind it are three more doors that must open in order, and in practice the place people get stuck is almost never the kernel.

Enabled on my server; never used

Here's the state on my server running Ubuntu 24.04 and kernel 6.8:

$ sysctl net.mptcp
net.mptcp.add_addr_timeout = 120
net.mptcp.allow_join_initial_addr_port = 1
net.mptcp.checksum_enabled = 0
net.mptcp.close_timeout = 60
net.mptcp.enabled = 1
net.mptcp.pm_type = 0
net.mptcp.scheduler = default
net.mptcp.stale_loss_cnt = 4

$ ip mptcp endpoint show
$ ip mptcp limits show
add_addr_accepted 0 subflows 2

$ nstat -az | grep -c MPTcpExt
63
$ nstat -az | grep MPTcpExtMPCapableSYNTX
MPTcpExtMPCapableSYNTX          0                  0.0
Enter fullscreen mode Exit fullscreen mode

The kernel supports MPTCP, it's on, it stands ready to keep 63 separate counters — and every one of them is zero. Not a single MPTCP connection attempt has ever left this machine. On most distributions the picture is identical; this table is the rule itself.

Why zero? Because the kernel being enabled is only one of four conditions.

Diagram

The first door: the application itself

MPTCP is not a transparent layer that slots in where TCP was. The application has to ask for it when it opens the socket; in kernel documentation terms, that means creating the socket with IPPROTO_MPTCP (value 262):

int sd = socket(AF_INET, SOCK_STREAM, IPPROTO_MPTCP);
Enter fullscreen mode Exit fullscreen mode

Even the errors give you the diagnosis: kernels older than 5.6 return EINVAL, a kernel built without MPTCP gives EPROTONOSUPPORT, and if MPTCP was turned off via net.mptcp.enabled you get ENOPROTOOPT. An application can also ask TCP_IS_MPTCP whether the connection really ended up as MPTCP.

The operational meaning is inconvenient: if the service you run doesn't carry that line in its source, flipping the sysctl changes nothing. The kernel documentation lists several techniques for forcing an application into MPTCP — LD_PRELOAD (mptcpize), eBPF (mptcpify), SystemTAP, GODEBUG on the Go side — and these four are the ones that matter in practice.

The first is systemd's socket activation. In .socket units, SocketProtocol= takes udplite, sctp or mptcp, and the last one opens the socket with IPPROTO_MPTCP. Mind the version trap: this support arrived in systemd 257, and my server on 255 doesn't have it. Why collecting systemctl --version across the fleet is the first step of a runbook is something we also discussed in the previous article, about soft-reboot; the same discipline applies here.

The second is mptcpize, which ships with the mptcpd package. In its man page's own words, it is a program that enables MPTCP on existing legacy services: mptcpize run prog runs the target program forcing MPTCP socket usage instead of TCP, while mptcpize enable unit updates the systemd unit file so the service runs under that launcher.

The third is changing the application. The option that sounds like the most work is actually the cleanest one when the service is yours: a one-line protocol change.

The fourth partly refutes my own thesis: Go. The language added MPTCP support in 1.21, but only used it if the application explicitly asked. Go 1.24 changed the default — GODEBUG=multipathtcp is now 2, meaning MPTCP is enabled by default on listeners, and you need 0 to get the old behaviour back. So a server built with Go 1.24 or later may be listening with MPTCP without anyone doing anything. "Nobody uses it" is increasingly true only of the client side.

The second door: path manager, endpoints and limits

Say the socket was opened with MPTCP. That alone still doesn't give you multiple paths; for additional subflows to be created, the path manager has to know what to do.

There are two kinds: the in-kernel manager applies the same rules to all connections and is configured with ip mptcp endpoint, while the userspace manager (mptcpd) lets you write per-connection rules. One note on how current this is: the net.mptcp.pm_type sysctl that selects between them is deprecated as of 6.15 and replaced by net.mptcp.path_manager. My 6.8 only has the old one — the name you read in the docs may not exist on your kernel.

The endpoint flags are the heart of it. signal means the address will be announced to the peer with an ADD_ADDR option; in the man page's words, a server would typically be responsible for this. subflow means that, if the limits allow, an additional subflow will be created using this address as the source after the connection is established; a client would typically do this. backup marks a subflow as backup: the remote peer only sends data on it when all non-backup subflows are unavailable. fullmesh tries to create a subflow towards every address the peer announces.

A laminar flag was added to that list recently: the patch landed on 26 September 2025 and shipped in 6.18. Its job is to create subflows towards the addresses the peer announces without leaving source address selection to the routing tables, which makes configuration easier when you don't know the announced address in advance; fullmesh takes precedence over it. The help output of iproute2 6.1 on my server still lists signal | subflow | backup | fullmesh, so pay attention to which version's documentation you're reading.

Then come the limits, and here's the real surprise:

$ ip mptcp limits show
add_addr_accepted 0 subflows 2
Enter fullscreen mode Exit fullscreen mode

add_addr_accepted 0 answers the question "how many addresses will I accept if the peer announces some?", and the default is zero. So even if your server announces addresses properly, nothing happens on the client side. Most people who say "I enabled MPTCP but it doesn't work" are standing exactly here; there is nothing wrong with the kernel, the policy is empty.

One more detail: passing dev with the interface name when adding an endpoint matters. The man page warns about it explicitly — without that binding, you may need extra IP rules and routes for packets from that source address to reach the right interface. On a multi-link machine, skipping this means the second subflow tries to leave over the first link, and you end up with one path wearing two names.

Concretely, the simplest setup between a server with two addresses and a client on a single uplink looks like this:

# server: announce the second address to the peer
ip mptcp endpoint add 203.0.113.10 dev eth1 signal
ip mptcp limits set add_addr_accepted 1 subflows 2

# client: allow an extra subflow towards the announced address
ip mptcp endpoint add 192.0.2.20 dev wwan0 subflow
ip mptcp limits set add_addr_accepted 1 subflows 2

# verification on both sides
ip mptcp endpoint show
ip mptcp limits show
Enter fullscreen mode Exit fullscreen mode

The asymmetry is deliberate: the announcing side uses signal, the side that creates the subflow uses subflow. Forget to lift the limits off zero on either side and every command returns successfully while nothing happens.

What actually happens on the wire

To see why these doors are so brittle, look at the handshake; RFC 8684 describes the mechanism plainly.

The first connection opens with the MP_CAPABLE option: the client states its intent in the SYN, the server sends its key in the SYN/ACK, and the client sends both keys together in the third packet. Those keys are never sent in the clear again; each end hashes its key with SHA-256 to produce a 32-bit token, and the connection is identified by that token from then on.

Second and subsequent paths join with MP_JOIN. The new subflow's SYN carries the token that answers "which connection am I joining", along with a random nonce; the peer replies with an HMAC derived from the exchanged keys. So an additional path is authenticated with the keys from the initial handshake, which is what stops an off-path attacker from adding a subflow to your session. Don't expect more than that: the keys travel in the clear during the initial handshake, and the goal the RFC states is for MPTCP's security to be "no worse" than today's TCP. Against someone who can observe the path, your guarantee is the same as plain TCP's.

Address announcements travel in ADD_ADDR, and those are signed with an HMAC too. The use case the RFC highlights belongs squarely to our world: when a NAT prevents setup in one direction, an address is announced so the other side can connect instead.

The mechanism that stitches data across paths is the DSS option, which maps subflow-level sequence numbers onto connection-level ones. It has an optional checksum whose purpose is interesting: detecting whether a middlebox unaware of MPTCP has adjusted the payload. On Linux it is off by default (net.mptcp.checksum_enabled = 0), a trade made in favour of performance. The RFC is blunt here: without checksumming, corrupt data may be delivered to the application if a middlebox alters segment boundaries or content, so checksumming is recommended unless you know the path contains no such devices. If you're speaking MPTCP over an internet path you don't control, don't skip that line.

And the fallback rule: if the SYN carries MP_CAPABLE but the SYN/ACK doesn't, the peer is treated as incapable; if the option is missing from the third packet, the session must fall back to plain single-path TCP. That strictness isn't a shortcoming — it is a deliberate choice for compatibility with middleboxes that drop TCP options.

The third door: the peer and the boxes in between

MPTCP requires end-to-end agreement. If the far end or any middlebox in between doesn't support it, the documentation says the connection is "downgraded" to plain TCP and continues on a single path. The application doesn't even notice — which is good design, but if you don't measure it you'll be fooling yourself.

The kernel is defensive about this — as long as it's new enough. net.mptcp.blackhole_timeout (3600 seconds by default) disables MPTCP on active sockets for a while when a firewall blackhole is detected, and the period grows exponentially if the problem recurs; net.mptcp.syn_retrans_before_tcp_fallback says how many SYN attempts carrying MPTCP options are made before falling back to TCP, defaulting to 2.

Now look again at my sysctl net.mptcp output above: neither of them is there. Blackhole detection arrived in 6.12 and the SYN retransmission knob in 6.14, so my 6.8 has neither reflex; path_manager is 6.15's work. The first thing to run when reading an MPTCP article is uname -r — the documentation doesn't always describe your kernel.

Knowing the reflex matters because on 6.12 and later, if your test once traversed a bad path, MPTCP staying off in later attempts may not be a configuration mistake. The place to look then is the counters: MPTcpExtBlackhole and MPTcpExtMPCapableSYNTXDisabled exist exactly for this. Fiddling with settings before measuring means chasing a bug that isn't there.

On the firewall side, expectations need correcting too. An additional subflow is a new TCP connection as far as the network is concerned: a different source address, usually a different interface. If your rule set only knows the address the first connection came from, the second path dies before it is born. net.mptcp.allow_join_initial_addr_port defaults to 1 and lets the peer send join requests to the address and port used by the initial subflow, which makes the narrowest scenario — no extra ports opened — possible. If the announcement goes unacknowledged, net.mptcp.add_addr_timeout steps in: the value (120 seconds by default) is the maximum retransmission timeout for ADD_ADDR, while the actual interval is derived from the connection's estimated round-trip time; set it to zero and it is never retried.

There's a sneakier obstacle on the server side, and it usually shows up on the first attempt: because MP_JOIN arrives with a new four-tuple, an L4 load balancer or ECMP in front may well send it to a different backend. That machine doesn't hold the token, so the join is refused and the MPTcpExtMPJoinNoTokenFound counter climbs. If you're deploying MPTCP behind a load balancer, you have to guarantee that every subflow of a session lands on the same backend.

What to expect and what not to

On the bandwidth side, expectations need to be set correctly. The kernel documentation lists aggregation as an explicit use case: using multiple paths at the same time for higher throughput, for instance combining a fixed and a mobile network to send files faster. So saying "it doesn't add up" would be wrong. But addition isn't a guarantee either: the gain depends on the latency and loss profile of the paths, on the receive buffer, and on head-of-line blocking. Combine two wildly dissimilar links and the total looks less and less like the arithmetic sum — which is also why the same list keeps "best network selection" as a separate item.

The resilience side, by contrast, is far more concrete. A subflow created with the backup flag waits quietly as long as the primary path works, and carries the session without a break when the primary goes down. How fast that switch happens is governed by net.mptcp.stale_loss_cnt: the number of retransmission intervals before a subflow is declared stale, defaulting to 4, with lower values making active-backup switching faster according to the documentation.

A little-known detail on this side is net.mptcp.close_timeout (60 seconds by default), the "make-after-break" timeout in the documentation's own words. As long as the application makes no close or shutdown syscall, the socket keeps its state for that long after the last subflow is removed, and only then moves to TCP_CLOSE. So a connection can sit path-less for a while. For it to carry on, the peer has to hold state too and a successful MP_JOIN has to arrive from the new address; if you can establish the second path before the break, you've already done better.

So if you buy MPTCP as a "bandwidth combiner" you're likely to be disappointed; look at it as "redundancy that doesn't drop the session" and it's the right tool.

Don't talk about it without measuring

The sneakiest thing about MPTCP is that everything looks normal when it isn't working. So learn the verification commands before you touch the configuration:

# list MPTCP sockets
ss -M

# watch subflow creation live
ip mptcp monitor

# client side: did I try, did the peer answer, did I fall back?
nstat -az | grep -E "MPCapableSYNTX|MPCapableSYNACKRX|MPCapableFallbackSYNACK|MPJoinSynAckRx"

# server side: did it reach me, did it complete, did it fall back, did a join arrive?
nstat -az | grep -E "MPCapableSYNRX|MPCapableACKRX|MPCapableFallbackACK|MPJoinSynRx|MPJoinNoTokenFound"
Enter fullscreen mode Exit fullscreen mode

Note that the counters are directional; getting this wrong is the most common way to misread them. The naming in the kernel source is explicit: MPCapableFallbackACK counts the server-side fallback and MPCapableFallbackSYNACK the client-side one. Likewise MPJoinSynRx increments on the side that receives MP_JOIN; on the client that creates the subflow, the counter to watch is MPJoinSynAckRx.

That yields three separate diagnoses. On a client, if MPCapableSYNTX is zero the application never attempts MPTCP — the first door. If there are attempts but MPCapableFallbackSYNACK is climbing, the peer or the path is refusing — the third door. If the connection is established as MPTCP yet MPJoinSynAckRx stays at zero, no additional subflow was ever born — the second door, meaning your endpoints and limits. On a server the same three questions are asked with MPCapableSYNRX, MPCapableFallbackACK and MPJoinSynRx; since my machine is a server, the meaningful zero there is MPCapableSYNRX.

Without that split, what you're doing isn't tuning, it's guessing.

You're not condemned to blindness inside the application either. The kernel exposes four socket options at the SOL_MPTCP level (value 284): MPTCP_INFO for connection-level information, MPTCP_TCPINFO for an array of per-subflow tcp_info, MPTCP_SUBFLOW_ADDRS for the subflow addresses, and MPTCP_FULL_INFO for subflow info plus the tcp_info array plus the connection-level information together. If you write your own service, exporting "how many paths am I running on" as a metric is a few lines of work — and it's the most honest way to see on a dashboard whether one of your links has quietly died.

If your policy needs to differ per connection, that's the sign the in-kernel manager is too narrow; that's when you move to the userspace manager, mptcpd. Unnecessary complexity for most setups, but the right place if you're going to write a rule like "sessions from this customer must not use the second link".

Is this really your problem?

MPTCP's biggest constraint is organisational: you need to control both ends. If you want multi-link resilience between your own client and your own server, it's the right tool. But if you want to split traffic heading for arbitrary internet services, you're left with plain TCP unless the far end speaks it.

So the first question to ask is whether the layer of your problem really is the transport layer. Sharing traffic leaving a branch office across two links, or picking a path based on link quality, are solved in the routing layer, not the transport layer; the approach in my article about path selection with SLA probes in SD-WAN is that domain's answer. MPTCP is about a single session surviving over two paths — a different problem entirely.

If you've decided to try it, an ordered checklist helps:

  • Kernel and tools: sysctl net.mptcp.enabled and the flag list in ip mptcp help.
  • Application: IPPROTO_MPTCP in the source, SocketProtocol=mptcp on systemd 257+, or mptcpize.
  • Policy: signal endpoints on the server, subflow on the client; lift subflows and add_addr_accepted off zero with ip mptcp limits set.
  • Routing: a dev binding on every endpoint, and source-based routes where needed.
  • Verification: the output of ss -M and the MPCapable* counters — see both the attempt and the fallback.
  • Firewalls: on 6.12+ kernels, if blackhole detection has kicked in, wait out blackhole_timeout or repeat the test over a clean path.
  • Persistence: ip mptcp endpoint and limits are runtime settings and vanish on reboot. Don't call it done until they're wired into networkd/netplan configuration or a unit that runs at boot.

Being enabled is not being used

The real lesson MPTCP left me with isn't about the protocol. In modern systems, a feature being "supported" no longer means it is working; the support chain runs from the application to the kernel, on to the far end and through every firewall in between, and any link in that chain can quietly revert to the old behaviour.

That silence shouldn't be underestimated either. The system doesn't raise an error; it simply picks the less capable path and tells you nothing. I read that not as a flaw but as a transfer of responsibility: showing that a feature actually works is now the job of whoever measures. The 63 zeroed counters on my server are the proof — the kernel has been doing its part for years; the missing party is me.

Official Sources

Top comments (0)