Inside Boxr’s pure-Rust UserNet: TAP frames, ARP, DNS, TCP proxying, and the trade-offs that keep it beta.
Rootless containers have a networking problem that looks simple until you follow the packet.
Creating an isolated network namespace is only the first step. The application inside the container still expects DNS, outbound TCP, port forwarding, and behavior close enough to a normal Linux host that tools such as curl or a package manager do not care how the connection was built. Meanwhile, the runtime cannot rely on host root privileges to create and configure everything the traditional way.
The common answer is a separate user-mode networking helper. That is a good answer, and mature tools have earned their place. But while building Boxr, a rootless Open Container Initiative (OCI) engine written in Rust, I wanted to explore a second path: could the runtime carry a small, understandable networking path inside its own binary?
That experiment became UserNet, an embedded Layer 2 through Layer 4 data path written in Rust. It is not a claim that a young implementation can replace every mature networking tool. It is a working fallback, a learning surface, and one of the most interesting pieces of Boxr to review.
Rootless containers still need a way out
Inside a private network namespace, a process can have its own interfaces, routes, and loopback device. That isolation is valuable, but it also cuts the process off from the host network.
A rootful runtime can create bridges, virtual Ethernet pairs, routing rules, and network address translation with host-level capabilities. A rootless runtime has to work within a tighter boundary. It needs to translate traffic without assuming it can reconfigure the host.
Boxr exposes several network modes because there is no single correct answer for every machine:
Auto uses pasta when it is installed, then falls back to UserNet.
UserNet selects the embedded Rust path explicitly.
Pasta uses the external rootless networking driver.
Bridge, host, and none support their expected isolation and connectivity models.
That ordering is deliberate. Shipping an embedded fallback makes a minimal installation useful, while keeping pasta available recognizes the value of a mature implementation. Engineering is usually better when “built here” does not become “must be used everywhere.”
A packet crosses five explicit stages
UserNet begins with a TAP interface inside the container’s network namespace. From the application’s point of view, it is using normal POSIX sockets and a normal Linux network stack. The Linux stack emits Ethernet frames through the TAP device, and Boxr reads those frames in user space.
The outbound path is:
Container application -> POSIX socket -> Linux network stack in the container namespace -> TAP interface (eth0) -> Boxr UserNet -> ordinary host UDP or TCP socket -> destination
The return path runs in reverse. Boxr receives bytes from a host socket, builds the protocol headers the container expects, computes checksums, wraps the packet in an Ethernet frame, and writes it back to TAP.
The implementation keeps the layers visible rather than hiding them behind a large abstraction. Ethernet parsing chooses between Address Resolution Protocol (ARP) and IPv4. IPv4 parsing then dispatches Internet Control Message Protocol (ICMP), User Datagram Protocol (UDP), or Transmission Control Protocol (TCP). That directness makes the code approachable, but it also makes every protocol assumption visible—and reviewable.
The small protocols are where trust begins
Before TCP can work, the container has to believe there is a network on the other side of its interface.
UserNet gives the container a small virtual network: the documented defaults use 10.0.2.15 for the container, 10.0.2.2 for the gateway, and 10.0.2.3 for DNS. When the container asks who owns the gateway or DNS address, the ARP handler returns a virtual gateway Media Access Control (MAC) address.
IPv4 handling validates the header shape and calculates the standard one’s-complement checksum for generated packets. ICMP echo requests to the virtual gateway receive matching echo replies. These pieces are small, but they matter: if ARP, lengths, byte order, or checksums are wrong, the higher layers fail in ways that are difficult to diagnose from inside the container.
DNS adds a more practical bridge. UserNet intercepts UDP queries sent to the virtual DNS address, forwards them through a host UDP socket to a resolver, then packages the response as UDP over IPv4 over Ethernet for the container. The application sees a DNS server. The host sees an ordinary UDP client. Boxr owns the translation between them.
This is also where defensive parsing becomes essential. The input is a byte slice, not a trusted Rust object. Every header needs length checks before fields are read. Declared payload lengths have to be bounded by the bytes actually received. Checksums must be generated over exactly the right pseudo-header and payload. Rust removes broad classes of memory errors, but it does not make protocol logic correct by itself.
TCP is the part that keeps the project honest
ARP, ICMP, and a DNS proxy are finite enough to explain in a few paragraphs. TCP is where the state space expands.
The current UserNet path recognizes connection setup and teardown flags, tracks sequence and acknowledgment numbers, opens unprivileged host TCP sockets for outbound destinations, and translates response data back into packets for the container. That is enough to demonstrate the architecture and exercise real traffic paths.
It is not the same as claiming a production-grade, general-purpose TCP implementation.
Real TCP behavior includes retransmission, duplicate acknowledgments, out-of-order segments, window scaling, backpressure, half-closed connections, resets, timing behavior, long-lived streams, and resource cleanup under failure. A happy-path handshake is the beginning of the work, not the end.
That is why Boxr is labeled beta. The most valuable review is not “nice project.” It is a packet trace that shows the state machine made the wrong choice, a reproducible teardown failure, a malformed frame that escapes a boundary check, or a workload that exposes blocking behavior.
I would rather make the boundary explicit than hide it behind the phrase “TCP/IP stack.” UserNet is useful today as an embedded path and a testable architecture. It still needs adversarial protocol review and sustained real-world use before anyone should treat it as mature network infrastructure.
In-process networking trades isolation for simplicity
Moving the networking path into the Boxr process changes the shape of the system.
The benefits are tangible:
One implementation language. Packet parsing, container lifecycle, and the surrounding runtime are all Rust.
A smaller installation surface. UserNet does not require an additional networking binary to exist on the machine.
A direct lifecycle. The engine owns the TAP loop and can stop it with the container.
An inspectable data path. A contributor can follow a frame from Ethernet parsing to a host socket without crossing a foreign-function boundary.
But “in process” is not automatically safer. A separate helper creates another component and another lifecycle to manage, yet it also creates a process boundary. With an embedded stack, packet parsing and the container engine share a fault domain. A panic, logic flaw, or resource-exhaustion bug can affect more of the runtime.
So the security question is not “Rust or C?” It is: what inputs cross the boundary, what privileges does each component hold, what happens on failure, and which design can be tested and maintained well enough for the intended threat model?
Boxr’s auto mode reflects that trade-off. It uses pasta when available and keeps UserNet as a zero-extra-dependency fallback. The goal is not purity. The goal is a rootless container that can reach the network while the architecture remains honest about its maturity.
The fastest path forward is to break it usefully
If you want to inspect the idea, the repository documents a direct UserNet path:
git clone https://github.com/kchaitanya863/boxr
cd boxr
cargo build --release
./target/release/boxr run --network usernet -p 8080:80 nginx:latest
Rootless networking depends on the host environment, including Linux user-namespace support and access to /dev/net/tun, so the most useful reports include the kernel, distribution, architecture, exact command, logs, and a minimal reproducer. A short packet capture is even better when the failure is in the data path.
The project recently shipped v0.1.44, which included Linux build and exec user-namespace fixes alongside refreshed launch assets. Boxr remains a young, single-maintainer beta, and that is precisely why focused contributors can have disproportionate impact now.
Three areas would benefit most from experienced eyes:
TCP behavior under unhappy paths. Retransmission, teardown, long-lived connections, and sequence handling deserve adversarial tests.
Isolation boundaries. Is an in-process stack the right fault domain, or should the design preserve the same Rust implementation behind a process boundary?
Packet-loop architecture. The current path is easy to understand; the next challenge is concurrency and throughput without making correctness impossible to reason about.
If those problems sound interesting, read the networking architecture, join the design discussion, or open an issue with a workload that fails. If Boxr runs on your setup, report that too—the boring compatibility matrix is what turns an experiment into infrastructure.
And if you want to follow a Rust container engine while the core design is still moving, star Boxr on GitHub. The next useful contribution might be a protocol test, a packet trace, or one carefully argued “this boundary should be somewhere else.”
Top comments (0)