DEV Community

Cover image for netchaos: testing network resilience in Go without leaving `go test`
João Paulo Gomes Rodrigues
João Paulo Gomes Rodrigues

Posted on

netchaos: testing network resilience in Go without leaving `go test`

If you've ever written a retry, a timeout, or a circuit breaker in Go, you've probably asked yourself: "does this actually work when the network starts failing?"

Most of the time the honest answer is: "I don't know, I've never really tested it." Testing network failure usually means spinning up a proxy like Toxiproxy, setting up infrastructure, or just trusting the code is right and hoping it doesn't break in production.

Trying to solve that problem in my own day-to-day work is what got me started on netchaos, a Go library for deterministic network fault injection, fully in-process — no proxy, no daemon, no external infrastructure.

The core idea

The inspiration came from testing/synctest, which the Go standard library introduced to virtualize time in tests. netchaos follows the same philosophy, just applied to the network layer:

If the stdlib decided to cover this layer too, this is probably how it would do it: deterministic, in-process, zero extra infrastructure.

In practice, that means simulated net.Conn and net.Listener implementations with configurable fault injection: latency, packet loss, network partitions — all seeded, so the same test fails (or passes) the same way every time, on any CI runner, in milliseconds.

func TestRetryOnPacketLoss(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        net := netchaos.NewNetwork(
            netchaos.WithPacketLoss(0.3),
            netchaos.WithLatency(50*time.Millisecond, 150*time.Millisecond),
            netchaos.WithSeed(42), // deterministic, reproducible failures
        )

        client := myservice.NewClient(net.Dial)

        err := client.FetchWithRetry(context.Background(), "resource-id")
        if err != nil {
            t.Fatalf("expected retry to succeed despite packet loss, got: %v", err)
        }
    })
}
Enter fullscreen mode Exit fullscreen mode

What netchaos is NOT trying to be

One deliberate design decision was to draw the project's boundaries clearly — partly because an earlier project of mine turned into a "framework trying to cover everything" and never shipped. This time, the scope is intentionally narrow:

Where it runs What it tests Setup cost
Toxiproxy (Shopify) External process, real TCP sockets Your whole binary against a degraded real network Low, but requires infrastructure (proxy + real network) running
gosim Source-translated program, custom runtime Full-program determinism: network, disk, goroutine scheduling High — rewrites how your entire program executes
netchaos In-process, plain library Your business logic against simulated network faults, inside a normal go test Minimal — swap a net.Dial call for a factory

In other words: it's not chaos engineering for production (that's Chaos Mesh's or Litmus's job), it's not aiming for whole-program determinism (that's gosim's or Antithesis's job), and it's not a replacement for real integration testing (that's still Toxiproxy's job).

v1 scope

To avoid scope creep from day one, v1 is deliberately locked down to the essentials:

  • Simulated net.Conn / net.Listener (TCP-shaped) with pluggable fault injection
  • Latency injection (fixed and ranged)
  • Packet loss (probabilistic, seeded/deterministic)
  • Network partition (drop all traffic between two simulated peers)
  • Seeded randomness for reproducible failure scenarios
  • Integration with testing/synctest for virtual time Packet reordering, full disk/syscall simulation, and UDP support are explicitly out of scope for now — on purpose.

Where it stands right now

The project is still in early development, with an unstable API that's subject to change — a good time to follow the design decisions or weigh in on the API shape before things settle. The repo has a docs/ folder with the full design documentation, including the roadmap and what's explicitly out of scope for v1.

Repo: https://github.com/jpgomesr/netchaos

If you work on distributed services in Go and you're tired of relying on luck to test retries, timeouts, and backoff, take a look — feedback and discussion on the API are very welcome.

Top comments (0)