DEV Community

Clint Mathews
Clint Mathews

Posted on

A Zero-Allocation Worker Pool for 10kHz Sensor Ingestion in Go

#go #performance #systemdesign

Part of the build log for PhotonicOps, a fully offline, air-gapped telemetry engine for silicon photonic biosensors. No cloud APIs, no internet dependency, Apple Silicon only. This post covers the worker pool sitting between the gRPC ingestion stream and the DSP handoff.

Short version: 10,000 frames/sec, one channel send on the hot path, 10 fixed workers, zero per-frame allocations via sync.Pool, backpressure instead of drops.

The Constraints

A photonic biosensor streams resonance wavelength shift (Δλ, picometers) at 10,000 samples/sec. That's one OpticalFrame roughly every 100μs over a gRPC client-streaming call. Two rules shaped the design:

  • Nothing stalls the receive loop. If the goroutine reading off the stream also does the work, a slow downstream consumer pushes backpressure straight into the TCP connection and the sensor.
  • No GC pauses on the hot path. At 10kHz, a few milliseconds of stop-the-world GC can silently drop a sample, and a dropped sample can hide the exact anomaly (a microbubble, a cell clog) the system exists to catch.

The Pool 

type FramePool struct {
    jobQueue  chan *pb.OpticalFrame
    wg        sync.WaitGroup
    forwarder *dsp.Forwarder
}

var frameSyncPool = sync.Pool{
    New: func() any { return make([]byte, 0, 1024) },
}

func NewFramePool(workers, queueSize int, forwarder *dsp.Forwarder) *FramePool {
    p := &FramePool{
        jobQueue:  make(chan *pb.OpticalFrame, queueSize),
        forwarder: forwarder,
    }
    for i := 0; i < workers; i++ {
        p.wg.Add(1)
        go p.worker(i)
    }
    return p
}
Enter fullscreen mode Exit fullscreen mode

Wired up in main.go as NewFramePool(10, 50000, forwarder): 10 workers, a 50,000-slot channel. At full throughput that's 5 seconds of buffering headroom before anything blocks.

The gRPC handler's entire job:

func (p *FramePool) Enqueue(frame *pb.OpticalFrame) {
    p.jobQueue <- frame
}
Enter fullscreen mode Exit fullscreen mode

One channel send. No allocation, no branching. If the channel fills, Enqueue blocks, which becomes backpressure through gRPC flow control back to the sensor. No silent drops. Non-blocking load-shedding is on the roadmap for genuine overload but isn't built yet.

Zero-Allocation Workers 

func (p *FramePool) worker(id int) {
    defer p.wg.Done()
    for frame := range p.jobQueue {
        buf := frameSyncPool.Get().([]byte)

        if err := p.forwarder.Push(frame); err != nil {
            log.Printf("worker %d: dsp forwarder push error: %v", id, err)
        }

        buf = buf[:0]
        frameSyncPool.Put(buf)
    }
}
Enter fullscreen mode Exit fullscreen mode

Reusable []byte scratch space, recycled by capacity instead of reallocated per frame. At 10,000 frames/sec, allocating small and letting GC reclaim is a GC storm in slow motion. sync.Pool turns it into pointer recycling.

Where the Frame Actually Goes

Each worker hands its frame to dsp.Forwarder.Push, which:

  1. Shards frames by sensor_id into per-sensor accumulators
  2. Buffers each sensor's stream into 100ms windows (1,000 frames each)
  3. Flushes full windows as a FrameBatch over a Unix domain socket, via gRPC, to a local Python DSP process

No TCP, no HTTP, no cloud. The Go to Python DSP boundary stays entirely on-box.

Why Fixed Size, Not Goroutine-per-Frame

go process(frame) per frame would be simpler. Two reasons it doesn't work here:

  • Unbounded goroutines under load means unbounded memory. A downstream hiccup (Python DSP pausing) shouldn't turn into an OOM in an offline clinical deployment with no one watching a dashboard.
  • A fixed pool bounds lock contention. dsp.Forwarder.Push holds a mutex over the per-sensor accumulator map, so capping workers caps contention instead of letting it scale with load.

Worst case is "the send blocks", not "the process dies."

What's Next

Phase 5 (Ingestion Hardening) on the roadmap: per-sensor ring buffers (today's is a single global buffer, not keyed by sensor), non-blocking load-shedding for Enqueue, and a /metrics endpoint so pool saturation is observable instead of inferred from pprof.

For anything that has to sustain a hard real-time ingestion rate, keep the hot path allocation-free and non-blocking, push everything else behind a bounded channel, and let a fixed pool of workers apply steady, predictable pressure downstream.

Top comments (0)