The OpenTelemetry memory limiter cannot save an exhausted exporter queue
The OpenTelemetry Collector often runs beside an application or in a central telemetry tier. Its receivers accept data, processors transform it, and exporters send it to a backend. A memory limiter processor can shed load before the Collector runs out of memory, but it is not a hard memory cap. The official processor documentation warns that incoming data can consume memory before the limiter sees it. Exporter queues can also retain data while a downstream system is unavailable.
Place memory_limiter first in the pipeline so it acts before downstream processors accumulate data. Set its check interval and limits from measured workload, not from a guess about container size. The Collector needs headroom for receiver buffers, batches already in flight, and objects referenced by queues. A soft limit that leaves no room for those objects may trigger garbage collection repeatedly without restoring available memory.
When an exporter endpoint slows down, its sending queue can fill. Once the queue cannot accept another item, data is typically dropped rather than entering the exporter's retry loop. This is a backpressure event, not a signing or filtering failure. The troubleshooting guide recommends queued retry settings for enabled exporters and warns that a missing or misconfigured memory limiter can cause memory pressure.
Size queues against the failure you expect
The sending queue has a capacity and a batch size. Raising capacity can absorb a short backend outage, but it also increases the amount of memory held by live references during that outage. A queue that is too small drops telemetry early. A queue that is too large can push the Collector toward the same failure the memory limiter is trying to prevent.
Persistent storage can help with a Collector restart, but it changes the failure model. The file_storage extension can persist the sending queue, and its disk capacity and permissions become part of the availability plan. Persistent queueing does not make an unreachable destination receive data. It only preserves accepted data locally until the destination returns.
Monitor queue size, queue capacity, send failures, refused data, and dropped data for each signal and exporter. A dashboard that shows only CPU and process memory hides the important transition from healthy retries to refused input. Alert when queue occupancy stays high after a short outage or when drop counters increase.
Keep privacy controls ahead of the queue
Telemetry can contain request attributes, error messages, user identifiers, and infrastructure details. Redaction and attribute filtering should happen before data enters a queue that may retain it on disk. A temporary outage should not turn a short-lived trace into a long-lived local data store that the privacy review never considered.
Verify that the storage directory is not exposed through a web server, shared volume, or container image layer. Restrict file permissions to the Collector process identity. Encrypt the volume where the platform requires it, and purge a retired Collector's persisted queue as part of decommissioning.
Test an actual downstream slowdown
Use a staging destination that accepts connections and then delays responses. Send a controlled telemetry load and observe queue growth, refusal counters, memory, and recovery after the destination returns. Repeat with the persistent queue enabled and with a Collector restart. Confirm that the service's documented delivery behavior matches the operational expectation.
A memory limiter is one part of a capacity and backpressure design. It becomes useful when limits, queue sizes, persistent storage, redaction, and metrics describe the same failure plan.
Top comments (0)