How to Add Correlation IDs to Background Job Logs So You Can Actually Trace a Failure

A job fails at 2am. The alert fires. You open the logs and find a stack trace with no order ID, no user ID, no request context, just an exception and a timestamp. Somewhere upstream, an API request triggered this job, and somewhere downstream, three other jobs it queued are probably also going to fail, but nothing in any of these log lines connects them to each other. This is the exact moment a correlation ID would have turned a two-hour investigation into a five-minute one.

Here is how to actually add correlation IDs to a background job system, not just the concept, the specific steps that make it work once jobs start queuing other jobs.

Step 1: generate the ID at the earliest possible point

The correlation ID needs to originate at the very start of the request or event that eventually triggers background work, not inside the job handler itself. If a web request creates a job, generate the ID when the request comes in, before any business logic runs. If an event from another system triggers a job, generate the ID as soon as that event is received, or better, use an ID the upstream system already generated if one exists.

import uuid

def handle_request(request):
    correlation_id = request.headers.get("X-Correlation-ID") or str(uuid.uuid4())
    # ... rest of request handling

Preferring an existing header over generating a new one matters once you have multiple services in the request path. If service A already assigned a correlation ID and service B generates its own instead of reusing it, you've broken the chain at exactly the boundary where tracing across services matters most.

Step 2: pass the ID into the job payload explicitly

The correlation ID needs to travel with the job as data, not as some kind of ambient context that gets lost the moment the job crosses from the web process into a worker process. Add it as an explicit field on every job payload.

enqueue_job("send_confirmation_email", {
    "order_id": order_id,
    "correlation_id": correlation_id,
})

This step gets skipped more often than it sounds like it should, usually because the job payload was designed before anyone thought about tracing, and adding a field to every enqueue call feels like unnecessary boilerplate until the first incident where you need it and it isn't there.

Step 3: attach the ID to every log line inside the worker

Once the worker picks up the job, the correlation ID needs to end up in every single log statement that job produces, not just the first one. Most logging libraries support this through a context variable or a logger adapter that automatically injects a field into every subsequent log call within a scope.

import logging
import contextvars

correlation_id_var = contextvars.ContextVar("correlation_id", default=None)

class CorrelationFilter(logging.Filter):
    def filter(self, record):
        record.correlation_id = correlation_id_var.get()
        return True

Set the context variable once at the top of the job handler, and every log call inside that handler and any function it calls picks up the value automatically, without threading the correlation ID through every function signature by hand. Most mainstream logging setups, including the standard library logging module in Python, support this filter-based approach without needing a third-party dependency.

Photo by Ryunosuke Kikuno on Unsplash

Step 4: propagate the ID when a job enqueues more jobs

This is the step that determines whether the whole system actually works end to end. If job A enqueues job B, job B needs the same correlation ID, not a new one. Without this, you can trace the first hop of a chain and lose the thread the moment one job spawns another, which is exactly the scenario, one failure cascading into several, where tracing matters most.

def process_order(payload):
    correlation_id = payload["correlation_id"]
    enqueue_job("send_confirmation_email", {
        "order_id": payload["order_id"],
        "correlation_id": correlation_id,
    })

Audit every place in your codebase where one job type enqueues another and confirm the correlation ID is being forwarded, not silently dropped because the enqueue call was written before this pattern existed.

Step 5: make the ID searchable in whatever you use to view logs

A correlation ID is only useful if you can actually query by it. If logs are structured and shipped to something like Elasticsearch or a hosted logging platform, the field needs to be indexed, not buried inside an unstructured message string. If you're on something simpler, grepping a centralized log file for the ID works fine as long as every relevant log line actually contains it, which is the entire point of steps one through four.

Tools built around OpenTelemetry take this further with distributed tracing that automatically threads context across service boundaries, which is worth adopting once a correlation ID convention alone stops being enough for a system with many interacting services. The W3C Trace Context specification documents the standard header format this tooling is built around, and it's worth reading even if you're not using a tracing library yet, because the header format is a reasonable convention to adopt for your own correlation IDs.

Even without adopting a full tracing library, a manual correlation ID convention is worth keeping in place long term, since it degrades gracefully in exactly the situations where automated tracing tends to fall over, a service that hasn't been instrumented yet, a background job triggered by a cron schedule rather than a traced request, or a quick script run manually during an incident where nobody has time to wire up a tracer first.

Why this matters more once your queue is durable

A durable job queue means failed jobs get retried instead of vanishing, which is good for reliability and bad for debugging if you can't tell which retry attempt you're looking at or which original request kicked off the chain. We covered building that durability layer, keeping job state outside process memory so a crash doesn't lose work, in a longer piece on building a job queue that survives a server restart. Correlation IDs are the piece that makes debugging that durable system tractable once jobs start failing and retrying at scale, rather than staying a single clean happy path.

Where to start if you have none of this today

Don't try to retrofit correlation IDs across an entire system in one pass. Start with the request path or event type that generates the most support tickets when something goes wrong, add the ID at the entry point, thread it through the first hop of jobs it triggers, and confirm you can actually find a specific incident's full trail in your logs before expanding to the rest of the system. This kind of tracing infrastructure is a common piece of the backend work 137Foundry's engineering team handles for client teams once a job system has grown past the point where a single log line tells the whole story.

A good test of whether the work is actually done: pick a recent, real incident and try to reconstruct its full trail using only the correlation ID and your existing log search. If that takes more than a couple of minutes, or if the trail goes cold at some hop where a job triggered another job without forwarding the ID, that's the specific gap worth closing next, rather than a sign the whole approach needs rethinking.