Why ThreadLocal Fails Silently in Streaming Agents: Zero-Intrusion Tool-Call Tracing in LangChain4j

Nozokime — Sat, 10 Oct 2026 16:43:49 +0000

While adding real-time tool-call tracing to a QC analysis agent, we went down two dead ends before landing on a zero-intrusion design: one agent instance per request, plus closure capture. This post documents the full journey — and why both seemingly perfect solutions fail silently.

1. The Problem: Let the Frontend "See" What the Agent Is Doing

The setting: a quality-inspection analysis agent built with Spring Boot 4 and LangChain4j 1.14.1. A user asks "analyze the root cause of report R20260808001", and the model autonomously decides which tools to call — getReportByNo → getReport → getDefects → getProductionParams → searchKnowledge — before writing its conclusion.

The traditional UX: the user stares at a spinner, then receives a wall of text seconds later. The process is a black box.

Our approach: push the tool-call trace to the frontend over SSE, rendered as running/done cards, so users can watch the agent "work":

Three requirements:

When a tool starts executing → push a running event (with arguments)
When a tool finishes → push a done event (with the result)
No instrumentation code inside the tools themselves (zero intrusion)

The first two are easy. The third is hard — plus a blocker we didn't see coming: the SSE emitter is a per-request object, but tool execution happens deep inside the model call, across thread boundaries. How do you get the emitter into the callback?

2. First, Look at Where the Threads Are

To sum it up in one sentence: the emitter is born on the servlet thread, but has to be used on the executor thread — and potentially a tool thread-pool thread.

3. Dead End #1: ThreadLocal — Looks Perfect, Fails Silently

Your first instinct is almost certainly ThreadLocal: set it when the request arrives, get it in the callback. So elegant.

TraceContext.set(emitter);
agent.analyze(question);
// inside the callback:
AgentTraceContext.send(TraceContext.get(), "tool", payload);

It's tempting: tool code stays clean, and callbacks just call get(). But it dies — in two distinct ways.

Mistake A: dies today. Set it in the controller method (servlet thread), read it in a callback running on the streamExecutor thread → you get null. ThreadLocal is thread-local storage; it doesn't cross threads by design.

Mistake B: works today, dies silently tomorrow. Set it inside the executor lambda instead. In the current code, tools run sequentially and the model call is synchronous, so everything happens to run on that same executor thread — and it works. But that rests on two fragile assumptions:

Assuming tools always run sequentially — enable executeToolsConcurrently() and tools get dispatched to a separate thread pool; the callback runs on a pool thread → get() returns null
Assuming the model is always synchronous — switch to StreamingChatModel and callbacks run on async callback threads → same problem

A minimal repro (20 lines, runnable):

public class ThreadLocalDemo {
    static final ThreadLocal<String> CTX = new ThreadLocal<>();
    static final ExecutorService POOL = Executors.newSingleThreadExecutor();

    public static void main(String[] args) throws Exception {
        CTX.set("the emitter for this request");
        POOL.submit(() ->
                System.out.println("read: " + CTX.get())   // expected: emitter
        ).get();                                            // actual: null
        POOL.shutdown();
    }
}

The scariest part isn't the failure — it's how it fails: silently. No exception, no error log; the frontend just never receives events. You can set a breakpoint in the controller and everything looks fine, yet in the callback get() is always null — you spend an afternoon on it and start questioning your sanity.

4. Dead End #2: `InvocationContext.methodArguments()` — Framework Internals

While digging, you'll come across InvocationContext in LangChain4j's callback context. It looks like you can read method arguments from it, so the temptation is to smuggle the emitter through the argument list.

Don't. That mechanism is how the framework internally locates managed types like ChatMemory (managedParameters / LangChain4jManaged). Using it as a custom argument channel is undocumented behavior — it can break on any framework upgrade.

Lesson: just because a framework leaves a door open doesn't mean it's meant for you to walk through.

5. The Solution: One Agent Instance Per Request + Closure Capture

The core idea in one sentence: the context shouldn't live on a thread or inside framework internals — it should be instance state.

5.1 The Three Components

AgentFactory — builds one agent instance per request:

@FunctionalInterface
public interface AgentFactory {
    QcAnalysisAgent create(SseEmitter emitter);
}

@Bean
@Profile("cloud")
public AgentFactory cloudAgentFactory(QcAgentTools tools) {
    // Build the model once: connection pool, API key and the token-usage
    // listener all live on the model
    ChatModel model = OpenAiChatModel.builder()
            .baseUrl(...)
            .apiKey(...)
            .modelName(...)
            .listeners(List.of(new TokenUsageListener()))
            .build();

    // Build the agent once per request: the closure binds the emitter to it
    return emitter -> buildAgent(model, tools, emitter);
}

private QcAnalysisAgent buildAgent(ChatModel model, QcAgentTools tools,
                                   SseEmitter emitter) {
    return AiServices.builder(QcAnalysisAgent.class)
            .chatModel(model)
            .tools(tools)
            .beforeToolExecution(b -> {
                // official hook 1: execution starts
                Map<String, Object> payload = new LinkedHashMap<>();
                payload.put("name", b.request().name());
                payload.put("args", b.request().arguments());
                payload.put("status", "running");
                AgentTraceContext.send(emitter, "tool", payload);
            })
            .registerListener(new AgentTraceListener(emitter))  // official hook 2: execution done
            .build();
}

AgentTraceListener — the tool-completion hook:

public class AgentTraceListener implements ToolExecutedEventListener {

    private final SseEmitter emitter;   // captured by the closure, travels with the instance

    @Override
    public void onEvent(ToolExecutedEvent event) {
        Map<String, Object> payload = new LinkedHashMap<>();
        payload.put("name", event.request().name());
        payload.put("status", "done");
        payload.put("result", truncate(event.resultText()));
        AgentTraceContext.send(emitter, "tool", payload);
    }
}

Why the closure works: once the emitter is captured by the lambda, it travels with the agent instance — it doesn't depend on "which thread am I on right now". Whichever thread fires the callback, the emitter inside the closure is always the one for this request. ThreadLocal's "thread-bound" assumption is bypassed entirely.

5.2 The Cost Tradeoff: Build the Expensive Thing Once, the Cheap Thing Per Request

The model (expensive): connection pool, API key, token-usage listener — built once, bean-scoped
The agent (cheap): measured AiServices.builder().build() at a steady-state average of 0.27ms (500 warm-up iterations, then 5000 measured, JDK 17). A full analysis takes about 5.2s (from logs), so the build overhead is roughly 0.005% — negligible

One honest small number: the very first build in a fresh JVM took 352ms (internal class loading + JIT compilation). It's a one-time cost that only affects the first request. Warm it up at startup if you care; ignore it if you don't.

5.3 Proof of Non-Intrusion

QcAgentTools contains four tools, and the entire file is business logic plus plain log.info lines — not a single line of instrumentation. Compare this with the intrusive alternative: every new tool would need two hand-written push calls, every tool author would have to know about "tracing", and forgetting one line means silently losing events. With this design, adding a new tool means caring about business logic only — the tool code knows nothing about tracing.

6. Summary

ThreadLocal fails silently in a streaming agent because of thread hops: the emitter is created on the servlet thread but consumed on executor / tool-pool / async-callback threads. Both dead ends (ThreadLocal, InvocationContext.methodArguments()) try to carry request context across threads — one via thread binding, the other via framework internals. Neither is reliable.

The final design moves the context from "thread" to "instance": one agent instance per request, with the emitter bound to the instance through closure capture. Whichever thread fires the callback, the emitter in the closure is always the right one. The model is built once (expensive), the agent once per request (0.27ms, cheap) — zero intrusion, zero silent failures.

You can call this pattern Per-request Instance + Closure-captured Context. It applies beyond SSE tracing: audit logs, per-request rate-limit markers, traceId propagation — the same family of problems. A few production details worth copying: silently skip pushes when the client disconnects, use LinkedHashMap instead of Map.of (values can be null), and truncate results to 800 characters.

Full implementation: github.com/CKX3197273653/Nozomi (package mate10/src/main/java/org/mate/mate10/agent/, written against LangChain4j 1.14.1)

DEV Community: Nozokime