In this article, we review how Langfuse parses JSON using web workers in the background. You will learn:
What is Web Worker?
json-parser.worker.ts
useParsedTrace.ts
What is Web Worker?
Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down.
A worker is an object created using a constructor (e.g., Worker()) that runs a named JavaScript file — this file contains the code that will run in the worker thread.
Worker
The Worker interface of the Web Workers API represents a background task that can be created via script, which can send messages back to its creator.
Creating a worker is done by calling the Worker("path/to/worker/script") constructor.
Workers may themselves spawn new workers, as long as those workers are hosted at the same origin as the parent page.
Learn more about Web Worker.
json-parser.worker.ts
In the json-parser.worker.ts in Langfuse codebase, you will find this comment.
/**
* Web Worker for background JSON parsing
*
* Parses large JSON data off the main thread using the iterative
* deepParseJsonIterative function to prevent UI blocking.
*/
import { deepParseJsonIterative } from "@langfuse/shared";
export interface ParseRequest {
id: string;
input: unknown;
output: unknown;
metadata: unknown;
}
export interface ParseResponse {
id: string;
parsedInput: unknown;
parsedOutput: unknown;
parsedMetadata: unknown;
parseTime: number;
}
self.onmessage = function (e: MessageEvent<ParseRequest>) {
const { id, input, output, metadata } = e.data;
const startTime = performance.now();
try {
// Parse with high limits since we're off the main thread
const parsedInput = deepParseJsonIterative(input, {
maxDepth: Infinity,
maxSize: 10_000_000, // 10MB
});
const parsedOutput = deepParseJsonIterative(output, {
maxDepth: Infinity,
maxSize: 10_000_000,
});
const parsedMetadata = deepParseJsonIterative(metadata, {
maxDepth: Infinity,
maxSize: 10_000_000,
});
const elapsed = performance.now() - startTime;
const response: ParseResponse = {
id,
parsedInput,
parsedOutput,
parsedMetadata,
parseTime: elapsed,
};
self.postMessage(response);
} catch (error) {
console.error("[json-parser.worker] Parse error:", error);
// Send back unparsed data on error
self.postMessage({
id,
parsedInput: input,
parsedOutput: output,
parsedMetadata: metadata,
parseTime: performance.now() - startTime,
error: error instanceof Error ? error.message : String(error),
});
}
};
Langfuse uses the web workers to parse large JSON. This deepParseJsonIterative function is imported as shown below:
import { deepParseJsonIterative } from "@langfuse/shared";
useParsedTrace.ts
When I searched for json-parse.worker. I found two instances:
In the useParsedTrace.ts, the following code is defined to create a worker
function getOrCreateWorker(): Worker | null {
if (typeof window === "undefined" || !window.Worker) {
return null; // SSR or no Worker support
}
if (!workerInstance) {
try {
// Next.js will bundle this as a separate chunk
workerInstance = new Worker(
new URL("@/src/workers/json-parser.worker.ts", import.meta.url),
);
workerInstance.onmessage = (e: MessageEvent<ParseResponse>) => {
const callback = pendingCallbacks.get(e.data.id);
if (callback) {
callback(e.data);
pendingCallbacks.delete(e.data.id);
}
};
workerInstance.onerror = (error) => {
console.error("[useParsedTrace] Worker error:", error);
};
} catch (error) {
console.error("[useParsedTrace] Failed to create worker:", error);
return null;
}
}
You will find that this is used in components/sessions/index.tsx
About me:
Hey, my name is Ramu Narasinga. Email: ramu.narasinga@gmail.com
Tired of AI slop?
I spent 3+ years studying OSS codebases and wrote 350+ articles on what makes them production-grade. I built an open source tool that reviews your PR against your existing codebase patterns.
Your codebase. Your patterns. Enforced.
Top comments (0)