DEV Community: Felipe Maschio

Masking a JVM thread dump without breaking the analysis

Felipe Maschio — Tue, 08 Sep 2026 02:21:26 +0000

A few weeks ago I posted on r/java about detecting virtual thread pinning from a plain thread dump. The technical discussion was fine. The comment that stuck with me had nothing to do with pinning: "the problem isn't taking the dump, it's that uploading a production dump to a third-party site is against policy". Not a preference. Policy. And the person saying it was right.

I run ThreadMine, a hosted thread dump analyzer, so that comment describes my own product. This article is about what I built in response: a small open-source CLI that rewrites the dump on your machine before it goes anywhere, in a way that keeps the analysis working. It's called tm-anon, it's MIT, and it works with any analyzer, mine included. The interesting part isn't the tool, it's the constraint: you can't just strip everything, because the analyzer needs most of what's in there.

What a thread dump actually leaks

Take one line from a real dump:

at com.acme.payment.LedgerService.applyEntry(LedgerService.java:95)

Package, class, method, source file. Multiply by a few hundred frames and you have a map of your codebase: module names, the vendor you integrate with (com.acme.payment.gateway.AcquirerClient), which parts of the system talk to which. Thread names are worse, because people put things in them. pgto-worker-3 is harmless; sync-tenant-acme-prod or a thread named after a request path is a customer identifier sitting in a text file.

Two dialects leak a lot more than the classic jstack output. An OpenJ9 javacore carries the full command line with its -D properties, the complete classpath, local file paths, monitor tables listing your class names, and the loaded-class list. And the JSON dialect of jcmd Thread.dump_to_file (JDK 21+) has a container field per thread group that holds the toString() of the executor or StructuredTaskScope owning it, something like com.acme.batch.LedgerScope@4f2b1a. It's one of the few places where one of your classes names itself outside a stack frame, and it's easy to miss if you only think in terms of frames.

Why "mask everything" doesn't work

A thread dump analyzer, any of them, doesn't read your code. It reads structure and public names. The detectors that matter key on things like java.util.concurrent.ThreadPoolExecutor.getTask to tell an idle worker from a stuck one, on http-nio-8080-exec- and ForkJoinPool-1-worker- to group pool threads, on lock addresses to build the deadlock graph, on thread states, and on the -N suffix of a pool thread to count how many workers share a prefix. Replace those with opaque tokens and starvation detection, pool grouping and deadlock analysis all go quiet at once.

So the rule that came out of reading the detector code is short: everything the JDK or a known framework put in the dump stays byte for byte, and everything that's yours becomes a token. The list of what stays, the allowlist, is a JSON file in the repository, with the rationale for each family written next to it, so a reviewer can disagree with an entry and rebuild. That list is the whole design; the rest is plumbing.

How a name becomes a token

Each identifier is hashed with HMAC-SHA256 under a 256-bit key that lives in a local vault file, and the first 40 bits become the token. Same key, same name, same token, in any dump, forever. That property is what keeps multi-dump comparison working after masking: the thread you were watching on Monday has the same token on Tuesday.

Keyed HMAC rather than a plain hash is not a stylistic choice. Java identifiers are guessable. SHA-256("com.acme.OrderService") falls to a dictionary attack in seconds, because the attacker can hash candidate names and compare. With a keyed hash, whoever doesn't hold the vault key can't even test a guess. The vault also stores the reverse map, and tm-anon init --encrypt seals it with a passphrase (PBKDF2, 600,000 iterations, then AES-256-GCM, both straight from the JDK), so a stolen laptop image is useless without it.

The token grammar has a shape, p…x… for a package segment, C…x… for a class, m…x… for a method, t…x… for a thread, and that shape was chosen against the analyzer's own regexes. Each package segment gets its own token so package.Class.method( still parses and a flame graph still groups by package. The numeric suffix of a pool thread survives, so pgto-worker-1 becomes t426f3xd05a4-1 and the analyzer still sees a pool of N. The x in the middle keeps the token from ever looking like a hex address or a request id, which is a real thing one of the detectors checks for.

The flow, from the point of view of someone who has never run it

You create a vault once per project:

$ tm-anon init
Created vault: /home/you/work/tm-anon-vault.json

Then you mask the dump. This writes a new file next to the original:

$ tm-anon mask payments-prod.txt
masked  payments-prod.txt -> payments-prod.anon.txt
lines:  78 preserved, 10 tokenized, 22 stripped, 0 redacted
verify: PASS - no identifier survived and the structure is intact.

The verify line is the part I'd point a reviewer at. Before writing anything, mask hands its own output to a separate verifier that re-derives every identifier from the masked text instead of trusting the rewriter, and refuses to write the file if anything that isn't a token, an allowlisted name or pure structure survived. A half-masked dump can't leave the machine by accident, because it never gets written.

This is what the same frame looks like on each side:

at com.acme.payment.LedgerService.applyEntry(LedgerService.java:95)
at pb536bxc27ec.pc2564xde165.pd98ecx128d7.Cfcbfdx33dfc.m65719x51697(Cfcbfdx33dfc.java:95)

JDK frames, lock addresses, thread states, cpu= and tid= fields and blank lines are untouched, because blank lines are what delimit threads and an analyzer that loses them loses the thread count. Lines the analyzer ignores anyway but that leak names for free, like Locked ownable synchronizers, are removed and replaced by a marker so the structure doesn't shift. A line no rule recognizes is redacted, never passed through.

You upload the .anon.txt, run the analysis, download the report, and bring the names back locally:

$ tm-anon unmask threadmine-report.html
Wrote threadmine-report.unmasked.html

unmask treats its input as opaque text and rewrites every token it finds, so it works on an HTML report, an export JSON, a CSV, or a paragraph an LLM wrote about your dump. The tokens come back inside prose too, which is why the grammar is deliberately distinctive: "the bottleneck is Cfcbfdx33dfc" turns back into "the bottleneck is LedgerService".

Does the analysis survive?

That was the question I couldn't answer when I replied on Reddit, so I measured it. I took the dumps in the test corpus (17 at the time), ran each one through the analyzer twice, original and masked, and compared the output: same set of detected problems, same severities, same health score on every one. Two edge cases behave differently and they're written down in the README rather than hidden, for example an application frame that today matches a detection pattern by accident (a fragment without a package, like Consumer.receive) stops matching once tokenized. The honest claim is "equivalent analysis", not "byte-identical".

The corpus is also where the tool caught its own worst bug before a user did. JDK 24 changed the text dialect of jcmd Thread.dump_to_file: where older JDKs print a lock as <0x000000061fcc9b90>, JDK 24 prints <com.acme.payment.LedgerLock@6e8cf4c6>. The rule "the contents of angle brackets are an address, keep them verbatim" was correct for every dump I'd looked at and was quietly handing over a class name on the newest JDK. It surfaced while building the test corpus for that dialect, and there's now a test that masks every fixture and runs the verifier over the result, so it can't come back unnoticed. I'd rather tell that story than pretend the first version was right.

What a security team can actually check

"We promise the tool doesn't upload anything" is worth nothing to the people who blocked the upload in the first place. What they can act on is a claim they can verify in an afternoon, so the repository is built around three of those.

The jar has no network code, and a test enforces it: it scans the production sources, the compiled bytecode constant pools and pom.xml for java.net, socket channels, javax.net, jdk.net and any HTTP dependency, and fails the build on the first hit. There is one dependency in the whole project, JUnit, at test scope, so there's no transitive supply chain to audit. And the allowlist that decides what stays verbatim is a readable JSON file, not a heuristic buried in code.

The threat model is written as "here is what this does not protect you from". A masked dump still reveals which frameworks you use, the shape of your thread pools and the depth of your package tree, and dumps masked with the same vault are linkable to each other by design. Whoever holds the vault file can reverse everything. This is pseudonymization reversible by its owner, not irreversible anonymization, and the document says so, because the alternative is a security reviewer discovering it on their own.

Where it stands

Version 0.4.0 handles the whole HotSpot family (jstack from JDK 8 to 25, jcmd Thread.print, both text and JSON forms of Thread.dump_to_file, ThreadMXBean/VisualVM output, virtual threads), OpenJ9 javacore in a strip mode that removes whole sections rather than masking line by line, Azul Zing and GraalVM native-image dumps. Anything it can't classify is refused rather than half-masked. It ships as a jar that needs Java 21, and as native binaries for Linux, macOS and Windows for people who'd rather not have a JVM on the machine holding the dump.

Disclosure, once more and explicitly: I'm the founder of ThreadMine, the analyzer this tool was built to feed, and the page describing how the two fit together is here. The CLI itself is at github.com/maschiojv/threadmine-anonymizer, MIT, analyzer-agnostic. If your policy still says no even to a masked dump, I'd genuinely like to know what it would take; that thread on r/java is where this started, and it's still the best feedback I've had on the problem.

I made a Claude Code plugin that turns your project into a demo video

Felipe Maschio — Tue, 21 Jul 2026 21:37:41 +0000

You know how much work goes into making a decent product demo. I built a Claude Code plugin that does most of it for you.

Because it runs inside Claude Code, it already knows your project. It can identify the main features, run the app and build the film with one command:

/product-film

The output is a single HTML file, and that file is the film. Open it in Chrome and you'll get a start screen with one card for each cut. I usually ask for a full demo and a shorter loop for Product Hunt. Each card has its own ▶ Watch and ● Record buttons, and there's a theme-music picker that applies to whichever cut you record.

Hit Record and the film captures itself from beginning to end, then drops an MP4 into your downloads. Firefox exports WebM. There's no screen recorder, OBS, editor or timeline involved.

A few tracks are included, and you can upload your own audio. A friend discovered it doesn't even have to be music: he recorded a narration, uploaded the MP3 and used that instead.

The bare command is enough, but you can also direct the film in the same message: which flow to show, what to leave out and which screen deserves more time. Since it's still a conversation with Claude Code, the first cut doesn't have to be the final one. You can say, "Drop the pricing scene and hold longer on the deadlock," and it'll generate a new version. You only hit Record when you're happy with it.

If you want to see the result, the demo on ThreadMine's landing page was made entirely with the plugin. The short loop plays on the page, and clicking it opens the full version with music.

The biggest advantage over AI video generators is that product-film doesn't have to guess what your application looks like. Those tools can make slick videos, but getting them to reproduce a coherent sequence of your actual screens is another matter. You either accept a slightly fictional version of the UI or spend a lot of time feeding them screenshots and correcting the details when they drift.

Recording yourself clicking through the app works, but the result usually still needs editing. Hiring an editor costs money and takes time. Claude Code already understands the project, so it can skip most of that setup.

It also doesn't upload your application or captured screens to a separate video service. The plugin works from a real run of your app on your machine.

In my case, the app is ThreadMine, a thread-dump analyzer. The demo shows a dump being genuinely analyzed: health score F, 30 out of 100, with a real deadlock where DeadLockThread1 waits on DeadLockThread2, which waits on DeadLockThread1.

I didn't choose 30 because it looked dramatic. That's the number the engine returned. Paste the same dump into the live tool and you'll get the same F 30. The video can't show a result the product didn't actually produce.

Before creating a single frame, the plugin runs the real application with Playwright, clicks through the actual flow and captures the real screens and values. That's the source of truth. The film is then choreographed on top of it: a cursor easing into buttons, captions appearing after each action and a slow push-in on whatever deserves attention.

Every visual is a pure function of t. That means the film can seek to any point, verify itself beat by beat and record a take that begins and ends exactly where it should.

It's open source and MIT licensed. Installation takes two commands:

/plugin marketplace add alliah-tech/product-film
/plugin install product-film@product-film-marketplace

One practical tip: open /plugin, go to Marketplaces and enable auto-update. Claude Code doesn't turn it on by default for third-party marketplaces, and I'm still releasing updates.

I'd also run it with the most capable model you have and set effort to high. The difficult part isn't writing the code. It's reproducing the real interface without deciding to "improve" it along the way. Weaker models sometimes get creative and prettify parts of the UI, which breaks the one rule that keeps the result from becoming AI slop.

I'm Felipe. I built ThreadMine, the analyzer used in the example, and product-film, the plugin. Happy to answer questions in the comments.

Your Loom App Quietly Became a Thread Pool Again: A Field Guide to Virtual Thread Pinning

Felipe Maschio — Fri, 10 Jul 2026 21:54:18 +0000

The incident that taught me to respect pinning looked like nothing. A service freshly migrated to virtual threads, a load test that plateaued at about 420 requests per second no matter how much traffic we threw at it, CPU sitting at 9%, zero errors, zero warnings, nothing in the logs. The machine had 8 cores, and the one downstream HTTP call in the hot path took about 19 ms. Do the arithmetic: 8 × (1000 / 19) ≈ 421.

The service that was supposed to scale to millions of virtual threads was serving exactly one request per CPU core. Loom had quietly handed us back a bounded thread pool, and the code looked perfectly innocent. That failure mode has a name — pinning — and this is the field guide I wish I'd had that night: what it is, the two (and only two) things that cause it, what JDK 24 changed, and how to catch it before your throughput graph does.

What pinning actually is

A virtual thread doesn't own an OS thread. It runs on a small pool of platform threads called carrier threads — concretely, the workers of a dedicated ForkJoinPool living in a thread group named CarrierThreads, with default parallelism equal to Runtime.availableProcessors(). When a virtual thread blocks — on I/O, a lock, a queue — it normally unmounts: it saves its stack, steps off the carrier, and frees that carrier to run another virtual thread. That unmount is the entire trick that lets a handful of OS threads serve millions of virtual ones.

Pinning is when the unmount can't happen. The virtual thread blocks but stays mounted, and its carrier sits there doing nothing useful for the whole duration. One pinned carrier is a rounding error. But the default carrier pool is only as big as your core count, so if a hot path pins routinely, you pin every carrier at once — and then no virtual thread anywhere makes progress. That's not a slowdown; it's scheduler starvation, and from the outside it looks a lot like a deadlock. You can raise the ceiling with -Djdk.virtualThreadScheduler.parallelism=N, but that only delays the moment of exhaustion. It doesn't fix anything.

The two causes — and it really is just two

There are exactly two situations where the JVM cannot unmount a blocked virtual thread:

1. Blocking inside synchronized (JDK 21 through 23). Up to and including JDK 23, an object monitor is tied to the carrier thread that entered it. If a virtual thread blocks — or calls Object.wait() — while holding a monitor, the JVM can't move it off the carrier without breaking monitor ownership, so it pins. This is by far the most common cause in real code, because a blocking call buried inside a synchronized method is trivial to write and invisible at the call site. And the monitor doesn't have to be yours: synchronized inside a library, or inside the JDK itself, pins exactly the same way. ConcurrentHashMap.computeIfAbsent runs your mapping function under an internal bin lock — put a blocking call inside it and you've pinned a carrier without a single synchronized keyword in your own code.

2. Native frames. When a virtual thread has a native method (JNI) or a foreign downcall (the Foreign Function & Memory API) on its stack and it blocks, the JVM can't capture and restore the native frame, so it pins. This one has no synchronized to blame — and it is not fixed by JDK 24. It also hides in a place nobody expects: class initialization runs through native frames, so a blocking call inside a static initializer pins even on the newest JDKs.

Just as important is what's not on the list: ordinary blocking I/O through the JDK (Socket, InputStream, Files), BlockingQueue, ReentrantLock, CompletableFuture, Thread.sleep() — all of it was re-plumbed for Loom and unmounts cleanly. Pinning is a short, specific list, which is exactly why it's detectable.

The canonical bug

Nearly every real pin I've read in a dump is some flavor of a cache or rate limiter guarding a slow call with synchronized:

public class PriceService {
    private final Map<String, BigDecimal> cache = new HashMap<>();

    // Looks harmless. On JDK 21-23 it pins the carrier for the whole HTTP call.
    public synchronized BigDecimal lookup(String symbol) {
        return cache.computeIfAbsent(symbol,
            s -> httpClient.quote(s));   // <-- blocks while holding the monitor
    }
}

Every cache miss blocks on the network while holding the monitor. On JDK 21–23 that virtual thread pins its carrier for the entire round trip. Run a few hundred concurrent requests and you've pinned every carrier; the rest of the workload queues behind a monitor that never unmounts. That's my 420-requests-per-second incident in five lines.

What JDK 24 changed (JEP 491)

JDK 24 shipped JEP 491, "Synchronize Virtual Threads without Pinning". It reworked monitor ownership so the monitor is associated with the virtual thread itself rather than its carrier — which means a virtual thread can now unmount while blocked inside synchronized, while waiting to enter one, or while parked in Object.wait(). The most common cause of pinning simply goes away on JDK 24+, with no code change.

Two practical consequences:

On JDK 24+, the only remaining pins come from native frames — JNI, FFM downcalls, and class initialization.
The old detection flag -Djdk.tracePinnedThreads was removed in JDK 24. Don't ship runbooks that depend on it.

If you're on JDK 21–23, though, synchronized pinning is very much alive, and upgrading is often the single cleanest fix you can make.

How to catch it

On JDK 21–23 — the legacy flag. Run with:

java -Djdk.tracePinnedThreads=full -jar app.jar

The JVM prints a stack trace every time a virtual thread pins, and the frame annotated <== monitors:1 is the culprit. That one line is the whole diagnosis. Just remember this flag no longer exists on JDK 24.

Everywhere — JFR. Since JDK 21 the JVM emits a jdk.VirtualThreadPinned Flight Recorder event when a virtual thread blocks while pinned. It's enabled by default — but with a 20 ms threshold, so short pins are invisible unless you lower it. In JDK 24 the event got better: it's emitted for every pinning occurrence and carries the reason and the carrier's identity. Since native-frame pins still fire it, this is the detection you should wire into production:

java -XX:StartFlightRecording=filename=rec.jfr,settings=profile -jar app.jar
jfr print --events jdk.VirtualThreadPinned rec.jfr

From a thread dump. Plain jstack won't show you virtual threads at all. Use the virtual-thread-aware dump:

jcmd <pid> Thread.dump_to_file -format=json dump.json

It lists the carrier threads and the virtual thread mounted on each. A carrier in the CarrierThreads group that is blocked while its mounted virtual thread sits in a synchronized frame (or a native frame) is the visual signature of a pin. Count how many carriers show it versus your pool size — that ratio tells you how close you are to full starvation.

How to fix it

Swap synchronized for ReentrantLock. java.util.concurrent.locks.ReentrantLock is Loom-aware: a virtual thread that blocks on it, or while holding it, unmounts cleanly. This is the direct, version-independent fix.
Upgrade to JDK 24+. JEP 491 removes the synchronized pin entirely. Native-frame pins remain.
Don't hold a lock across an external call. Often the honest fix is structural: compute the value outside the critical section and only lock the map update.
For native/FFM pins, isolate the path. Run unavoidable blocking native calls on a dedicated platform-thread executor, or size the carrier pool so a few concurrent pins can't starve everything.

Here's the rewrite of the example. One trap to avoid: don't just move the blocking call into ConcurrentHashMap.computeIfAbsent — as noted above, its mapping function runs under an internal bin lock, and on JDK 21–23 you'd have rebuilt the same pin one layer down.

public class PriceService {
    private final Map<String, BigDecimal> cache = new ConcurrentHashMap<>();
    private final ReentrantLock lock = new ReentrantLock();

    public BigDecimal lookup(String symbol) {
        BigDecimal cached = cache.get(symbol);
        if (cached != null) return cached;
        lock.lock();                       // Loom-aware: unmounts if it blocks
        try {
            cached = cache.get(symbol);    // re-check under the lock
            if (cached != null) return cached;
            BigDecimal quote = httpClient.quote(symbol);  // blocks; carrier is freed
            cache.put(symbol, quote);
            return quote;
        } finally {
            lock.unlock();
        }
    }
}

This no longer pins anywhere — though it still serializes cache misses behind one lock, which is fix #3's territory: the next refinement is not holding any lock across the network call at all.

Why I ended up automating the read

Doing this analysis by hand — turn on a flag, reproduce, dump, find the carriers, match frames — is fine once. It's tedious by the tenth incident, and worse, half the tooling depends on remembering to enable something before the problem happens. So I built a tool that does the read on any thread dump you give it: it finds the carriers, checks what's mounted on each, flags the pinned ones with the offending frame, and reports pinned-carriers-versus-pool-size — the number that tells you whether you're one bad path away from starvation. It's ThreadMine; the web analyzer is free and takes a dump with no signup. Full disclosure: it's my project — I got tired of reading dumps by hand, so I automated the part I kept repeating. And a fair caveat: a dump is a snapshot, so for intermittent pinning, JFR is still the better signal.

If you want the deeper reference — carriers, JEP 491, the full detection matrix — I keep it updated here: virtual thread pinning.

Pinning is the one Loom failure mode that cancels your scalability story without a single error in the logs. The rules are short: only synchronized (pre-JDK 24) and native frames pin; detect with jdk.tracePinnedThreads on 21–23 and the jdk.VirtualThreadPinned JFR event everywhere; fix with ReentrantLock, an upgrade, or by not holding locks across slow calls. Know the shape, and it stops being invisible.

Felipe Maschio is the founder of ThreadMine, a free JVM thread dump analyzer that detects deadlocks, thread leaks, pool exhaustion, CPU spikes and virtual thread pinning.