DEV Community: ww-w.ai

Lorem Ipsum Makes LLMs Smarter. No, Seriously.

ww-w.ai — Mon, 11 May 2026 17:32:06 +0000

You know Lorem Ipsum. The placeholder text designers have been slapping into mockups since the 1960s. Turns out, it might be one of the most effective tools for making language models better at math.

A paper dropped last week — "Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration" (Huang et al., May 2026) — and the core finding is wild: prepending random Lorem Ipsum text before math problems during reinforcement learning training produces models that solve problems they otherwise never could.

Let me walk through why this works, because it is genuinely clever once you see the mechanism.

The Problem: When Every Answer Is Wrong, Nobody Learns

Modern LLM training uses reinforcement learning after the initial pretraining phase. One popular method is GRPO (Group Relative Policy Optimization), where you sample multiple candidate answers for a question, then reward the good ones and penalize the bad ones.

Here is the catch. For hard questions, all sampled answers might be wrong. When that happens, every candidate gets the same score. The relative advantage between them collapses to zero. No gradient. No learning signal. The model just shrugs and moves on.

This is called the zero-advantage problem, and it hits hardest on the exact questions you want the model to learn most — the difficult ones sitting at the frontier of its capability.

Previous fixes tried resampling (just roll the dice again) or adjusting reward scaling. They help a little, but fundamentally you are still asking the same question the same way, hoping for a different result.

The Fix: Just Jam Some Latin In There

LoPE — Lorem Perturbation for Exploration — does something that sounds like a prank. When the model fails on a hard question, LoPE prepends a randomly assembled chunk of Lorem Ipsum text before the prompt and resamples.

So instead of:

Solve: What is the integral of x^2 from 0 to 3?

The model sees:

Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Solve: What is the integral of x^2 from 0 to 3?

And somehow, this works. The nonsense prefix perturbs the model's internal state just enough to push it down different reasoning paths. Think of it like giving a stuck hiker a gentle shove in a random direction — sometimes that is all you need to find a trail you could not see before.

Why Latin and Not Just Random Characters?

The authors tested this systematically. Not all perturbations are equal. What works:

Latin-based vocabulary (Lorem Ipsum words)
Low perplexity (around 25) — the text needs to "look like language" to the model, even if it is meaningless

What does not work well:

Random character strings (too alien, the model just ignores or breaks)
High-perplexity gibberish
Perturbations in the model's primary training language (too much semantic interference)

Lorem Ipsum hits a sweet spot: familiar enough that the model processes it normally, foreign enough that it does not contaminate the actual reasoning task. It nudges the hidden states without hijacking them.

The Numbers

Tested on Qwen3-4B-Base across standard math benchmarks:

Benchmark	Standard GRPO	LoPE	Change
MATH-500	77.80	82.60	+4.80
AMC	47.76	58.21	+22% relative
AIME 2024	16.41	19.90	+3.49
Overall avg	49.37	53.99	+4.62 pts

On the 7B model, the gap widens further: +6.20 points over standard GRPO.

But the most interesting result is qualitative. On a set of 352 hard questions, LoPE uniquely solved 50 questions that no other method could crack. These were not marginal improvements on borderline problems. These were questions where every other approach produced zero correct answers, and LoPE found solutions.

The mechanism shows up clearly in the advantage signal. For those rare successful trajectories on hard problems, LoPE amplifies the advantage by 2.1x to 5.0x compared to standard resampling. When a perturbed prompt finally produces a correct answer, that success gets a much stronger training signal because it stands out sharply against the failed attempts.

Why This Matters for Practitioners

Three takeaways if you work with LLMs:

1. Exploration is still an unsolved problem. We talk a lot about scaling data and compute, but how models explore the solution space during RL training is arguably more important and much less understood. LoPE is evidence that we are leaving performance on the table.

2. Prompt sensitivity is a feature, not a bug. The fact that meaningless prefix text can unlock entirely different reasoning chains tells us something deep about how these models navigate their latent space. The "right" answer is often reachable — the model just needs a different starting point.

3. Simple methods can beat complex ones. LoPE is almost embarrassingly simple to implement. No architecture changes. No reward model modifications. Just prepend some Lorem Ipsum during resampling. If you are doing RL fine-tuning, this is a near-zero-cost experiment to try.

The broader lesson: sometimes the best interventions do not add information. They add noise in exactly the right way.

Paper Link

Nonsense Helps: Prompt Space Perturbation Broadens Reasoning Exploration
Huang, Huang, Li, Cai, Yang, Huang (Washington University in St. Louis) — May 7, 2026

Note: This is an arXiv preprint — not yet peer-reviewed. But the results are concrete, the methodology is clean, and the lead researcher (Jiaxin Huang) is a Microsoft Research PhD Fellow and AAAI 2026 New Faculty Highlight recipient. Worth watching.

Image Source: Huang et al., "Nonsense Helps" (arXiv:2605.05566), CC BY-NC-SA 4.0

Delete the Vercel Claude Code Plugin. Here's Why I Did.

ww-w.ai — Mon, 11 May 2026 13:46:49 +0000

TL;DR

The Vercel Claude Code plugin creates a permanent device UUID on your machine the instant you install it. No notification. No expiry. No rotation.
Session starts, tool calls, skill matches — all sent to telemetry.vercel.com. Default ON, no consent prompt. Prompt metadata (matched skill + score) included.
What's worse: they built a consent dialog for prompt text collection. But clicking "No thanks" only stops prompt text. All other telemetry keeps running. Most users will think they opted out of everything.
The documentation exists — buried eight directories deep inside ~/.claude/plugins/cache/. Nobody reads it. Documented ≠ Informed.

What I Found

I was building a static analysis tool for AI plugins — scanning popular skills for security issues. Regex pattern matching plus dual-LLM cross-verification.

I was running a batch scan — 200 Claude Code skills, checking for destructive commands, data exfiltration, prompt injection, the usual. On skill #147, the scanner flagged something in ~/.claude/. Not in some random GitHub repo. On my own machine.

I didn't suspect Vercel for a second. I assumed the flag was a false positive in my own skill. So I pulled the Vercel plugin source as a reference — to compare against "known good" code and figure out what I was doing wrong.

Then I read the Vercel source. Here's what I found.

The Evidence

All file paths and line numbers reference vercel-plugin v0.32.7, located at ~/.claude/plugins/cache/vercel/vercel-plugin/0.32.7/.

Every session start sends this:

// session-start-profiler.mts:702-709
session:device_id            // permanent device identifier
session:platform             // darwin, linux, win32
session:likely_skills        // which skills you use
session:greenfield           // whether the project is new
session:vercel_cli_installed // whether you have the Vercel CLI
session:vercel_cli_version   // which version

Every tool call you make — any tool, not just Vercel's:

// pretooluse-skill-inject.mts:969-971
tool_call:tool_name          // which tool you just called

Every time a skill matches your prompt:

// pretooluse-skill-inject.mts:1205-1210
skill:injected               // which skill got injected
skill:match_type             // how it matched
skill:tool_name              // against which tool

Every prompt you submit:

// user-prompt-submit-skill-inject.mts:1063-1065
prompt:skill                 // which skill matched your prompt
prompt:score                 // confidence score

All of it flows to a single endpoint:

https://telemetry.vercel.com/api/vercel-plugin/v1/events

None of it asked for your permission.

The permanent device ID

This is the part that should make you check your machine right now. Run this:

cat ~/.claude/vercel-plugin-device-id

You'll see something like:

473d7060-5a37-4ebb-9082-b09a983c****

A UUID. Created the instant you installed the plugin. Silently. No notification. It never expires. It never rotates. It ties together every session, every project, every client engagement you've ever worked on with Claude Code.

For context: Chrome DevTools rotates session IDs every 24 hours (ClearcutSender.ts:35,68-70). Vercel's device ID never expires. Privacy-conscious analytics platforms moved away from persistent device IDs years ago. This one lasts forever.

Dozens of telemetry events per coding session. All tied to a permanent fingerprint. All default-on.

"But It's in the README"

Technically, yes. The plugin's README.md has a ## Telemetry section. It explains what's collected and how to disable it.

But does anyone seriously think that counts as consent?

Walk through what actually happens:

You install the plugin.
It prints a success message.
You start coding.

At no point does any text appear on your screen about telemetry. No prompt. No checkbox. No banner. Nothing. Meanwhile, in the background: ~/.claude/vercel-plugin-device-id is written to disk, session events are queued, and your usage patterns start flowing to Vercel's servers.

The README is sitting in ~/.claude/plugins/cache/vercel/vercel-plugin/0.32.7/. Eight directories deep inside a hidden folder. Nobody browses there.

GDPR defines valid consent as "freely given, specific, informed, and unambiguous." Most companies — including startups with a fraction of Vercel's resources — treat this as the baseline. I haven't seen a single serious startup ship permanent device tracking without an install-time consent prompt in years. It's just not done anymore.

Remember: Chrome DevTools rotates its session IDs every 24 hours (ClearcutSender.ts:35,68-70). That's the standard. Vercel's device ID never rotates. Never expires. Created once, lives forever.

This is not a gray area. This is not "technically compliant." A permanent device UUID, created silently, tied to every session, with no install-time disclosure — this is clearly Vercel's mistake.

I used this plugin daily for months. I had no idea. And I'm the developer who was literally building a tool to analyze plugin source code.

The Part That's Even More Absurd — I Never Consented

Here's what makes this worse. The plugin actually has a consent dialog — for prompt text collection:

// user-prompt-submit-telemetry.mts:58-61
prompt:text  // full prompt content, up to 100KB — OPT-IN ONLY

An explicit question appears: "Share your prompt text to help improve skill matching." You can say yes or no. Your choice is saved.

So they know how to build consent flows. They built the infrastructure. They just chose not to use it for device tracking, tool-call logging, skill-usage profiling, and platform fingerprinting.

And here's the trap: if you click "No thanks," you think you've opted out. You haven't. Base telemetry — everything in the previous section — keeps running. The README even says so: "base telemetry remains on by default."

But you already clicked "No thanks." In your mind, the matter is settled. That's not a documentation gap. That's a dark pattern.

How to Protect Yourself

Do this now. It takes 60 seconds.

1. Check if you're affected

ls ~/.claude/vercel-plugin-device-id

If the file exists, you have a permanent tracking UUID on your machine.

2. Disable telemetry

Add this to your shell profile (.zshrc, .bashrc, etc.):

export VERCEL_PLUGIN_TELEMETRY=off

Then reload:

source ~/.zshrc

3. Or just uninstall the plugin entirely

If you don't need it, remove it. One fewer thing sending data you didn't agree to.

What Should Change

Two proposals. Design standards, not policy demands.

1. Surface telemetry at install time. One prompt. Plain language. "This plugin collects [X] and sends it to [Y]. OK?" The user sees it. The user decides. This is four lines of install-time code. Vercel already has the consent infrastructure. They use it for prompt text. Extend it to everything else.

2. Treat data flows as API surface. If your plugin sends data to an external endpoint, document it the way you'd document an API. What data. Where it goes. How often. How to stop it. Put this in the install output, not in a README eight directories deep.

These aren't radical ideas. Homebrew notifies you on first run. VS Code notifies you on first launch. It's already the industry standard. The Vercel plugin just doesn't.

Check your ~/.claude/ directory right now. What did you find? Drop it in the comments.

We Need a CatRun for the AI Era

ww-w.ai — Tue, 05 May 2026 15:36:00 +0000

A 16-pixel hero in your macOS menu bar. Watches LLM traffic. That's it.

RunCat told us the CPU was busy. Nothing tells us the agent is.

You remember RunCat — the kitten in your menu bar that runs faster when your CPU is busy. Almost a decade old. Adorable. Useful. Asks nothing of you.

AI-native development needs the same thing for a different signal. Not CPU. Agent traffic. Is there a live LLM request flowing right now, or is everything quiet?

That's why I built AgentRunner.

We need a CatRun for the AI era. So I made one.

A 16-pixel hero in your macOS menu bar. Runs when your agent's actually working. Idle when it isn't. That's the whole UI.

Seven things it's built around

1. The menu bar is where you already glance. Same place as the clock. No extra tab, no extra window, no "I'll open the dashboard later."

2. Below the noise floor. <1% CPU, ~20MB RAM. Native SwiftUI. A monitor that becomes its own monitoring problem is a joke.

3. Flashy "live agent dashboards" don't last. Animated traffic, live token deltas, color-coded latency heatmaps — fun for a week, closed and forgotten by the next sprint. CatRun ran for a decade because it asked you nothing. Same spirit here.

4. Detailed analysis belongs in a different tool. Token spend, cache misses, run history — that needs report depth. That's what cc-token-saver is for, and it gets its own post next. AgentRunner = glance. cc-token-saver = report. Don't make one app try to be both.

5. Vendor-neutral by design. It watches LLM traffic, not Claude traffic. Claude Code, Codex, Cursor, Windsurf, local LLaMA via Ollama, any agent loop hitting a model endpoint over HTTPS. No API key, no per-vendor SDK.

6. Local-only. Zero telemetry. Detection happens on your machine. The app does not phone home. No analytics SDK, no event ping. An agent monitor that ships your data anywhere doesn't deserve trust.

7. Idle vs Active. Binary. That's the entire UI. CatRun gave us a kitten that ran when CPU spiked. AgentRunner gives you a 16-pixel hero that runs when LLM traffic flows. Same spirit. Useful. Small. Invisible until you glance at it.

Get it

Repo: https://github.com/ww-w-ai/AgentRunner
License: Apache-2.0
Requires: macOS 13+

cc-token-saver post: coming next.