DEV Community: Nikhil Verma

How I'd do engineering for the next year

Nikhil Verma — Tue, 14 Jul 2026 18:27:22 +0000

Until recently, an AI agent would do a few things, then stop and ask me questions, or ask me for the next steps. Somewhere in the last six months that changed: now it just runs, until it genuinely cannot go any further.

There was no lightning-bolt moment for me. It was a collection of moments, a slow creep. You're working along, and one day you catch yourself thinking: hang on... the task that took me some time a couple of months back was now completed almost autonomously by this AI agent. And then it happens again a few weeks later, with something bigger.

Here's what that creep did to my work: in roughly the last year, I've written fewer than a hundred lines of code by hand. I don't really write code anymore; I review it. And lately I don't even review every line; I sample it.

Since you probably don't know me: I've spent almost twenty years building web applications and user-facing products, several of those managing teams of ten-plus people, including products with millions of users at Bumble.

This post is about what changed between December 2025 and July 2026, where the trend is going, and how I'm changing the way I do engineering to get ahead of it. Compressed into one thought: the rules have changed, and people who think clearly and write clearly are going to matter more than ever.

Three eras in three years

First, AI was a good autocomplete and not much else. Then came about a year of closely-directed work, the Copilot era: genuinely useful, but you had to direct it tightly: look at these three files, change this function to behave like this, put this UI component here, follow that reference file. If you did that, it did a pretty good job. Honestly, it felt like things wouldn't change much from there.

Then, over a few weeks in late 2025, the frontier labs shipped a wave of models¹, and agents stopped needing step-by-step instructions. They started taking high-level intent and carrying it over hours of work.

Why? My hypothesis (and it is only a hypothesis) is that while humans were closely directing AI through the Copilot era, the AI companies were learning from that interaction. We were acting as quasi-agents, and the agents learned from us. The standard account from people who train these systems is different: reinforcement learning on verifiable rewards, meaning models trained against outcomes like "the tests pass" in sandboxed environments.² The two stories aren't mutually exclusive, and either way the result is the same: the agents stopped needing me between every step.

My job is a factory now

My work has turned into a car factory. If you hand-build a car, you inspect every piece, every gear. If you have smart workers on an assembly line, you direct instead of build, but you still check everything that comes off the line. The stage after that is where I live now: one agent makes, another agent checks, and my job is directing. I'm an orchestrator, there are minions doing the work, and I'm sampling their output and providing the taste.

Diligence doesn't disappear, it becomes random sampling: random lines, random areas, enough distribution to catch what usually goes wrong. The move that makes it compound: everything sampling catches gets converted into static enforcement (a lint rule, a code-review instruction, a pipeline check) so the same mistake can't happen again. Google's recent whitepaper lands on the same rule: "add a rule every time the agent does something it should not do again."³

Probably the most important thing I've learned this year: the loop you run depends entirely on what you're building.

Different context, different judgments

Before any work starts, a few knobs get set: how critical is this to the business (an experimental feature, by default, cannot be critical), how big is it, how many decisions are hiding inside it. Days-based estimation doesn't work anymore, because things happen in hours, but t-shirt sizing actually works now. You get a rough size and pick a path.

The throwaway path. For quick internal tools and interim artifacts, I've written zero lines and reviewed zero lines. I jump literally from my prompt to the output: prompt, agent works, I sample the output, go again. The code is never opened. I know how reckless that sounds; I'll come back to why I don't think it is.

The real-project path. For the codebase I actually live in, I'm more involved, because I don't want the models drifting in the wrong direction. The loop becomes: I do the rant, I do the plan, I break the plan into reviewable milestones. (The rant is literal: my typing never kept up with my thinking, so prompts start as speech through a dictation app called Handy.⁴)

The milestone loop is honestly a bit funny: the agent writes the code and opens a draft pull request against me, using my own account; I go in as a human, review my own pull request, leave comments in GitHub, and tell the agent I've left comments for it. Big features get built as vertical slices that fit together like Tetris, each slice testable on its own, the whole thing done when the slices line up, not as a waterfall where everything collapses at the end.

The floor under every path. Whatever the AI builds, a human actually has to use the feature. You are the taster. Someone made food; you taste it, you check whether it's tastefully done, whether it makes sense. This part is mandatory.

Between those extremes, review depth scales with stakes. Business-critical code, I read all of it. Non-critical, I read some. Isolated one-offs, sometimes none at all. Different context, different judgments.

Nobody reads every line

This is where I part ways with the official advice. Google's whitepaper, which agrees with almost everything else in this post, says plainly: review every line the agent produces that is going to ship.³

I don't think that's realistic, for two reasons.

The first: AI is really, really good at pattern recognition. Distill the patterns a reviewer should look out for, and one AI can impartially catch issues while another fixes them, in a recursive loop, combined with tooling we've had for years: static analysis, linters, type checkers, test runners, smoke tests, property-based testing, fuzzing. The agents can help you write those tools too. With good enough guardrails, you don't have to stress about reading every single line. It's a combination of tools, not one person sitting on a laptop with nothing else in place.

The second is more uncomfortable: realistically, no human being honestly reads every line of code they review. I know this is controversial, and most people will say they do. But almost every engineer I've worked with has admitted, at some point, that they didn't read everything. They sampled it, checked the conventions, looked hard at the critical parts they knew about, and approved. We've been sampling all along, just without the guardrails we have now. And it's not like every line of code is headed for a nuclear or medical facility. Applying the right depth of review in the right context has always been the actual job.

The tier of code where you read everything is real, but it's shrinking, and guardrails are what shrink it.

I've seen this play out at scale twice this year. In public: Bun rewrote roughly a million lines of Zig into Rust in eleven days, using 64 parallel agent instances in an implementer/reviewer pattern, with its TypeScript test suite as the conformance harness.⁵ It shipped with 19 known regressions (since fixed) and drew fair criticism for its heavy use of unsafe. I'm including those deliberately, because they make the point: the test suite caught what mattered.

On my own team: we migrated our entire Vue component library from .vue single-file components to TypeScript TSX (still Vue as the framework) using AI plus static migration tools I built. The static tools converted what they could mechanically, the agents took care of the rest. With good test coverage going in, I'd estimate 95% of the work landed without a single bug; what bit us were migration artifacts (global Vue and Nuxt configuration changes) and some minor class-ordering issues. (I've written about the TSX setup in a previous post.)

The counter-evidence: a widely-cited METR study found experienced developers using AI took 19% longer on certain tasks, mostly verifying and correcting AI output.⁶ I can't argue with their data, but it doesn't match anything I've seen, and I think the difference is the guardrails. What I've seen is prototyping going from weeks to hours, almost a J curve. At Bumble, on a highly experimental platform, these tools would have let us ship ten times the experiments. If verification is where the time goes, verification is what you should industrialize, which is the rest of this post.

The curve is accelerating

Everything above already happened. The reason I'm confident it keeps going is the trend line: METR tracks the length of task, measured in human-professional time, that models complete autonomously, and that horizon has been doubling roughly every seven months since 2019, accelerating since 2023.⁷ The Bun rewrite is one point on that curve. Extrapolate twelve months, and the question stops being whether the agent can do the task, and becomes whether your engineering team is set up to direct and verify that much output.

If I were running a ten-person team, here's what I'd change on Monday morning.

What I'd change Monday morning

Everyone runs agents in parallel. You can direct multiple tasks at once now; one agent per person leaves most of the value on the table.

The agents move off laptops. When your laptop closes, your agents die; on a cloud environment they keep running. We attach with tmux, and recently I've been trying out Herdr.⁸

Onboard agents like employees, but respect the context. An agent with no guidance does whatever it wants, but stuffing its context with your documentation fails just as drastically. Context is precious; don't pollute it. What works: self-documented code that records the why, and a knowledgebase the agent can grep when it decides it needs more.

Force the why. Agents love to narrate what they're doing, but the code already tells you that. Force them to document why: the thing you need three months later, staring at a line and wondering why it was written in the first place.

Gate on evals, not demos. Almost everybody should have an internal eval for what they're building; a passing demo tells you very little about how the thing behaves the rest of the time.⁹

Review from altitude. AI reviewers take the boring layer (style violations, easily-exploitable security gaps) and your review moves up to architecture, reusability, data structures, the bigger patterns. You're allowed to zoom out. A developer becomes an architect now, and that's a huge thing.

On hiring: I don't have an answer. Hiring has become difficult, because every portfolio project is now vibe-coded, so you have no idea what you're looking at. You have to somehow evaluate taste: how much care and attention went into the work. We've struggled with it, and I won't pretend otherwise.

The discipline: fighting brain rot

I'd be lying if I ended on the prescriptions, because the hardest problem on that Monday morning is me.

These models make it so easy to not read anything and just ship. You have to actively fight yourself, and I struggle with this personally. The temptation: prompt something, barely test the output, say "yep, ship it", because the models are good enough that whatever they produce will mostly work, and the saved time feels like it buys the next feature, the next experiment, the next side project.

But it leads to complete brain rot, for both the agent and for you. You lose all sense of what's actually being built: an old man yelling at the cloud, making claims about your own system without verifying that any of them back up. Google's paper puts it well: AI "multiplies both your strengths and your weaknesses."¹⁰ The taste test, a human actually using the thing, is what keeps that multiplier pointed the right way.

Code as a compiled artifact

Everything above is my plan for the next year. Here's where I think it goes a couple of years out.

I don't write binary. I don't write assembly. I don't even write C. Node and Chrome are built on C, which becomes assembly, which becomes something I've never once needed to look at, and I just write JavaScript on top. With the next generation of agents, I think the whole codebase becomes almost like a compiled artifact. Just as I don't read assembly today, you won't need to read the code agents produce. It will be some internal representation in their latent space.

Someone will say: but you're becoming dumb. Which is true, but I'm also dumb now. I don't know assembly, and somehow I still write TypeScript and do useful work.

It's not a one-to-one mapping, of course. But when your code gets compiled, most of the time (and I want to stress most of the time) you don't care what bytes it compiled to, because it works and everything is fine. Similarly, give an agent a detailed specification and guardrails, and most of the time the code it produces just works, as expected. Neither case means you go blind; it's always context. If your application is high-performance and the compiled code isn't performing, you start investigating. If your prompted code isn't good, you do the same thing: you dig down into it.

So the rules change a lot. Your ability to think clearly, to write clearly, to communicate clearly becomes the important thing, because good thinkers and good writers can impart their taste, their wisdom, and their vision, and the agents do the work.

That's the bet I'm making for the next year. I'm writing it down so that next July I can come back and check whether any of it backed up.

Gemini 3 Pro (Nov 18, 2025), GPT-5.1 Codex-Max (Nov 19), Claude Opus 4.5 (Nov 24), and GPT-5.2 (December). Simon Willison's 2025: The year in LLMs is the best chronicle of the wave. ↩
Nathan Lambert's writing at Interconnects is the clearest explanation of reinforcement learning with verifiable rewards, a term his team coined. ↩
Google, The New SDLC With Vibe Coding (May 2026), by Addy Osmani, Shubham Saboo and Sokratis Kartakis; there's an author's summary on Osmani's blog. A genuinely useful paper, with the caveat that its later chapters lean promotional about Google's own tooling. ↩
Handy: free, open source, fully offline speech-to-text. Plenty of similar apps exist, with new ones every month; I just haven't had a reason to switch. ↩
Simon Willison, Rewriting Bun in Rust, and The Register's coverage of the merge. ↩
METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (paper). Worth noting the study used early-2025 tools (Cursor with Claude 3.5/3.7 Sonnet), from before the autumn wave this post is about. ↩
METR, Measuring AI Ability to Complete Long Tasks: the ~7-month doubling time, revised faster in their Time Horizon 1.1 update (January 2026). ↩
Herdr: an agent-aware terminal multiplexer: tmux that understands agents, showing at a glance which are blocked, working, or done. ↩
"Set the bar at the eval, not the demo." (The New SDLC With Vibe Coding, p. 44.) ↩
"AI is a force multiplier — and it multiplies both your strengths and your weaknesses." (The New SDLC With Vibe Coding, p. 47.) ↩

How I am using Vue with Typescript 7 today

Nikhil Verma — Fri, 10 Jul 2026 10:48:02 +0000

TypeScript 7 shipped this week. The Go-native compiler, the one with the order-of-magnitude speed claims, is now just typescript@7 on npm. And buried in the announcement is a caveat that reads like bad news for Vue developers: tools that embed TypeScript into their own language services — Volar among them — can't use it yet, so "workflows that use Vue, MDX, Astro, Svelte and others will likely not yet be able to leverage TypeScript 7."

Our Vue app has been on the native compiler the whole time. Every component, type-checked by TypeScript 7's toolchain, on GA week, no plugins, no waiting for anything to be ported.

We write Vue in .tsx files. The reason that works points at a misconception about what Vue actually is, and the payoff goes well beyond one fast compiler.

The .vue file was always optional

Say "Vue" and most people picture a .vue file: a <template> block, a <script setup> block, maybe some scoped styles. The single-file component is Vue's signature authoring format, it's what every tutorial teaches, and it's genuinely pleasant. Somewhere along the way, the public assumption calcified into Vue requires .vue files.

It doesn't, and Vue has never claimed it does. The docs describe render functions and JSX as a fully supported, first-class way to write components. Vue ships a JSX transform, a jsxImportSource, and complete JSX type definitions out of the box. The SFC is sugar over a component model that is plain JavaScript: an object with a setup function that wires up reactive state and returns a function that renders it.

Which means a Vue component can be an ordinary TypeScript file:

import { defineComponent, ref } from "vue";

export default defineComponent({
 name: "UsageBadge",
 props: {
  label: { type: String, required: true },
  limit: { type: Number, default: 100 }
 },
 setup(props) {
  const used = ref(0);
  const bump = () => used.value++;

  return () => (
   <button class="badge" onClick={bump}>
    {props.label}: {used.value}/{props.limit}
   </button>
  );
 }
});

Everything here is documented Vue. setup runs once per component instance and closes over the reactive state; the arrow function it returns is the render function, and Vue re-runs that — and only that — when something it read changes. This shape is more honest about Vue's execution model than the SFC is: the one-time setup and the re-running render are two visibly different functions, rather than a compiler convention you have to know about.

The configuration to make TypeScript understand it is two lines:

{
 "compilerOptions": {
  "jsx": "preserve",
  "jsxImportSource": "vue"
 }
}

There's no preprocessor and no custom language service to install. A .tsx file with Vue as its JSX source is just TypeScript, and every tool that speaks TypeScript — the compiler, the linter, your editor, and as of this week a very fast Go binary — handles it natively.

The heroics you no longer need

To appreciate what "just TypeScript" buys you, look at what it takes to type-check the file format everyone assumes is mandatory.

TypeScript has no idea what a .vue file is. So Volar — the language tooling behind Vue — performs a genuinely impressive maneuver: it intercepts TypeScript's view of the filesystem, and wherever the compiler would read a .vue file, Volar hands it generated TypeScript instead. Your template becomes synthesized code full of type-level gymnastics; source maps stitch the compiler's errors back onto your template expressions. On the editor side you need the Vue extension to make VS Code play along, and for a while in 2024 you needed to understand its "hybrid mode" — which half of the language service answered which kind of request — just to work out why autocomplete had stopped showing suggestions.

This is admirable engineering. Volar is one of the most technically interesting projects in the frontend ecosystem, and for SFC codebases it works well. But it is a shim layer, and shim layers have a cost that shows up on the margins: a squiggle that lands a line off because a source map didn't quite align, a rename refactor that doesn't reach into templates, a new TypeScript release that the shim has to chase. This week is the sharpest example yet — TypeScript 7 doesn't expose the API surface the shim hooks into, so the entire .vue world waits for TypeScript 7.1 and a Volar port before it sees any of the new compiler's speed.

None of this is a criticism of Vue. It's the structural price of any custom file format, and Svelte, Astro and MDX are all in the same waiting room. Vue is the one framework in that list that doesn't make you pay it. The escape hatch is documented, and — we can now report from production — entirely boring to live with.

The reactivity system comes with you

The reflex objection: isn't this just React with extra steps? If I'm writing JSX anyway, what's left of Vue?

The answer is everything that matters, because Vue's soul is the reactivity system. The difference is easiest to see in the code you don't write.

In React, the runtime re-runs your whole component function on every update and cannot know what your effects depend on, so you tell it, by hand, forever:

const total = useMemo(
 () => items.reduce((sum, i) => sum + i.price * i.qty, 0),
 [items] // you maintain this list. every hook. every edit.
);

useEffect(() => {
 document.title = `Cart: ${total}`;
}, [total]);

Every React developer knows the failure modes: a stale closure because a dependency was missed, a render loop because an object identity changed, an exhaustive-deps lint rule arguing with you about what you meant. It's manual memory management for dataflow.

Vue tracks dependencies at runtime, automatically. A computed or a watchEffect records exactly which reactive values it read while running; when one of those changes, it re-runs. There is no list to maintain because the system derives the list from the truth of what executed:

const items = ref([{ price: 1, qty: 2 }]); // deeply reactive by default

const total = computed(() =>
 items.value.reduce((sum, i) => sum + i.price * i.qty, 0)
);

watchEffect(() => {
 document.title = `Cart: ${total.value}`;
});

No dependency arrays, no useCallback rituals. And because setup runs once, you don't re-create closures on every render or reach for useRef to smuggle values past the re-render boundary — state just lives in scope, and the render closure reads it.

Writing this in TSX changes none of it. The reactivity and the automatic tracking come from the runtime you import from "vue"; the file extension has no say in it. You keep the framework's superpower and hand the authoring problem to the most heavily-tooled language format in the industry.

There are a few habits to pick up — slots become object literals of functions, v-model becomes an explicit prop-plus-handler pair, and strongly-typed wrapper libraries sometimes want a props spread — but each is a small convention you learn in a day. All three make previously-implicit machinery explicit, which our code reviews turned out to appreciate.

The proof: TypeScript 7, in production, this week

The compliance platform I work on is a Nuxt app with roughly eight hundred TSX components — pages, layouts, the app root, all of it plain TSX with jsxImportSource: "vue". Nuxt wires up the JSX transform automatically.

Because every component is real TypeScript, our type-checking pipeline runs on the native toolchain — the same Go compiler that GA'd as TypeScript 7 — and has for months, across the preview releases. A full type-aware pass over the entire app takes about 46 seconds on a laptop. When the GA landed this week, our upgrade was a version bump in one package.json. The components didn't know anything happened.

Meanwhile the .vue path, through no fault of its own, is on TypeScript 6 until the 7.1 API ships and Volar rebuilds on top of it — "at least several months," per the TypeScript team, with no committed date. I don't say that to gloat; the port will land and SFC users will get their speedup too. But the gap between the two paths was always there, and TypeScript 7 just made it visible as a version number.

The migration, briefly

We didn't start here. We started as a normal SFC codebase, and moved to TSX in one big-bang change this spring: about 1,500 files in a single PR. The mechanical bulk was done by a converter I open-sourced as vue-to-tsx — it parses the SFC with Vue's own compiler and rewrites templates to JSX render functions, <script setup> macros to defineComponent, scoped styles to plain CSS imports. Directives with no JSX equivalent get flagged for a human (or, optionally, an LLM) to resolve. It converts a file like this:

<template>
 <button @click="emit('update', count)">Count: {{ count }}</button>
</template>
<script setup lang="ts">
import { ref } from "vue";
const props = defineProps<{ initial?: number }>();
const emit = defineEmits<{ (e: "update", value: number): void }>();
const count = ref(props.initial ?? 0);
</script>

into this:

import { defineComponent, ref } from "vue";

export default defineComponent({
 emits: ["update"],
 setup(props, { emit }) {
  const count = ref(props.initial ?? 0);
  return () => (
   <button onClick={() => emit("update", count.value)}>
    Count: {count.value}
   </button>
  );
 }
});

A handful of .vue files survive in our repo — vendored UI-library primitives we treat as third-party code, plus a couple of components leaning hard on scoped styles. The goal was to move the codebase's center of gravity, and it moved.

The migration's original motivation wasn't TypeScript 7 at all. We did it because we wanted components that read as plain code and tooling that didn't need a translation layer. The native-compiler dividend arrived months later, unearned, precisely because we were sitting on the mainline TypeScript path when it got faster. That's the property I'd sell harder than any benchmark: standard formats appreciate in value while you sleep.

The takeaway

Don't tie Vue to .vue. The framework's best ideas — reactivity with automatic dependency tracking, effects the runtime manages for you — live in the runtime and travel intact into plain TSX. The single-file component remains a great format, and if it's serving you well, nothing here says to move.

But if you've ever assumed Vue can't work without its custom files, or dismissed it because you didn't want a bespoke toolchain between you and your type checker: that assumption is wrong. The path is documented and proven in production, and as of this week it comes with the fastest TypeScript compiler ever shipped. You can use Vue in a modern, native way — today.

Building a harness that makes a small LLM reliable

Nikhil Verma — Wed, 20 May 2026 00:00:00 +0000

When I started building a multi-turn agent that could actually do things — search records, update a status, link a piece of evidence, the usual kit — my first instinct was the one I think everyone has: reach for the biggest, smartest model and hope it behaves.

I ended up doing nearly the opposite. The agent I shipped runs on Haiku 3 — small, cheap, fast, and on its own absolutely not what you'd call a careful reasoner. The reliability didn't come from the model. It came from the scaffolding I built around it. And that shift — from "make the model smarter" to "make the harness stronger" — is honestly the most useful thing I've picked up about getting agents to work.

Let me walk you through two pieces of that harness, because they did most of the heavy lifting. First, though, the little incident that kicked it all off.

I was scrolling back through a long session, looking at tool-call errors. (You should be watching those, by the way — a model retrying its own failed tool calls is normal, but a pattern of failures usually means a tool is confusing or a contract is too loose.) And I spotted a few calls hitting IDs that didn't exist. The IDs looked perfect: flawless, textbook UUIDs. Right shape, right everything. They just... weren't real.

The model made them up. An LLM loves to match patterns, and Haiku had watched hundreds of UUIDs stream past in its context, absorbed the shape, and somewhere between "search for items" and "mark this one done" it confidently produced a plausible fake.

Now, the lazy fix is "use a better model" — a bigger model hallucinates less. But that's the wrong lesson, and I want to spend a paragraph on why, because it shapes everything below.

The thing I got wrong for too long: the harness is the lever, not the model

It's so tempting to treat reliability as something you buy from the model. Agent flaky? Upgrade. And it sort of works — partly, and expensively. But a frontier model with no guardrails still invents IDs and still skips steps; it just does it rarely enough that you stop noticing, which is arguably worse.

What actually moved the needle for me was flipping the assumption: assume the model will get things wrong — early, often, in daft ways — and build a harness that makes those mistakes either flat-out impossible or automatically recoverable. Do that, and a lovely thing happens: the bar for how clever the model has to be drops through the floor. A small model on a strong harness beats a big model on a weak one, and it costs a fraction as much per turn.

So the two patterns below aren't "prompt it better." They're rails the model can't leave even if it wants to.

Raw IDs are basically a loaded gun

If you put raw UUIDs into the model's context, sooner or later it'll hand raw UUIDs back. Some real, some invented, and from the outside — just staring at the text — there's no reliable way to tell which is which. The smaller the model, the sooner this happens.

You can ask nicely, of course — "only use IDs I've actually given you" — but a prompt is a polite request, not a contract. Give a model a long context, a bit of instruction-following pressure, a small distraction, and it'll cheerfully forget. And there's no feedback loop: it has no idea the ID was wrong until your database has already either rejected it or, on a bad day, quietly swallowed it.

I dug into the why of this a while back — how models fabricate IDs and a couple of ways to stop it — in a separate post on UUID hallucination. The short version: enum constraints work nicely for tightly-controlled flows, and token aliasing (swapping UUIDs for simple ITEM-1/ITEM-2 handles) works for multi-turn agents. That post is the background, if it's useful. What I want to get into here is what token aliasing has to grow into once a small model is allowed to actually write to your database — and how it becomes one leg of the harness.

Either way, the fix isn't to ask harder. It's to make raw IDs invisible to the model in the first place. If it never sees a UUID, it can't parrot one back.

Pattern 1: the ref proxy

The idea is dead simple. Every UUID that leaves your system on its way to the model gets swapped for a short, session-scoped nickname first. The model only ever sees things like ref_A3k9Xp2Q. Your database only ever sees real UUIDs. A little registry sits in the middle and does the translation.

Here's the shape of that registry:

type EntityRefRegistry = {
  version: 1;
  // ref token → { entityType, entityId, createdAt, lastSeenAt }
  refs: Record<string, EntityRefRecord>;
  // "type:uuid" → ref token (reverse lookup)
  entityToRef: Record<string, string>;
};

I park this as a durable JSONB column on the chat thread row. It survives across turns, so the same entity always gets the same nickname for the whole session — no confusion for a model that's already easily confused.

The proxy works both ways. On the way out (system → model): any time your tools hand back data, a proxyForModel pass walks the output, finds every UUID-shaped string, and swaps it for a ref token — minting a new registry entry if it hasn't seen that entity before.

function proxyForModel<T>(
  value: T,
  registry: EntityRefRegistry
): { value: T; registry: EntityRefRegistry } {
  // walk value, replace UUIDs with ref_* tokens
  // registry is append-only — same UUID always gets same token
}

On the way in (model → system): when the model calls a tool with a ref_* value, a proxyForSystem pass turns it back into the real UUID before your handler ever lays eyes on it.

function proxyForSystem<T>(
  value: T,
  registry: EntityRefRegistry
): { value: T; unresolvedRefs: string[]; rawUuids: string[] }

Notice it hands back two lists rather than throwing a tantrum. unresolvedRefs catches refs the model invented that aren't in the registry. rawUuids catches the times it bypassed the whole scheme and pasted a real UUID straight in — which happens more than you'd hope, especially early on when some old prompt or context snippet leaked raw IDs in before the proxy was watching the door.

The tool wrapper checks both lists before it runs anything:

execute: async (...args) => {
  const systemInput = proxyForSystem(args[0], registryRef.current);

  if (systemInput.unresolvedRefs.length > 0) {
    return toolInputError(
      `Unknown model reference(s): ${systemInput.unresolvedRefs.join(", ")}.` +
      ` Use a ref returned earlier in this chat, or search again for a fresh ref.`
    );
  }

  if (systemInput.rawUuids.length > 0) {
    return toolInputError(
      `Raw UUIDs are not accepted: ${systemInput.rawUuids.join(", ")}.` +
      ` Use the corresponding ref from earlier in this chat.`
    );
  }

  const result = await execute(systemInput.value, ...rest);
  // Outbound: swap UUIDs in result for refs
  const modelResult = proxyForModel(result, registryRef.current);
  registryRef.current = modelResult.registry;
  return modelResult.value;
}

Here's the bit that matters most: these are recoverable errors. The model gets to read them and try again, in the same conversation, without anything blowing up. This is how the harness teaches a weak model on the fly — not by being cleverer, but by catching the mistake and handing back a specific, actionable correction.

The simplest way is to return a structured error as the tool's result instead of the normal output. toolInputError here is just a small helper of mine that returns a plain object with a message in it:

function toolInputError(message: string) {
  return { ok: false, error: message };
}

The model reads that like any other tool output, goes "ah, my mistake," and tries again. It works purely because the message text spells out what to do next — no magic, the model is just reading English. Which means, for a small model, the wording of that error is part of your prompt engineering. "Invalid input" gets you nowhere; "Unknown ref ref_Q7zXm1 — search again to get a fresh one" gets you a correct retry. Be specific.

(You can also achieve the same thing by throwing and letting your agent loop catch the error and feed it back as the next tool result — whichever your framework makes easiest. The one myth worth killing: a thrown tool error doesn't have to abort the turn. Whether it does is up to how your loop is wired, not some law of nature.)

And the schemas get rewritten before the model ever sees them, too. Any { type: "string", format: "uuid" } field becomes:

{
  "type": "string",
  "pattern": "^ref_[A-Za-z0-9]{6,12}$",
  "description": "Use the model-visible ref_* value from chat/page context. Do not provide a raw UUID."
}

Worth being honest about what this does, though: that schema is a hint, not a fence. In plain JSON Schema, format and pattern are annotations — the model treats them as guidance, and a vanilla validator won't necessarily enforce them. The actual enforcement is my proxy layer rejecting anything that isn't a genuine ref. So the schema tells the model what good looks like; the harness is the thing that holds the line. Belt and braces — which is the whole game when the model is small.

The entire tool set gets wrapped in a single call:

const modelVisibleTools = wrapToolSetWithRefProxy(yourTools, registryRef);

Your existing tools don't change at all. The proxy is transparent to them — which is the point. The harness should be something you bolt around your tools, not something you have to rewrite them for.

Bonus prize: the registry doubles as a complete audit trail. Every entity the model touched, when it first appeared, when it was last referenced — all sitting in the thread row. For a system that has to answer "what did the agent actually do," that fell out for free.

Pattern 2: mandatory tool contracts

The second failure mode was sneakier, and it's classic small-model behaviour. The model would breeze through a whole workflow and then skip the one step that actually commits the result — it'd narrate what it found, lay out a lovely conclusion, and then end the turn with prose instead of calling the tool that records the verdict.

Prompting did not reliably fix it. "You MUST call submit_verdict before ending" helps a bit, but it's a vibe, not a guarantee, and the smaller the model the more of a vibe it is. Long multi-step runs drift from earlier instructions. Context compaction eats your constraints. So instead of nagging, I made the endTurn tool aware of what has to have happened before it's allowed to succeed.

Workflows declare their required tools right in the frontmatter:

---
id: control-assessment
title: Control Assessment
mandatory_tools:
  - name: search_evidence
    minCalls: 3
    onTooFew: "You must search for evidence at least 3 times before concluding."
  - name: submit_assessment
    onMissing: "You must call submit_assessment before ending this turn."
---

The runner normalizes these into rules and attaches them to the conversation. Before the stream even starts, it also replays the prior message history to count any mandatory calls that already happened in earlier turns — so a tool called back in turn 2 still counts when you're in turn 5.

Enforcement comes in two layers. First, every mandatory tool's execute gets wrapped with a counter: a successful call ticks it up, an errored call (including the ref-proxy rejections above) does not. A failed attempt doesn't get to check the box.

Second, the endTurn tool gets a bouncer on the door:

if (name === "endTurn") {
  const violations = validateMandatoryToolCounts(
    state.rules,
    state.counts
  ).filter(violation => violation.rule.name !== "endTurn");

  if (violations.length > 0) {
    return mandatoryToolErrorResult(violations);
  }
}

mandatoryToolErrorResult returns a structured (again, returned, not thrown) error:

{
  isError: true,
  message: "You cannot end this turn yet. Required tool \"submit_assessment\" has not been called...",
  missingMandatoryTools: ["submit_assessment"],
  violations: [{ toolName, count, minCalls, message }]
}

The model reads it, realises it hasn't held up its end, and calls the missing tool. With Haiku the self-correction rate here was genuinely high — because the message is specific enough that there's no thinking required, just following.

And there's a second lock on the same door. The stop condition itself is gated, so even a stray endTurn won't actually halt the loop. It's a small enough check to just write by hand — whatever your agent loop is, it almost certainly calls some "should I stop now?" predicate after each step. Mine only lets the loop stop once the model has called endTurn and every mandatory tool has hit its count:

const MAX_STEPS = 200;

// Called by the agent loop after each step to decide whether to stop.
function shouldStop(steps: Step[]): boolean {
  if (steps.length >= MAX_STEPS) return true; // hard safety cap, no matter what

  const lastStep = steps[steps.length - 1];
  const calledEndTurn = lastStep.toolCalls.some((c) => c.toolName === "endTurn");
  if (!calledEndTurn) return false; // not trying to end — keep going

  // The model wants to end. Only allow it if the contract is satisfied.
  const counts = countToolCalls(steps);
  return validateMandatoryToolCounts(rules, counts).length === 0;
}

If the model somehow sneaks endTurn past the wrapper, this just refuses to fire. The stream keeps going and it gets another crack at the contract. It isn't getting out until the work is done.

And for the genuinely grim cases — a run that dies on the step limit or a network blip without ever satisfying the contract — the runner checks on finish and can replay the turn, this time forcing the very next step to call the missing tool directly (most agent loops let you pin which tool runs next). That's the last line of defence: the harness physically walking the model to the till.

The thread running through both: recover, don't demand perfection

If you squint, both patterns are the same move. Neither one assumes the model is good. Both assume it'll fumble and build the catch around it.

The ref proxy says: the model cannot produce a valid raw ID, and if it tries, it gets a precise nudge and another go. The mandatory contract says: the model cannot end before the work is committed, and if it tries, it gets a precise nudge and another go. The capability that used to live in the model — "remember to use real IDs," "remember to commit" — now lives in the harness as something structural. The model just has to follow rails, and following rails is something even a small model is pretty good at.

This is also why the error wording isn't an afterthought. For a small model the recoverable error is the teaching signal. "Invalid input" gets you nowhere; "Unknown ref ref_Q7zXm1 — search again to get a fresh one" gets you a correct retry. You're not arguing with the model, you're handing it the next instruction at exactly the moment it's listening hardest.

What changed — and the bit that surprised me

Before: occasional hallucinated IDs slipping into the database, semi-regular turns that fizzled out in prose instead of a committed result, behaviour that wobbled with context length. Death by a thousand "huh, that's odd"s.

After: hallucinated IDs became structurally impossible — there's no UUID surface left to hallucinate against. Required-tool completion became something I could verify instead of hope for — not "the model said it was done" but "the counter hit the minimum." And the audit trail came free.

The genuinely surprising part was the cost side. I'd assumed reliable agent behaviour meant paying for a big model on every turn. A small, cheap model — Haiku 3 — carried the whole thing once the rails were in place, at a fraction of the per-turn cost and noticeably faster. The harness was a one-time build; the savings are every single request, forever.

The same harness makes a bigger model better too. Guardrails help across the board. The win isn't "make the frontier model slightly more reliable." It's that you stop needing the frontier model for a big chunk of the work.

The takeaway

I spent a long time treating the model as the variable I could turn up when things got unreliable. The thing I'd tell my earlier self is: turn up the harness instead.

There are two failure modes people lump together when they say "my agent won't follow instructions." One is behavioural — the model drifts, gets muddled, picks the wrong priority. That one genuinely is a prompting problem, and a bigger model genuinely helps. The other is structural — your "enforcement" is just text in a prompt that the model can and eventually will ignore. No amount of bold MUST saves you there, and no model is big enough to fully trust.

For the structural stuff you need structural fixes, and the nice surprise is that once you've built them, the model underneath can be small and cheap. Make hallucinated IDs invalid by construction. Make skipping a step mechanically impossible. Make every mistake recoverable with a clear nudge. Do that, and a model like Haiku 3 will quietly do work you assumed needed something four times the price.

The mental model I use now: if the correctness of a workflow depends on the model choosing to do the right thing, stop and ask whether the harness can make that choice for it. It usually can. And it's almost always cheaper than reaching for a smarter model to paper over the gap.

Rate limiting AI APIs across distributed workers

Nikhil Verma — Fri, 24 Apr 2026 00:00:00 +0000

So picture this. The system is throwing Anthropic 429s every two or three minutes during big processing runs. Fine — expected, honestly. We were processing thousands of documents in parallel across a fleet of Temporal workers, so of course we were leaning on the rate limit. What was not expected was that the 429s got worse after we added retry logic. You add the thing that's supposed to help, and it makes everything angrier. Cool stuff.

We'd implemented exponential backoff with a max of 120 seconds, by the time most of those long backoffs fired, the quota they were waiting on had already come back. We were manufacturing dead air on top of the rate-limit problem and then patting ourselves on the back for it.

Let me walk you through the three-layer pattern that actually fixed it. But first I have to clear up a misconception I had, because it's the whole reason the backoff was wrong.

The thing about TPM limits that I got wrong

I used to picture token-per-minute limits like a parking meter: the clock ticks over at 60 seconds and bang, the whole allowance resets to full. So my mental model was "if I get a 429, just wait until the next reset and I'm golden." Backing off past that reset point is obviously pointless, right?

Turns out that's not how it works. A rate limit doesn't snap back all at once — it replenishes gradually. Whatever a given provider uses under the hood, a token bucket that refills at a steady rate or a sliding window that lets old usage age out, the effect is the same: your headroom climbs back up continuously across the window rather than in a single tick at the top of the minute. There's no parking-meter moment to wait for. So when you've completely drained your quota, getting back to full takes roughly the length of the window — call it a minute — and it happens a little at a time the whole way there, not in one lump at the end. (Most providers document exactly this; Anthropic, for instance, describes its limit as a continuously replenished token bucket.)

So why does that change anything? Because the practical upshot is almost the same but the reason is completely different. If you've fully drained the bucket, it refills back to maximum over roughly a minute. So the useful ceiling on a backoff is about 60 seconds — not because a window "resets," but because after ~60s of trickle the bucket has recovered all the headroom it's ever going to give you. Wait longer than that and you're just standing there while a full bucket waits for you to do something with it.

Standard exponential backoff doesn't know any of this. It just keeps doubling:

wait = base * 2^attempt + jitter

With a base of 10 seconds, attempt 4 lands you at ~160 seconds of waiting. But the bucket finished refilling about 100 seconds ago. You're backing off from a problem that no longer exists. Every second past 60 is pure waste.

So cap it at the window's worth of refill — 60 seconds — and stop there. Not 2 minutes, not 5.

const RATE_LIMIT_BASE_BACKOFF_MS = 10_000;
const RATE_LIMIT_MAX_BACKOFF_MS = 60_000; // ~one bucket-refill's worth

function calculateBackoffMs(attempt: number): number {
    const exponential = RATE_LIMIT_BASE_BACKOFF_MS * 2 ** (attempt - 1);
    const capped = Math.min(exponential, RATE_LIMIT_MAX_BACKOFF_MS);
    const jitterRange = Math.max(0, capped - RATE_LIMIT_BASE_BACKOFF_MS);
    return RATE_LIMIT_BASE_BACKOFF_MS + Math.floor(Math.random() * (jitterRange + 1));
}

Once that clicked, the rest of the architecture kind of fell out on its own.

Layer 1: tell everyone about the 429, immediately (Redis)

Here's where it gets fun with multiple workers. Per-worker backoff is useless on its own when you've got a whole pool hammering the same endpoint. One worker gets a 429 and starts politely backing off — meanwhile the other nine are still firing away, blissfully ignorant. They each catch their own 429, each start their own little timer, and the whole pool spends the next few minutes flailing before it settles. It's like everyone in the pub trying to get to the bar at once and nobody noticing it's already three-deep.

The fix is to make a 429 everybody's problem the instant it happens. When any worker hits a rate limit, it writes a pause state to Redis. And every worker checks that state before every single AI call.

// When we get a 429
await redis.set(
    `rate_limit:pause:${modelKey}`,
    JSON.stringify({ pauseUntil: now + waitMs, lastThrottleTime: now }),
    "EX",
    300 // 5-minute TTL; auto-expires if workers die
);

// Before every AI call
const pauseState = await redis.get(`rate_limit:pause:${modelKey}`);
if (pauseState && parsedState.pauseUntil > Date.now()) {
    await sleep(parsedState.pauseUntil - Date.now() + jitter);
}

One detail that matters more than it looks: namespace the key by modelKey (provider:model). A 429 on provider-a:model-x has no business pausing work on provider-b:model-y — those are completely separate buckets. Pause everything every time one model throttles and you've just tanked your throughput for no reason.

And that 5-minute TTL is the safety net. If a worker keels over mid-flight with a pause state still written, the TTL makes sure the rest of the pool isn't held hostage forever by a ghost.

Layer 2: jitter on the way back, or you'll do it all again

Right, so here's the sneaky one that survives even after you've added the cross-worker pause. The pause expires at the exact same instant for everybody. So all the workers wake up together, all check the gate together, all see it's clear together, and all fire together — perfectly recreating the original spike that caused the 429 in the first place. You've built a thundering herd and handed it a starting pistol.

The fix is to add a bit of random jitter on each worker's wake-up:

const remainingPause = pauseState.pauseUntil - Date.now();
const jitter = Math.floor(Math.random() * 30_000); // 0-30s
await sleep(remainingPause + jitter);

Now the wake-ups smear out across a 30-second window. Worker A pops up at T+0 and fires one request, worker B at T+7s, and so on down the line. By the time the whole pool is back in business, that initial burst has been staggered into something the provider can actually stomach. Bog-standard thundering-herd mitigation — but so easy to forget, because you add the cross-worker pause, feel clever, and assume the problem's solved.

Layer 3: kill the client library's own retry (this was the villain)

This was the bug. The one that made everything worse the moment we added rate limiting.

Almost every LLM client library has retry logic baked in. By default, when a call gets a 429, the client quietly retries on your behalf with its own exponential backoff — often two or three extra attempts you might not even realise are happening. Now stack that on top of your Temporal activity retries and your own withRateLimitRetry wrapper, and congratulations: you've got three independent retry loops running at once, none of them aware of the others.

The interaction is genuinely nasty. Temporal fires an activity. Inside it, the client library eats a 429 and starts its own retry, sleeping a while. Meanwhile your withRateLimitRetry wrapper has also clocked the 429 and started its own sleep. When the client's retry eventually fires, it goes back through withRateLimitRetry, which records another 429 and sets another pause. It cascades. And the worst part is the logs make it look like the backoff is working a treat, when really it's just multiplying your wait times behind your back.

So turn the client's built-in retry off and let a single layer own the whole thing. Most libraries expose this as a max-retries option — set it to zero:

// Whatever your LLM client is, find its retry knob and switch it off,
// so the outer layer (Temporal + withRateLimitRetry) is the only thing retrying.
const result = await llm.generate({
    model,
    prompt,
    maxRetries: 0,
});

One retry layer that actually understands how the quota refills will always beat three that are each guessing in the dark.

Going proactive: token accounting with a Redis ZSET

Everything above is reactive — it stops the bleeding after a 429. But by the time you've got a 429, you've already paid for it: a failed network call, wasted tokens, and a provider that's now mildly cross with you. The proactive layer's job is to stop the 429 ever happening.

The trick is a sorted set in Redis, keyed by provider:model. Every completed call drops its token count in as a member, scored by the timestamp:

await redis.zadd(
    `token_usage:${modelKey}`,
    Date.now(),
    `${reservationId}:${tokens}`
);
// Evict entries older than 60 seconds
await redis.zremrangebyscore(key, 0, Date.now() - 60_000);

Before each call, you read the set, sum the tokens still in the last 60 seconds, and check against your TPM ceiling. Getting close to the edge? Wait for the oldest entries to age out instead of lobbing in a call that's just going to 429. (This proactive accounting earns its keep doubly with Anthropic, by the way, because the bucket also enforces short bursts — a "60 requests per minute" limit can show up as "one request per second" — and it doesn't love sharp usage ramps. Smoothing your own flow keeps you on its good side.)

Now, the detail that'll bite you if you skip it: reserve upfront, commit afterward. When a bunch of workers are calling concurrently, they each read the current usage and each see lovely headroom. If they all fire at once, the real usage is the sum of everyone's reservations — not the number any one of them read. So the read-then-write has to be atomic. A tiny Lua script does it in a single round-trip (and Redis runs Lua atomically — nothing else gets to sneak in mid-script):

-- Atomic: inspect bucket, reserve in one shot or reject
local currentTpm = sumBucketEntries(KEYS[1], windowStart)
if currentTpm + reserveTokens > tpmLimit then
    return {0, 'tpm', retryAfterSeconds, deficit}
end
redis.call('ZADD', KEYS[1], now, reservationId .. ':' .. reserveTokens)
return {1}

After the call finishes, commit the actual usage to replace the reservation. Models almost always generate fewer tokens than max_tokens, so committing the real number hands the unused budget back to the shared pool. Be greedy on the reservation, honest on the commit.

This is the layer that keeps the system from ever really needing the reactive pause under normal load. The 429 handler becomes the fire alarm, not the day-to-day traffic cop.

The whole picture

Three layers, working together:

Redis ZSET sliding window — proactive token accounting with atomic reservation. Workers see each other's consumption before they fire. Stops most 429s ever happening.
Cross-worker pause state — the reactive 429 handler that writes to Redis the instant it's hit. The whole pool backs off together, and the backoff caps at one bucket-refill's worth (~60s) instead of some open-ended exponential fantasy.
One retry layer — SDK retry switched off. Temporal's activity retry plus the withRateLimitRetry wrapper own the semantics. No more three independent loops tripping over each other.

End result: 429s went from a regular feature of every big batch run to a rare event the system shrugs off gracefully when it does show up.

And the general lesson travels well beyond Temporal — BullMQ, Celery, your own hand-rolled queue, doesn't matter. The key fact is that TPM limits are time-windowed, and time-windowed limits have a natural ceiling on how long backing off is even worth it. Build your retry ceiling around how long the bucket takes to refill, and make that state shared across all your workers, and you'll get further than any amount of lovingly hand-tuned exponential backoff coefficients ever will.

Agents should be able to time travel

Nikhil Verma — Tue, 06 Jan 2026 00:00:00 +0000

When I work on a tricky problem I often go down random directions. I explore, learn what I can, and bring the learnings back. And I remember not to go down there again. This exploration in my mind is similar to Age of Empires fog of war or slime mold exploring its surroundings and then finding the right path.

AI agents could learn to benefit from doing the same, and they do for the most part, but they've got a bottleneck. They can't forget useless things. It's all appended in their context, or compacted when the context window is about to be reached. This doesn't help and can be improved.

Right now there are a few ways agents deal with long context

Summarization Compress old messages into a summary. But summaries are lossy. You lose the reasoning chain the failed attempts, the "why" behind decisions
Stores Store stuff externally and retrieve when needed. But this is passive. The agent doesn't control what's stored or when it's retrieved.
Just... keep everything Until you hit the context limit and things get weird.

There's also some cool research happening here. The Reflexion framework lets agents reset their environment and try again, keeping self-reflections in memory. Tree of Thoughts explores multiple reasoning paths. And there's recent work like "Memory as Action" and "Git Context Controller" that treat context management as something the agent should actively control.

But they're either architectural changes (you need to rebuild your whole system) or they're too drastic (reset everything and start over).

history.go(-10, "Bruh this path was a dead end")

What if we gave the agent a tool? Something dead simple:

Ok I've learned enough here. Take this message as my learning, drop the last 20 messages, and let's continue.

That's it. The agent decides when to use it. The agent decides what to keep, decides how far back to go.

The important bit here is that the agent initiates this, not some external system watching token counts. The agent realizes "I've been spinning my wheels" or "I found what I needed" and actively chooses to clean up after itself. I think the smarter models like Opus 4.5 and GPT-5.2 will be able to do this well.

I'm imagining a tool like:

time_travel(
  learning: "The API doesn't support batch operations, need to loop instead",
  steps_back: 15 // or go_to: "message_id_12345"
)

The agent calls this. The system:

Takes the learning and injects it back as context (system prompt or user message)
Drops the last N messages from the conversation
Continues from there

There's this thing called Physarum polycephalum. It's a slime mold. It doesn't have a brain, but it can solve mazes and find optimal paths between food sources. Researchers even used it to recreate the Tokyo rail network. Here is a cool video of it in action:

How does it do this? It explores everywhere at first. Then it withdraws from dead ends while reinforcing successful paths. It doesn't keep track of every failed path — it just stops going there.

That's what I want for agents. Explore, learn, withdraw, reinforce. Not "append everything forever and hope the model figures it out."

LLMs as Unreliable Narrators: Dealing with UUID Hallucination

Nikhil Verma — Fri, 14 Nov 2025 00:00:00 +0000

TL;DR

Problem: LLMs make up identifiers (UUIDs) that don't exist in your data, which breaks your retrieval flow
Solution 1: Use enum constraints with readable identifiers when you control the workflow
Solution 2: Map UUIDs to simple tokens (ITEM-1, ITEM-2) for longer agent conversations
Result: Way fewer hallucinations and more reliable ID references

You've spent weeks building a neat RAG (Retrieval-Augmented Generation) system. Your embeddings are sharp, your retrieval is fast, and you've got the LLM hooked up to search through your data. You test it out, and then... the model returns a UUID that doesn't exist in your system.

This is one of those problems that catches you off guard because the LLM sounds confident. It's not hedging or saying "I'm not sure." It's just making up an identifier like a7f3c9e2-1b4d-4e8f-9c5d-2a8f7b9e3d1c with complete conviction.

Smaller models (like mini/fast variants) tend to do this more often. They'll invent IDs to sound authoritative.

The Core Problem

LLMs are pattern-matching machines. When you ask them to pick from a list of items using UUIDs, they see the format - that alphanumeric pattern with hyphens - and they try to reproduce it. But they're not really "remembering" your specific IDs. They're just generating something that looks like a UUID, and half the time, it's completely made up.

This creates problems downstream: you can't match outputs to stored objects, logging gets messy, and audit trails break. You end up with a bunch of "ID not found" errors.

Solution 1: Enums for Controlled Flows

If you're building a structured, agentic workflow where you control the entire decision path, you have a powerful tool: structured outputs with enums.

The idea is simple: instead of letting the LLM generate IDs freely, you constrain its response to only pick from valid options. Here's how it works:

// Helper to normalize titles and avoid duplicates
function normalizeTitle(t: string) {
  return t.trim().toLowerCase();
}

// Build a list of valid choices from your data
const seen = new Map<string, number>();
const validChoices = items.map((item) => {
  const norm = normalizeTitle(item.title);
  const count = (seen.get(norm) ?? 0) + 1;
  seen.set(norm, count);
  // Handle duplicate titles by adding a suffix
  return count === 1 ? item.title : `${item.title} (${count})`;
});

// Create a schema where the model must pick from an enum
const schema = z.object({
  selectedTitle: z.enum(validChoices as [string, ...string[]]),
  reasoning: z.string().max(240), // Cap verbosity
  confidence: z.number(),
});

// Use your LLM's structured output feature
const response = await llm.parse({
  schema,
  messages,
});

When you pass an enum constraint to the LLM, it can't hallucinate. The model knows it must pick one of the valid options. If it tries to return something outside the enum, the parsing will fail and you can retry.

The key trick: Use simple, human-readable identifiers (like titles) in your enum, not UUIDs. UUIDs are harder for models to reproduce accurately because they're random. Titles are memorable patterns.

Then verify the match server-side:

const selectedTitle = response.selectedTitle;

// Find the actual item by matching the normalized title
const actualItem = items.find(
  (item) => normalizeTitle(item.title) === normalizeTitle(selectedTitle),
);

if (!actualItem) {
  // Invalid selection - add feedback and retry
  messages.push({
    role: "user",
    content: `That title wasn't in the list. Please pick from: ${validChoices.join(", ")}`,
  });
  // Retry with feedback (consider exponential backoff)
}

When to use this: Controlled flows where you're calling the LLM as part of a defined process. Think: multi-step decision trees, validation agents, classification pipelines.

Limitations: Watch out for enum size - OpenAI caps schemas at 1000 total enum values, with character length limits for larger enums. If you're dealing with hundreds of items, you might want to do some top-N prefiltering or break it into chunks.

Solution 2: Deterministic Alias Tokens for Multi-Turn Flows

But what if your LLM is having a longer conversation where you're not controlling every single prompt? The agent is exploring, reasoning, making its own calls. Enums get awkward here because the agent might reference the same ID across multiple turns, and rebuilding your enum every time is messy.

That's where token mapping comes in.

The idea: create a simple mapping from tricky identifiers (like UUIDs) to easy ones (like ITEM-1, ITEM-2).

// Create token maps at the start of your agent run
const idToToken = new Map<string, string>();
const tokenToId = new Map<string, string>();
let tokenCounter = 1;

// When the agent needs to search or reference items,
// return tokens instead of real IDs
function mapItemsToTokens(items) {
  return items.map((item) => {
    // Reuse existing token if we've seen this item before
    let token = idToToken.get(item.id);
    if (!token) {
      token = `ITEM-${tokenCounter++}`;
      idToToken.set(item.id, token);
      tokenToId.set(token, item.id);
    }

    return {
      id: token, // Return the token, not the real UUID
      title: item.title,
      description: item.description,
    };
  });
}

Now when the agent works with items, it's dealing with ITEM-1, ITEM-2, etc. These are way harder to make up because they're simple and predictable. The agent picks them up quickly and usually sticks with them.

At the end, just decode the tokens back to real IDs:

// Helper for safer decoding
function decodeTokenOrThrow(token: string): string {
  const id = tokenToId.get(token);
  if (!id) throw new Error(`Unknown token: ${token}`);
  return id;
}

const decodedMatches = agentResponse.matches
  .map((match) => {
    const realId = tokenToId.get(match.itemId);
    if (!realId) {
      logger.error(`Agent returned unknown token: ${match.itemId}`);
      return null;
    }
    return {
      itemId: realId,
      confidence: match.confidence,
    };
  })
  .filter((m) => m !== null);

Why this works: Tokens are simple and sequential. The model picks them up easily and reuses them instead of making up new IDs. You've basically given the agent a clean vocabulary to work with.

When to use this: Multi-turn conversations, exploratory agents, any flow where the LLM is doing its own thing and needs to reference items consistently.

Pro tip: Every few turns, remind the model what tokens are active ("Active items: ITEM-1:titleA, ITEM-2:titleB...") to keep things fresh in longer chats.

Comparing the Approaches

Aspect	Enum Constraints	Token Aliasing
Setup Cost	Low (list mapping)	Medium (bidirectional maps)
Prompt Size Impact	High for large lists	Low (small tokens)
Multi-Turn Stability	Requires rebuild each turn	Naturally consistent
Hallucination Prevention	Complete (hard constraint)	Very high (simple patterns)
Best For	Single-shot decisions	Long conversations
Failure Mode	Parse error (retryable)	Unknown token (logged)

Combining Both Strategies

If you're building something more involved, you can mix both approaches:

Use tokens as your main interface to keep hallucinations low
Add enum validation for critical decisions as an extra safety net
Verify everything server-side before you commit the data

This gives you layered protection: the LLM rarely makes mistakes (tokens are simple), and when it does, you catch it.

The Takeaway

LLMs love to make up identifiers. They see the UUID pattern and just run with it. But you've got options:

For structured flows: Use enums to lock them into valid choices
For exploratory flows: Give them simple tokens to work with instead of UUIDs

Either approach cuts down hallucinations significantly. Pick whichever fits your setup better, and you'll save yourself a lot of debugging headaches.

Moving from React to Vue? Here's what to expect

Nikhil Verma — Mon, 21 Aug 2023 14:33:26 +0000

I moved from a React to a Vue project two years ago from a company change. If you are about to undergo a similar journey, this post will help.

I will not cover the basic differences between them. I will instead attempt to explain the conceptual differences that I found when switching.

Embrace Vue's reactivity

My favourite part about Vue is that you can simply modify state. Vue will track the changes and update the computed properties and UI automatically. In React this is done explicitly via setters or using libraries which internally do the same.

Before joining the Vue project I had read how mutating state is bad because you can't track where the changes are coming from. While this is a valid point, in practice a reasonable data organisation and development discipline makes it a non issue.

The productivity gains from modifying a state property and knowing that the UI will update automatically are incredible.

This also integrates well with effects and computed properties. You don't need to pass dependencies in your effects/computed properties, it "just works".

Here is an example code that works in Vue out of the box.

/* state.js */
import { ref } from "vue";
export const count = ref(0);

/* App.vue */
<script setup lang="ts">
import { count } from "./state";
</script>

<template>
    <div>The count is {{ count }}</div>
    <button @click="count++">count++</button>
</template>

Doing something similar in React will need a state management library.

Know your Vue version

There is a big difference between Vue 2 and 3 experiences. With the latter being more polished and better supported in the modern web development experience.

I have written about the pain we went through when migrating from Vue 2 to 3.

In React, you wouldn't have to worry about this as there haven't been major breaking API changes between version updates.

Vue 2 will reach end of life support on December 31st, 2023. My advice would be to plan a migration to Vue 3 as soon as possible.

Understand the core difference between Vue templates and JSX

API differences aside, you must have a basic understanding of how Vue templates operate internally and how they differ from JSX.

Vue templates are closer to HTML, JSX is closer to Javascript. (You might have heard the phrase "JSX is Javascript").

In JSX you can refer to local variables, use Javascript expressions and use JSX inline with other JS code.

In Vue you must define them inside a template block. They can only refer to values you have either provided in your component definition (options API) or present in your setup scope (composition API).

You also can't easily split Vue templates into inline code like JSX. They must be defined in a separate SFC file or a string template inside a render function (which needs runtime compilation).

An example of what I am talking about. Let's take two bits of Vue and React code. They basically behave in the same way. And let's compare their code (as you would write it) and their compiled output.

React

import HelloWorld from "./components/HelloWorld";
import Header from "./components/Header";
import HeaderAlt from "./components/HeaderAlt";

const App = () => {
  const isAlternateHeader = Math.random() > 0.5;
  const items = [1, 2, 3];

  return (
    <HelloWorld
      msg="Vite + React"
      header={isAlternateHeader ? <HeaderAlt /> : <Header />}
    >
      {items.map((item) => (
        <div {...{ [isAlternateHeader ? "alt-header" : "header"]: true }}>
          {item}
        </div>
      ))}

      {isAlternateHeader ? "Alt header" : "Header"}
    </HelloWorld>
  );
};

export default App;

React (as compiled by Babel)

import HelloWorld from "./components/HelloWorld";
import Header from "./components/Header";
import HeaderAlt from "./components/HeaderAlt";
import { jsx as _jsx } from "react/jsx-runtime";
import { jsxs as _jsxs } from "react/jsx-runtime";

const App = () => {
  const isAlternateHeader = Math.random() > 0.5;
  const items = [1, 2, 3];
  return /*#__PURE__*/_jsxs(HelloWorld, {
    msg: "Vite + React",
    header: isAlternateHeader ? /*#__PURE__*/_jsx(HeaderAlt, {}) : /*#__PURE__*/_jsx(Header, {}),
    children: [items.map(item => /*#__PURE__*/_jsx("div", {
      [isAlternateHeader ? "alt-header" : "header"]: true,
      children: item
    })), isAlternateHeader ? "Alt header" : "Header"]
  });
};

export default App;

Now let's look at a similar Vue code

Vue

<script setup lang="ts">
import HelloWorld from "./components/HelloWorld.vue";
import Header from "./components/Header.vue";
import HeaderAlt from "./components/HeaderAlt.vue";

const isAlternateHeader = Math.random() > 0.5;
const items = [1, 2, 3];
const dynamicAttribute = isAlternateHeader ? 'alt-header' : 'header';
</script>

<template>
  <HelloWorld msg="Vite + Vue">
    <template #header>
      <HeaderAlt v-if="isAlternateHeader" />
      <Header v-else />
    </template>

    <div :[dynamicAttribute]="true" v-for="item in items" :key="item">
      {{ item }}
    </div>

    {{ isAlternateHeader ? 'Alt header' : 'Header' }}
  </HelloWorld>
</template>

Vue (compiled by Vue SFC Playground)

import { defineComponent as _defineComponent } from "vue";
import {
  openBlock as _openBlock,
  createBlock as _createBlock,
  renderList as _renderList,
  Fragment as _Fragment,
  createElementBlock as _createElementBlock,
  unref as _unref,
  toDisplayString as _toDisplayString,
  normalizeProps as _normalizeProps,
  createElementVNode as _createElementVNode,
  createTextVNode as _createTextVNode,
  withCtx as _withCtx,
} from "vue";

import HelloWorld from "./components/HelloWorld.vue";
import Header from "./components/Header.vue";
import HeaderAlt from "./components/HeaderAlt.vue";

const __sfc__ = _defineComponent({
  __name: "App",
  setup(__props) {
    const isAlternateHeader = Math.random() > 0.5;
    const items = [1, 2, 3];
    const dynamicAttribute = isAlternateHeader ? "alt-header" : "header";

    return (_ctx, _cache) => {
      return (
        _openBlock(),
        _createBlock(
          HelloWorld,
          { msg: "Vite + Vue" },
          {
            header: _withCtx(() => [
              isAlternateHeader
                ? (_openBlock(), _createBlock(HeaderAlt, { key: 0 }))
                : (_openBlock(), _createBlock(Header, { key: 1 })),
            ]),
            default: _withCtx(() => [
              (_openBlock(),
              _createElementBlock(
                _Fragment,
                null,
                _renderList(items, (item) => {
                  return _createElementVNode(
                    "div",
                    _normalizeProps({
                      [_unref(dynamicAttribute) || ""]: true,
                      key: item,
                    }),
                    _toDisplayString(item),
                    17 /* TEXT, FULL_PROPS */
                  );
                }),
                64 /* STABLE_FRAGMENT */
              )),
              _createTextVNode(
                " " +
                  _toDisplayString(isAlternateHeader ? "Alt header" : "Header"),
                1 /* TEXT */
              ),
            ]),
            _: 1 /* STABLE */,
          }
        )
      );
    };
  },
});
__sfc__.__file = "src/App.vue";
export default __sfc__;

The translation from JSX to JS is simple because tags map to function calls and props map to object properties.

Vue has a more custom DSL where more code is needed to translate it to an executable Javascript code. This also comes into play when using it with Typescript.

Vue's Typescript support is... fine

If you are on a < Vue 2.7 project, you will have a bad experience with Typescript. You might even have to use other libraries to rewrite your Vue components using classes.

Even with a modern Vue 3 project, the Typescript support can be described as "fine". Since Vue templates don’t easily translate to Javascript, it has to do complex compilation to get the types working. Even then, certain things like context (provide/inject) and event arguments need typing manually.

Do you remember the earlier app example? Here is the internal Typescript representation of it's code using Volar (gist.github.com). I had to use a gist because the generated code is huge to allow proper type inference to work.

This is the outcome of using a custom DSL in Vue, it has it's tradeoffs.

Scoped styles out of the box

Vue also provides scoped styles out of the box. This means the following code will not leak CSS.

<style scoped>
.example {
  color: red;
}
</style>

<template>
  <div class="example">hi</div>
</template>

Final thoughts

I still prefer JSX over Vue templates but I vastly prefer the ref/reactive/computed systems of Vue.

My ideal UI library will offer first class template syntax of JSX and reactivity of Vue. Vue does support JSX but it has it's own set of gotchas.

Vue was designed with a certain set of tradeoffs, and it does them really well. If you have a Vue 3 codebase you will not find it too difficult to switch from React.

If you have a Vue 2 codebase however... Read my post here.

Why are you testing?

Nikhil Verma — Fri, 18 Aug 2023 04:55:32 +0000

You might already have some quick answers in mind for this question. I totally get it — I do too.

I'm not here to talk you out of testing or the usual testing processes. I just want to nudge you to think about why you're testing in the first place. Figuring out and questioning the "why" can really steer you towards the right testing methods and tools.

Here are a few questions you need to ask yourself.

Your project

What's the baseline quality that's expected for a project like yours?
Is uniform quality necessary across all interactions? For example, consider the user experience of your sign-up process. Is it worthwhile to channel efforts into refining that experience rather than testing a section that only a few users might visit?

Your audience

If you are building something which has yet to see the light of the day. Would you rather have it

Launch early but in a rough shape but risk a lukewarm reception from initial users?
Launch it later in a polished state but risk finding out that no one's interested?
Which user paths will be the most traveled? Consider interactions like visits to the settings page, are they frequent?

Your team

How big is your team?
Is everyone aligned on what the goal is?
What skill set and experience does your team possess?
What budget are you operating under? How long can you afford to develop before you need to generate revenue.

Considering the context that you are in will help you truly answer the question of "Why are you testing?".

Testing is not a one-size-fits-all approach. It requires thoughtful consideration of your audience, team, and project context.

By asking these questions and reflecting on your unique circumstances, you can shape your testing strategy to deliver a project that truly meets the needs of your users.

From Vue 2 to 3: A long journey

Nikhil Verma — Thu, 17 Aug 2023 14:19:49 +0000

TL;DR

This is a long post detailing my journey of migrating a project from Vue 2 to 3. It contains the setbacks and the victories I faced and providing breakdown of the issues as well.

1. Learning Vue

I joined my current team on May 2021, the project I was working with used Vue 2.6 And Vuex 3.4 and plain Javascript.

Vue 3.0 was already out at that point yet it was not the "default" Vue version. It won't be the case until Feb 7, 2022

2. The need for types

I was new to Vue back then and came from a React and Typescript codebase.

My first commit was to add a tsconfig.json just to have some intellisense. The pain point at that time was that Vue.extend for Vue 2 had poor support for Typescript. I wanted to fix it.

The solution came with the usage of vue-class-component with a combination of vue-property-decorator. Using a combination of the two allowed us to define components in a way that provide a reasonable amount of type safety.

The Vue component code would look something like this

import { Vue, Component, Prop } from 'vue-property-decorator'

@Component
export default class YourComponent extends Vue {
  @Prop(Number) readonly propA: number | undefined
  @Prop({ default: 'default value' }) readonly propB!: string
  @Prop([String, Boolean]) readonly propC: string | boolean | undefined
}

The migration to class components was done incrementally, any new features or components written using the class syntax. Older components were refactored out whenever we needed to change them.

We also switched our VSCode plugin from Vetur to Volar which had better Typescript support.

3. Using Vuex wrong

The Vuex stores were in the "traditional" redux style. They had a file structure like so:

store/
    actions.js
    getters.js
    index.js
    mutations.js
    state.js
    types.js

Good luck finding the right store by name! Or landing on the right action/mutation because there was no support for intellisense.

This got solved by migrating the stores to use vuex-module-decorators which came with a much better support for Typescript. Plus you get all actions/mutations/getters etc in a single file so it's easier to understand. This was also done incrementally.

Here is how the syntax for the decorator stores will look like.

import { Module, VuexModule, Mutation, Action } from 'vuex-module-decorators'

@Module
export default class Counter extends VuexModule {
  count = 0

  @Mutation
  increment(delta: number) {
    this.count += delta
  }

  @Action({ commit: 'increment' })
  incr() {
    return 5
  }
}

4. First setback

There was a spike done into the feasibility of migrating to Vue 3. It didn't last long because of the following reasons:

We used an internal UI library which depended on components that weren't compatible or were unstable with Vue 3 at the time
We couldn't figure out a reasonable strategy to migrate the UI library to Vue 3. Because then we would need to manage two versions of the same library.
We did not want to stop shipping our product while the migration was happening, it either had to be incremental or quick.

5. `vee-validate` and the `$slots` problem

The biggest hurdle that we faced was vee-validate. It is a form validation library built for Vue. The pain was that the library API for Vue 2 and 3 were entirely different.

You would almost call it another library, this was due to the new breaking $slots implementation in Vue 3 (more on that later).

Our UI library had used vee-validate to create a "form builder" that was used over 30 times in our application. We couldn't find an incremental upgrade path here. Either we migrate them all or we migrate none.

6. $slots between Vue 2 and Vue 3

The official migration documentation for slots unification goes over the API differences. It omits a major point: In Vue 3 it's not possible to get the DOM elements of the slots directly from the reference to the slot.

You would need to render your parent, then use the parent's HTML node to get the child slot selector. Or use some kind of inject/provide workaround.

This is the main reason why the vee-validate library could not use it's earlier, simpler API.

Here is a link to the problematic part of the code.

This wouldn't work anymore in Vue 3 because you can't directly extract the DOM nodes of children from the $slots property.

7. A new hope

On 1st July 2022 Vue 2.7 was released. It finally ported many features from Vue 3 back to Vue 2. This was a pivotal moment in our development because we it came with a solid Typescript support out of the box using defineComponent. This was better than vue-class-component because it needed some hacks to get inter-component type inference to work.

@matrunchyk had done a fantastic job creating a codemod to automatically migrate Vue class components to the earlier syntax. It was not perfect but workable.

I made some further modifications to it to get it to work on our project. It's available here if you want to try it for yourself: NikhilVerma/vue-codemod. It's still not polished enough to consider an upstream pull request but you can hack around to migrate your project with it.

We finally migrated to Vue 2.7 and defineComponent using the codemod in a very short time. Also used this to migrate most of the codebase to Typescript. Here is the timeline recorded in our JIRA ticket.

Date	JS Files
5th December 2021	244
22nd Feb 2022	173
23nd Feb 2022	138
24th Feb 2022	109
10th Mar 2022	103
7 Apr 2022	88
26 Apr 2022	54
9 May 2023	24
17 Aug 2023	22

At the same time, we noticed several library dependencies migrated to support Vue 3 with minimal API changes. This was exciting!

8. The solution to vee-validate

Now the only thing preventing a smooth migration was the vee-validate library. We were in luck because our UI team was working on a second version of the Flow UI library which used Lit. It also offered a replacement Form builder which could take over the vee-validate one.

Since the new library used Lit, it was agnostic of Vue 2 or 3 so incremental migration was easy.

9. Let's do this!

The final plan was in place:

Migrate all old form and vee-validate instances with the new Lit ones
Migrate remaining UI components in the library to support Vue 3
Migrate the application to Vue 3 in a single PR

The first step took around a month to complete, as the instances has to be migrated incrementally. The second step was faster, around a week to migrate because most of the other libraries had good backwards compatibility.

The third was easy too. Vue codemod helped a lot, and ESLint + Typescript caught most issues. The lifesaver was our automation suite which made sure we didn't break any critical functionality in the application.

Here is a screenshot of the pull request which migrated our app to Vue 3. (edited out the internal bits)

After almost two years. We migrated to Vue 3. Because we prepared so well the final merge was uneventful. We had a few bugs discovered afterwards, mostly due to how boolean attributes have changed in Vue 3.

10. Aftermath

We are happily using Vue 3 since then. And I hope that future Vue upgrades will be less eventful and not need writing posts like these.

I hope this was helpful to you, I am sure that there are others who are still using Vue 2 on larger codebase and having a tough time with it. And I hope by sharing their experiences it will encourage others to start their migration journey.

XPaths in the modern age

Nikhil Verma — Thu, 01 Jun 2023 13:07:53 +0000

XPaths are selectors you can use to query the elements in your document (HTML, XML etc). They are similar but complex and more featureful versions of CSS query selectors many are used to.

This is an example XPath which can find the Google Search button by starting from the search input. It's a roundabout way of doing this but it shows the powers of XPath over normal query selectors which can't do this.

$x(`//textarea/ancestor::form//input[@value="Google Search"]`);

XPath with Web components

Since XPath is a pretty old specification (created in 1999 and last updated in 2016). It doesn't specify how to operate with Web components and shadow DOM.

So if you want to write an XPath which selects an element inside the shadow DOM of a component you simply can't. The only solution is to find the component with the XPath. Then use the shadowRoot node to then further drill down into the component. For nested components it quickly becomes impractical.

To be fair this problem with shadow DOM is also present in querySelectors which don't work with it either. Lit has it's own section dedicated to using custom @query decorators to find elements.

XPath with Vue 3

There is a very specific and peculiar problem that you face when using XPaths with Vue 3, which took me a long time to debug and find out.

Vue 3 when rendering child slots appends what's called "anchor" nodes to help it optimise it's updates. For example if you have a component like this:

<Text>
   Hello World!
</Text>

And you render it in Vue 3 like this.

<span>
    <slot />
<span>

What Vue 3 will do is output the following DOM structure

<span>
    ""
    "Hello World!"
    ""
</span>

Here "" is an empty text node. This trips up the contains(text()) API of XPath. So if you were relying on your elements containing a specific text you won't be able to do that anymore.

Here is a Github issue with examples. vue/core/issues/8444

A workaround

To solve both of the problems above, I had to ponyfill XPath in our application.

There was an excellent xpath polyfill library which I forked and modified to add support for both Shadow DOM and Vue 3 text nodes.

This makes it "non spec" but it solves our needs and doesn't impact our development velocity.

It's published here xpath-next

import { parse } from 'xpath-next';

const expression = "//span";
const contextNode = document.body;

const nodes = parse(expression).select({ node: contextNode, isHtml: true });

My key takeaway from this experience is reliable technologies stop being so because the ecosystem moves on and makes them incompatible.

One could argue that Vue 3 behaviour should be fixed. But the fact that Web components don't work transparently with XPaths makes it hard to justify using it in it's original specification.

DEV Community: Nikhil Verma

How I'd do engineering for the next year

Three eras in three years

My job is a factory now

Different context, different judgments

Nobody reads every line

The curve is accelerating

What I'd change Monday morning

The discipline: fighting brain rot

Code as a compiled artifact

How I am using Vue with Typescript 7 today

The .vue file was always optional

The heroics you no longer need

The reactivity system comes with you

The proof: TypeScript 7, in production, this week

The migration, briefly

The takeaway

Building a harness that makes a small LLM reliable

The thing I got wrong for too long: the harness is the lever, not the model

Raw IDs are basically a loaded gun

Pattern 1: the ref proxy

Pattern 2: mandatory tool contracts

The thread running through both: recover, don't demand perfection

What changed — and the bit that surprised me

The takeaway

Rate limiting AI APIs across distributed workers

The thing about TPM limits that I got wrong

Layer 1: tell everyone about the 429, immediately (Redis)

Layer 2: jitter on the way back, or you'll do it all again

Layer 3: kill the client library's own retry (this was the villain)

Going proactive: token accounting with a Redis ZSET

The whole picture

Agents should be able to time travel

history.go(-10, "Bruh this path was a dead end")

LLMs as Unreliable Narrators: Dealing with UUID Hallucination

TL;DR

The Core Problem

Solution 1: Enums for Controlled Flows

Solution 2: Deterministic Alias Tokens for Multi-Turn Flows

Comparing the Approaches

Combining Both Strategies

The Takeaway

Moving from React to Vue? Here's what to expect

Embrace Vue's reactivity

Know your Vue version

Understand the core difference between Vue templates and JSX

React

React (as compiled by Babel)

Vue

Vue (compiled by Vue SFC Playground)

Vue's Typescript support is... fine

Scoped styles out of the box

Final thoughts

Why are you testing?

Your project

Your audience

Your team

From Vue 2 to 3: A long journey

TL;DR

1. Learning Vue

2. The need for types

3. Using Vuex wrong

4. First setback

5. vee-validate and the $slots problem

6. $slots between Vue 2 and Vue 3

7. A new hope

8. The solution to vee-validate

9. Let's do this!

10. Aftermath

XPaths in the modern age

XPath with Web components

XPath with Vue 3

A workaround

5. `vee-validate` and the `$slots` problem