DEV Community: Jurij Tokarski

Mirror Your Site as Markdown

Jurij Tokarski — Thu, 28 May 2026 00:00:00 +0000

Cloudflare runs isitagentready.com — a scorecard that grades sites on how well they expose themselves to AI agents. The tool covers five categories (discoverability, content accessibility, bot access control, protocol discovery, commerce), but varstatt.com is a content site, so I ran the ?profile=content view and focused on what it flagged there. I took the gaps as a checklist, shipped what was useful. This post is what came out of that.

The Problem with HTML for Agents

An LLM that lands on a marketing page sees layout markup — divs, buttons, style tags. It has to parse that to extract the content — pricing, FAQ, what the page is selling. It often gets it wrong, or burns tokens guessing.

The fix isn't to rebuild the page. The fix is to expose the same content in a format an agent can read directly. Markdown is the obvious choice — it's what every model was trained on, and it strips the layout noise without losing structure.

The HTML version is for humans. The .md version is the same content, stripped of layout, with stable URLs and structured frontmatter.

Same Content, 10× Fewer Tokens

Page	HTML	Markdown	Reduction
Homepage	23,908	2,130	11×
JSON Formatter	8,315	463	18×
Code audit blog post	16,699	2,159	8×

An agent that wants to know what varstatt.com sells either burns 24K tokens parsing layout, or fetches 2K tokens of clean structured markdown. The interactive toolkit tool is the most extreme — the HTML ships an entire React app for what is, in markdown, a FAQ and a how-it-works list.

Two Entry Points

Agents discover the markdown mirror two ways.

URL suffix. /jurij/p/what-a-code-audit-looks-like.md returns the markdown for that page. Predictable, cacheable, works in any HTTP client.

Accept negotiation. Accept: text/markdown on the same URL returns the same markdown. The original URL stays canonical.

Both routes go through middleware that rewrites them to a single static endpoint:

// /foo.md → /api/markdown/foo
if (pathname.endsWith(".md")) {
  const contentPath = pathname.slice(0, -3) || "/";
  return rewriteToMarkdown(request, contentPath);
}

// Accept: text/markdown → /api/markdown/<path>
if (accept.includes("text/markdown") && isContentPath(pathname)) {
  return rewriteToMarkdown(request, pathname);
}

Prebuilt at Deploy

The markdown route is a Next.js catch-all with force-static and generateStaticParams. Every known content URL — homepage, blog posts, principles, packages, discovery tools, toolkit entries — gets prerendered at next build and served as a static file from the CDN.

export const dynamic = "force-static";
export const dynamicParams = true;

export async function generateStaticParams() {
  return getAllPagePaths().map((path) => {
    if (path === "/") return { path: undefined };
    return { path: path.replace(/^\//, "").split("/") };
  });
}

170 paths prebuilt. The route handler runs once per page at build time, never at request time. No serverless function cost, no cold starts, no runtime errors.

dynamicParams: true means unknown paths still hit the handler — those return a markdown 404 with links to main sections, so even an agent that guesses a wrong URL gets useful context back.

Mirroring JSX Pages

The package pages — /, /jurij/p/what-a-6-week-mvp-build-looks-like, etc. — are MDX files that compose React components. No markdown body. The content lives in component props: ,, ``.

To produce a real markdown mirror, the renderer evaluates the MDX with @mdx-js/mdx, then walks the React tree. Each section component has a markdown stub — a function that takes the same props and returns a string:

`javascript function Pricing({ preset }) { const data = PRICING_PRESETS[preset]; return [ `## ${data.title} — ${data.planLabel}`, `- **Price:** ${data.price} ${data.priceUnit ?? ""}`, ...data.includes.map((i) => ` - ${i}`), ].join("\n"); } `

MDX creates React elements with these stub functions as type, but never invokes them. The walker calls each one manually, joins the results with blank lines. The output is a faithful mirror of the page — same hero, same pricing, same FAQ — as markdown a human could read like a GitHub README.

Toolkit Tools — Render from Frontmatter

The 48 toolkit tools (Base64, JSON formatter, regex tester, and so on) had a different shape. Each MDX file is mostly empty in the body — , , `` — with all the content in YAML frontmatter:

howItWorks:
  - step: "Choose text or file mode"
    description: "Use the Text tab for string encoding..."
faqs:
  - q: "What is Base64?"
    a: "Base64 encodes binary data as ASCII text..."

The markdown handler skips MDX evaluation entirely for tools and renders straight from frontmatter. Each tool gets a structured markdown page: description, How It Works, FAQ, Usage, Related Tools.

The Usage section adds something specific to interactive tools: a note that the tool needs a browser (since markdown can't show a working JSON formatter), plus a URL prefill example for tools that support it:

## Usage

This tool runs entirely in the browser — visit the URL above to use it.

Prefill inputs via URL parameters:

- `https://varstatt.com/toolkit/regex?pattern=...&flags=...&input=...`

Discovery tools get the same treatment — they're AI chats that need a browser, so the markdown explains the tool, lists the questions an agent could prepare answers for, and shows how to prefill the chat with ?context=....

llms.txt and llms-full.txt

llms.txt follows the llmstxt.org convention — a structured index of what's on the site. Services, portfolio, discovery tools (with prefill hints), toolkit, principles, recent blog posts. 170 lines, 14 KB. Generated dynamically from the same content sources as the rest of the site, so it never goes stale.

llms-full.txt is the optional companion: every page's markdown concatenated into one file. 12,459 lines, 651 KB. Each section starts with # so any quoted snippet can be traced back to its source page.

Both are prebuilt static files. The index for navigation, the full file for one-shot loading.

Make the Markdown Discoverable

An agent that hits the HTML version doesn't always know the markdown exists. RFC 8288 has a header for that:

Link: <https://varstatt.com/jurij/p/what-a-code-audit-looks-like.md>; rel="alternate"; type="text/markdown"

Middleware adds it to every HTML response. A HEAD request reveals the markdown URL without any guessing. The agent can switch over without re-fetching.

Robots.txt with Content Signals

Cloudflare's Content Signals spec lets you declare intent for bot use: search, ai-input, ai-train. One line in robots.txt:

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

Then explicit allow blocks for GPTBot, ClaudeBot, PerplexityBot, Google-Extended, and the other AI crawlers. Explicit beats implicit when the spec is still settling.

What I Skipped

Premature standards adoption is one of the worst forms of tech debt — you can't refactor away from a public protocol once agents start consuming it.

Worth shipping: markdown negotiation, llms.txt, llms-full.txt, Link headers, Content Signals, AI bot allowlist.

Skipped: MCP Server Card, WebMCP, Agent Skills, x402, MPP, UCP, ACP. All early specs with no real adoption yet. I'll revisit when they settle.

What This Costs

The whole thing is static. Build time goes up by a few seconds while 170 markdown files get generated. Runtime cost is zero — every .md URL serves from CDN cache. No serverless function invocations, no streaming, no on-demand rendering.

If you ship content from a git repo and rebuild on push, this is the right shape: one renderer, one shared content source, one build step, every page mirrored.

What an Agent Sees Now

Three things changed:

Every page has a .md URL with the full content in clean markdown
The HTML version advertises its .md mirror via Link header
llms.txt indexes everything; llms-full.txt concatenates everything

An agent that lands on the homepage knows where to find the markdown without trying URLs. An agent that wants context loads one file. An agent that wants a specific page fetches one URL. None of it requires the agent to parse HTML.

The site stayed the same for humans. For agents, it became a different surface — one they can actually read.

404 Pages as Lead Magnets

Jurij Tokarski — Wed, 20 May 2026 00:00:00 +0000

The 404 page is the one page on a site that only appears when something went wrong. The default response — "page not found, go home" — confirms the failure and the user bounces.

Tom Orbach made the case that 404 pages should be lead magnets — especially now that AI hallucinations are sending real traffic to URLs that never existed. His version: a branded gift in exchange for an email. I wanted mine to do something different.

The Fake Crash

Hit a broken URL on varstatt.com and you see a pixel-perfect replica of the Next.js dev error overlay. Red bar, stack trace, source preview. For about half a second, you think the site is broken.

Then the bar turns amber. A braille spinner appears. The text changes to "Handling the issue..." After a beat, the bar turns green, the message reads "Handled", and the whole thing morphs into a chat.

Under three seconds. Fast enough that you don't leave, slow enough to notice what happened. The "Handling" state has a minimum dwell of 500ms even if the AI responds in 80ms — without it, the morph feels fake.

The URL Is the First Message

The chat doesn't ask "how can I help?" It reads the URL path as the question:

/pricing → talks about the retainer
/json → links to the JSON formatter
/hire → explains how to engage
/joke-of-the-day → tells you a joke

The system prompt names the requested path and instructs the model to treat it "as if it's a question."

Gibberish paths get "what were you looking for?" Anything with semantic meaning gets a real answer.

The Content Index Is Live

The chat knows every page on the site — pricing, portfolio, 45+ developer tools, blog posts. The index is built at request time from the same sources the sitemap uses. Add a new page, the 404 chat knows about it on the next request. No rebuild.

The AI is also forbidden from inventing URLs. Linking to a page requires calling a search_pages tool first, and only returned paths can be used. The page that exists because of broken URLs doesn't get to invent its own.

Every Question Is a Software Project

Off-topic questions — how to bake bread, how to find love, travel advice — don't get refused. They get answered through the lens of a software project.

Bread is a build pipeline. Love is shipping without user research. Travel is a migration with no rollback strategy.

Delivered straight-faced. Short, dry, designed to be screenshotted. And it's a literal instruction in the system prompt, not an emergent quirk.

Email Capture Without the Popup

Every conversation with three or more messages pings my inbox — full transcript, requested path, any email shared. The lead capture is decoupled from the email field.

The chat does ask for an email after a few messages, but only once. Ignore it and the conversation continues. If someone asks a Varstatt question the index can't answer, the AI says "I don't have that, drop your email and I'll get back to you." The knowledge gap becomes the capture moment.

The Prompt Does Most of the Work

The interesting part isn't the model — it's the instructions. A few lines from the system prompt that shape the whole personality:

"Think in short statements, not paragraphs. 'I don't know' is a complete sentence — no hedging, no disclaimers. Be opinionated by default."

"Give one sharp observation per answer, like a senior dev giving a code review on someone's life choices."

"Write like you're firing off a quick DM, not writing documentation. 1-3 sentences max. No bullet points. No headers. No emojis."

That last block is why the chat never falls into the LLM default of structured, hedged, three-paragraph answers. The model wants to write like documentation. The prompt tells it not to.

Why Bother

The traffic was already hitting that URL. Now it does something.

Break it yourself: varstatt.com/how-to-bake-bread.

Works Locally, Fails After Deployment

Jurij Tokarski — Thu, 30 Apr 2026 07:53:00 +0000

The Production Bugs That Never Threw an Error was about systems that reported success while running the wrong thing. 200 OK, Data Wrong was about APIs that returned clean responses with incorrect output. These four are different — the code was correct, the build passed, but the deploy target had constraints that only surfaced in production.

The File That Wasn't Bundled

Local dev worked. Preview builds worked. The production deployment threw a runtime error trying to read a file that was clearly in the repository.

Vercel's build system uses output file tracing to determine which files a deployment actually needs. When the path to a file is computed at runtime — built from variables, not a literal string — the tracer can't detect the dependency. The file gets left out of the deployment bundle.

// next.config.js
experimental: {
  outputFileTracingIncludes: {
    '/api/report': ['./data/templates/**/*'],
  },
}

Any file read with a computed path needs to be declared here. The tracer won't find it on its own.

The Path That Moved

A Cloud Run service authenticated with a service account JSON file. Locally I used a relative path — ./service-account.json — and it resolved fine because the working directory was my project root. In Cloud Run, if the Dockerfile doesn't set WORKDIR, the container starts in /. The relative path resolved to somewhere that didn't exist, and the error came back as a generic authentication failure rather than a file-not-found.


const keyFile = path.resolve(__dirname, 'service-account.json');

__dirname is the directory of the compiled file, which is stable regardless of where the process is invoked from.

The Routes That Vanished

A CDK stack had a CloudFront distribution with behaviors for several path patterns. /api/* was supposed to route to a Lambda. /* was a fallback to S3. After a deploy, API routes started returning S3 content instead of Lambda responses.

CloudFront applies behaviors in the order they're defined in the array. /* had been added first, and it matched everything — including /api/* — before the more specific pattern had a chance to evaluate. CDK doesn't validate or sort behavior order. Specific patterns first, catch-all last.

The OG Image That Disappeared

Three bugs stacked on one OG image. First: I was rendering at 1200x600. Facebook needs 1200x630 for a full-size preview — anything smaller renders as a tiny thumbnail or gets skipped entirely. Neither platform logs an error. I discovered it by running the URL through Twitter's Card Validator after a campaign showed zero image previews.

Second: the image used CSS Grid layout, clip-path, and backdrop-filter. In the browser it looked right. Satori — the library Next.js uses to render opengraph-image.tsx server-side — only supports a subset of CSS. No display: grid. No pseudo-elements. No clip-path. No backdrop-filter. When it encounters something it doesn't support it doesn't throw — it silently skips the rule or collapses the element.

Third: after fixing dimensions and CSS, I redeployed. Twitter still showed the old version. Social platforms cache OG tags aggressively — Twitter for up to a week, Facebook for days. Redeploying does nothing. The only way to force a re-fetch is through each platform's debug tool.

What They Share

Local dev doesn't have a file tracer. It doesn't change the working directory between builds. It doesn't apply CDN behavior ordering. It doesn't validate OG dimensions or enforce a CSS subset.

Every one of these bugs lived in the gap between what local dev simulates and what production actually enforces. The build passing tells you the code compiles. It tells you nothing about whether the deploy target will accept it.

When the LLM Remembers Too Much

Jurij Tokarski — Tue, 28 Apr 2026 14:14:00 +0000

Multi-step AI pipelines break in production the same way every time. The output looks wrong — stale reasoning, invented constraints, downstream logic that doesn't add up. It looks like a bug in the later step. It's not. The model retained something from a prior step that it shouldn't have.

The Budget That Wasn't

A multi-step pipeline walks users through sequential analysis stages. Market sizing feeds competitive analysis, which feeds the tech stack recommendation, which feeds the cost estimate. Ten steps, each reading prior outputs.

One step produced a vendor rate of $997/week as a reference point for agency development costs. Three steps later, the tech stack recommendation cited "$997/week" as the user's budget. The model had reinterpreted a comparison rate as a constraint.

The prior step outputs were injected as raw text:

<prior_context>
{{costCalculatorOutput}}
</prior_context>

The model had no way to distinguish "this is a vendor rate for comparison" from "this is what the user can spend." The fix was scope guards — labeling what context represents, and what the model should not do with it:

<prior_context step="cost-calculator">
This output contains vendor rate comparisons used for cost estimation.
These are NOT the user's budget. Do not use these figures as budget
constraints in the tech stack recommendation.
{{costCalculatorOutput}}
</prior_context>

The Reasoning That Leaked

An eight-step pipeline ran sequential analysis stages — opportunity analysis, SWOT, bid strategy, and so on. Each step consumed the prior step's output via previous_response_id.

The first symptom was token creep. Responses were getting longer and slower. When I traced what the model was reasoning about in step three, it was hauling in thinking from step one — competitor analysis referencing initial opportunity framing that was no longer relevant. The chain didn't carry forward selected outputs. It carried everything: the model's reasoning, intermediate tool calls, abandoned lines of thought.

The fix was a clean break at every step transition:

function isSyntheticThreadId(threadId: string): boolean {
  return threadId.startsWith("synthetic_");
}

if (source === RunSource.NEXT_STEP) {
  threadId = buildSyntheticThreadId(sessionId);
  message = buildStepStartContext(artifacts, recentMessages)
    + "\n\n---\n\n" + userInput;
}

await llm.streamOrchestrator({
  threadId,
  previousResponseId: isSyntheticThreadId(threadId)
    ? undefined
    : threadId,
  message,
});

At step boundaries, the thread ID resets to a synthetic value. The driver checks isSyntheticThreadId() before setting previous_response_id — if synthetic, the call is fully stateless. Context comes from a structured text block built from the current state, not from the chain's accumulated history.

The Fix That Worked Everywhere

Every failure had the same shape: implicit state — whether it was unscoped context or a chained conversation history — created a contract the code assumed would hold. The model remembered too much, and the wrong things.

The fix was the same each time. Stateless at step boundaries, chained within.

function buildStepStartContext(artifacts: Artifact[], messages: ChatMessage[]): string {
  const lines: string[] = [];
  lines.push("[Previous artifacts — current versions including user edits]");
  for (const artifact of artifacts) {
    lines.push(`<artifact step="${artifact.step}" type="${artifact.type}">`);
    lines.push(artifact.content);
    lines.push(`</artifact>`);
  }
  lines.push("[Recent conversation — last 30 messages]");
  for (const msg of messages.slice(-30)) {
    lines.push(`${msg.role}: ${msg.message}`);
  }
  return lines.join("\n");
}

At every step transition: reset the thread ID, fetch the current state of all artifacts from the database, inject that state as a structured text block with semantic labels. Within a step — retries, follow-ups, agent callbacks — chain normally for conversational continuity.

Each step becomes independently resumable. Contaminated reasoning is gone. A comparison rate stays a comparison rate because the label says so.

The cost surprised me. Stateless reconstruction loses the cached token discount from chaining. Across a full eight-step workflow, it was about 27% cheaper anyway. Early steps have little to cache. Later steps were paying to cache stale content the user had already edited.

Chain for conversational depth within a step. Break the chain and inject labeled context at every step boundary.

Null Bytes, Dead Streams, Last Chunk

Jurij Tokarski — Fri, 24 Apr 2026 08:37:00 +0000

Streaming LLM output to a browser means wiring together SSE, TCP, fetch, and browser lifecycle APIs that weren't designed for this combination. Each one has constraints that only surface when you integrate them.

The Parser That Choked

Server-Sent Events is the natural choice for streaming. SSE supports multiple event types via the event: field and handles multiline JSON by splitting across data: lines. But when every chunk needs an event: line, one or more data: lines, and a blank line delimiter — and you're sending hundreds of small text fragments interleaved with structured tool call events — the framing adds up and the parser becomes more complex than the problem requires.

A null byte as the delimiter is simpler. \0 is rare enough in practice — it can appear as \u0000 in JSON but almost never does in LLM output or natural language — that it works as a reliable record separator without escaping.

// Server: wrap each event
function sendEvent(stream, event) {
  stream.write(JSON.stringify(event) + '\0');
}

// Client: split and route
let buffer = '';
decoder.on('data', (chunk) => {
  buffer += chunk;
  const parts = buffer.split('\0');
  buffer = parts.pop(); // keep the incomplete trailing segment
  for (const part of parts) {
    if (part) handleEvent(JSON.parse(part));
  }
});

Each event is a JSON object with a type field — text_chunk, tool_call, tool_result, done. The client splits on null bytes, parses each segment, routes by type. Text chunks accumulate in the UI. Tool events trigger loading states or commit structured data.

The Stream That Stopped Talking

TCP keepalive keeps a connection open. It doesn't tell you the connection has gone silent at the application level. Occasionally — maybe once every few hundred sessions — a stream stops mid-sentence. No error event. No close event. The connection is alive, the response is still "streaming," and the user is staring at a half-finished message with a spinner that will never resolve.

The LLM API hasn't errored — it just stopped sending chunks.

An idle timer catches this. Reset it on every incoming chunk. Fire it if silence crosses a threshold.

let idleTimer;

function resetIdleTimer(controller) {
  clearTimeout(idleTimer);
  idleTimer = setTimeout(() => {
    controller.abort();
  }, 30_000);
}

stream.on('data', (chunk) => {
  resetIdleTimer(controller);
  processChunk(chunk);
});

stream.on('end', () => {
  clearTimeout(idleTimer);
});

Thirty seconds is generous for interactive chat — users notice after five. The threshold isn't the important part. The pattern is: connection-level timeouts don't catch application-level silence. You need to track it yourself.

The Chunk That Vanished

Browsers kill in-flight fetch() calls during page unload. If you stream audio in chunks via POST, the final chunk — whatever is still buffered when the user stops recording or closes the tab — lives in memory until the next flush. That flush never happens. The final segment of every session is silently dropped.

// Killed on page close:
await fetch('/v3/audio/stream_chunk', { method: 'POST', body: chunk });

// Survives:
fetch('/v3/audio/stream_chunk', {
  method: 'POST',
  body: chunk,
  headers: { Authorization: `Bearer ${token}` },
  keepalive: true,
});

No await. No .then(). You can't await a response during unload — any result is swallowed. Fire and forget. The browser queues the request and completes it even after the page is gone, as long as the total payload is under ~64KB.

navigator.sendBeacon() survives unload too, but it doesn't support custom headers. If your backend expects an auth header, fetch({ keepalive: true }) gives you the full request API.

The Gaps Between Protocols

Every integration has these. You wire together two or three tools that work fine on their own, but nobody tested them together — and no documentation covers the seams. The workarounds aren't published as best practices. They accumulate as know-how, one project at a time. These are three I've accumulated for LLM streaming.

200 OK, Data Wrong

Jurij Tokarski — Tue, 21 Apr 2026 12:23:00 +0000

The Production Bugs That Never Threw an Error was about systems that reported success while running the wrong thing — stale tokens, cached artifacts, stripped paths. These five are different. Every one is an API that accepted valid input, returned a clean 200, and delivered the wrong output. The call worked. The result didn't.

The Image That Wasn't What I Asked For

Imagen has a prompt rewriter enabled by default — an LLM that rewrites your prompt before generation to "add more detail and deliver higher quality images." The rewritten version is only returned in the API response if your original prompt is under 30 words. Above that threshold, you get an image generated from a prompt you never see.

The image I got back was valid, well-composed, and completely wrong. The main subject was replaced by something adjacent. The response was 200. No flag, no warning, no indication that the input was rewritten. I assumed the safety filter had intervened — but the documentation says safety filters either block with an error or omit images entirely. They don't silently substitute. The prompt rewriter does.

Setting enhancePrompt: false in the request disables it. After that, the images matched the prompt.

The File Search That Found Nothing

For retrieval I was uploading source documents via the Files API with file search enabled. One batch worked correctly. Another batch would upload without error but return no results in search.

The difference was the filename. The batch that failed was uploaded with a generic name — something like upload_1 — with no extension. File search uses the filename to infer content type before indexing. A file without a recognized extension gets indexed as an unknown type, and the error is generic enough that it looks like a search quality issue rather than an upload problem.

Adding .pdf, .txt, or the correct extension to every filename at upload time fixed retrieval immediately.

The Transcription That Came Back as Garbage

A voice dictation feature recorded audio in the browser and sent the blob to a Lambda Function URL behind CloudFront. The Lambda passed it to Whisper. The transcription came back — but it was nonsense. No error, no rejection, just wrong text.

Lambda Function URLs base64-encode binary request bodies at the HTTP interface layer. The event includes an isBase64Encoded flag, but if you treat the body as raw bytes in all cases, the buffer is silently corrupted. Whisper doesn't throw on bad audio — it produces garbage.

const audioBuffer = event.isBase64Encoded
  ? Buffer.from(event.body, 'base64')
  : Buffer.from(event.body);

Any Lambda that accepts binary payloads — audio, images, PDFs — needs to check that flag before consuming the body. The cost of missing it is not an error. It's wrong output that looks like a model quality issue.

The Search Console That Had No Traffic

I wired up Google Search Console data fetching for a site with real traffic — I could see it in the GSC web UI. The API call went through, no errors, no 403. It returned zero rows.

The site was registered as a domain property. Domain properties require sc-domain:example.com as the siteUrl, not https://example.com. The API doesn't say "wrong format" or "property not found." It returns empty data as if the site has zero search traffic.

// Returns empty data, no error
const res = await webmasters.searchanalytics.query({
  siteUrl: 'https://example.com',
  requestBody: { startDate, endDate, dimensions: ['query', 'page'] },
});

// Returns actual data
const res = await webmasters.searchanalytics.query({
  siteUrl: 'sc-domain:example.com',
  requestBody: { startDate, endDate, dimensions: ['query', 'page'] },
});

Calling sites.list() shows the exact format the API expects. I spent time checking date ranges and service account permissions before running that call and seeing sc-domain: staring back at me.

Silent Truncation in Structured Outputs

With json_schema and strict: true, OpenAI guarantees valid JSON — except when the response hits max_output_tokens. When that happens, the stream ends with truncated JSON and response.status set to 'incomplete'. This is not surfaced as an error. response.completed still fires normally.

if (event.type === 'response.completed' && 'response' in event) {
  if (event.response.status === 'incomplete') {
    const reason = event.response.incomplete_details?.reason || 'unknown';
    log.error('Response truncated', { reason });
    await params.onChunk('internal.error', 'The AI response was too long and got cut off.');
    await params.onChunk('internal.finished', '');
    return;
  }
  captureUsageStats(event.response.usage);
}

This one had a bonus bug that made it harder to find. Before OpenAI supported structured output streaming, I used XML-like tags in the prompt to get parseable responses — <next_action>, <message>, that kind of thing. When structured outputs shipped, I switched to json_schema but left the XML parser in the catch branch:

try {
  return JSON.parse(input);
} catch (jsonError) {
  return parseXMLResponse(input); // left in "just in case"
}

When a truncated JSON response hit this code, JSON.parse failed, the catch branch fired, and the XML parser found no tags. It returned nextAction: null with the entire raw JSON string stuffed into the message field. The failure surfaced as a null-check bug three layers downstream — not as a parser problem. Dead code from a previous architecture, silently eating every truncation error.

What These Five Have in Common

Every failure surfaced downstream as something that didn't look like an API problem. Corrupted audio looked like a model quality issue. Empty GSC results looked like a permissions problem. Truncation looked like a null-check bug three layers away. The API boundary said success, and the real problem hid behind that signal.

The only reliable defense is asserting on the output, not the status code — the kind of thing a code audit catches systematically. Check that the image matches the prompt. Check that the buffer is actually binary. Check that the response has rows. Check that the JSON is complete. If you only verify that the call succeeded, you'll find the failure when your users do.

Filling Forms No Tool Can Template

Jurij Tokarski — Thu, 16 Apr 2026 10:11:00 +0000

It Works, But You Can't Ship It covered the compliance wall — code execution sandboxes can fill DOCX forms, but they only exist in regions that don't match every customer's data residency policy. This post covers what I learned building the feature before that discovery.

Every Form Is Different

Tender response forms have nothing in common with each other. One agency sends a form with merged table cells and numbered question blocks. The next sends checkboxes inside conditional formatting with section breaks in unexpected places. There is no shared structure, no recurring field names, no predictable layout.

Every DOCX templating tool I evaluated — docx-templates, easy-template-x, docxtemplater — works the same way: you prepare a template with {variable_name} placeholders, pass in data, get a rendered document. That assumes you control the template. Tender forms come from government agencies. You don't control anything. You can't insert placeholders into a form you receive the day the tender opens.

Filling these forms requires understanding an arbitrary document's structure, finding the insertion points, and knowing what content goes where. That's not a deterministic templating problem. It's a comprehension problem.

DOCX Is a ZIP of XML

My first PoC tried the next obvious thing: extract the DOCX to markdown, send it to the model with draft content, get back filled markdown, convert to a new DOCX. Clean pipeline, completely useless output. The regenerated document lost every merged cell, every checkbox, every conditional format. The output was a different document that happened to contain similar text.

A DOCX file is a ZIP archive of XML. word/document.xml holds the content in OOXML format. The correct approach is to give the model the original binary, let it read the XML, find the insertion points, write modifications back, and save the modified ZIP. XML surgery on the original file — not regeneration.

That's the only reliable way to fill DOCX forms with AI — operate on the XML, not on a lossy text conversion. And the only way to do this through an API, without deploying a separate Python service, is a code execution sandbox. Both OpenAI's code_interpreter and Anthropic's code execution tool provide a sandboxed Python environment where python-docx is available and the model can operate on the file directly.

Once that architecture clicked, the API quirks started.

OpenAI: The File Goes in the Container

My first attempt passed the uploaded DOCX as an input_file content block in the user message — the pattern you'd use for images or PDFs.

Expected context stuffing file type to be a supported format... but got .docx

Context stuffing only supports PDFs, images, and plain text. The file has to go into the code_interpreter container instead:

tools: [
  {
    type: 'code_interpreter',
    container: { type: 'auto', file_ids: [uploadedFile.id] }
  }
]

The message itself is plain text — you tell the model the filename so it knows what to look for in the sandbox. No file reference in the content block at all.

Getting the filled file back had its own problem. The SDK exposes client.containers.files.content, which looks callable. It isn't — it's a resource object. The working call is client.containers.files.content.retrieve(containerId, fileId). Neither the types nor the error message make this obvious. I found it by running Object.getOwnPropertyNames on the object at runtime.

Anthropic: Five Things at Once

Claude has the same capability, but it requires five specific pieces in a single request. Miss any one and you get a cryptic failure.

The file upload needs an explicit MIME type — not inferred from the extension. The API call needs two beta flags active simultaneously: files-api-2025-04-14 and code-execution-2025-08-25. The file must be referenced as container_upload in the content block — not document, not file. The tool declaration needs the full versioned type string code_execution_20250825. And the download call needs the same beta flags passed again.

const response = await client.beta.messages.create({
  model: 'claude-sonnet-4-6',
  max_tokens: 16384,
  betas: ['files-api-2025-04-14', 'code-execution-2025-08-25'],
  messages: [{
    role: 'user',
    content: [
      { type: 'text', text: userMessage },
      { type: 'container_upload', file_id: uploaded.id },
    ],
  }],
  tools: [{ type: 'code_execution_20250825', name: 'code_execution' }],
});

No single documentation page covers all five requirements together. Each piece is documented somewhere. The combination isn't.

GPT-5.1 Broke the Document

I tested the form-filling flow across GPT-5.1, 5.2, and 5.4, on both Azure OpenAI and the public API.

GPT-5.1 on Azure — our production deployment — wrote code that opened the DOCX but ignored formatting preservation entirely. Merged cells collapsed, checkboxes vanished, section breaks shifted. The output was a broken document. Same result on the public API — not an infrastructure issue, a model capability issue. GPT-5.2 was inconsistent: partially filled on one test, failed on the next. GPT-5.4 was the first in the lineup that reliably understood the OOXML structure, applied targeted modifications with python-docx, and returned a valid binary with all formatting preserved.

Every Claude Model Could Do It

After the GPT results I tested Claude 4.6 — Opus, Sonnet, and Haiku — through Anthropic's code execution sandbox. Opus and Sonnet completed the task cleanly. The OOXML structure stayed intact, insertions landed in the right cells, formatting survived the round trip. Haiku was inconsistent — similar to GPT-5.2, partially filling on some runs and failing on others.

The gap between the top-performing models was stark. GPT-5.1 couldn't preserve the structure at all. Claude Opus and Sonnet preserved it reliably. The model version matters more than the provider for this task. But which model you can actually deploy depends on where your customer's data is allowed to live — a tech strategy decision, not a code decision. I didn't.

SVG Animation Is Not DOM Animation

Jurij Tokarski — Mon, 13 Apr 2026 10:00:00 +0000

I had a bar chart race sitting in a private repo for over five years. A coding challenge from 2020 or so — built it, moved on, forgot about it. When I started building the toolkit on varstatt.com — free browser-based dev tools — it seemed like an obvious candidate to resurrect.

The new version would be React with SVG, part of a suite: bar chart race, line chart race, area chart race, bubble chart race. Same idea, four visualizations. Upload a CSV, watch the data animate.

Every animation technique I reached for broke in a way I didn't expect.

CSS Transitions Do Nothing on Geometric Attributes

First attempt on the line chart: CSS transitions on SVG elements. transition: cx 300ms ease, cy 300ms ease on the <circle> dots tracking data points. Expected smooth interpolation between positions.

The dots snapped. No animation. Chrome, Firefox, same result.

/* This does nothing useful */
circle {
  transition: cx 300ms ease, cy 300ms ease;
}

CSS transitions animate CSS properties. cx, cy, r, points are not CSS properties — they're SVG attributes. They live in the DOM, but the browser's animation engine doesn't see them. You can change them from JavaScript and the element moves, but there's no interpolation. It jumps.

transform and opacity work because those are actual CSS properties that SVG elements happen to support. Everything that describes SVG geometry — positions, sizes, path data — sits outside that system.

Two Animation Systems on the Same Property

The bar chart race had horizontal bars with CSS transitions on top and width. I set transition: top 1000ms ease-out, width 1000ms ease-out and advanced frames with setInterval. That worked.

Then I switched playback to requestAnimationFrame for continuous interpolation — a float position updating at ~60fps instead of integer jumps every second.

The bars turned jittery. Every RAF tick (~16ms) set a new top value. Each value restarted the 1000ms CSS transition before the previous one completed. The browser's transition engine was fighting the RAF loop. Two animation systems controlling the same property, neither finishing.

// RAF updates position every ~16ms
const top = rank * BAR_HEIGHT;
// CSS transition: "top 1000ms ease-out"
// Every 16ms: cancel current transition, start new 1000ms transition
// Result: jittery mess

The fix was to remove every CSS transition from every element that RAF touches. Bar positions, widths, SVG coordinates, label positions — all computed directly from the playback float.

// RAF computes position directly — no CSS transition
const interpolatedRank = prevRank + (nextRank - prevRank) * frac;
const top = interpolatedRank * BAR_HEIGHT;
// style={{ top, transition: 'none' }}

CSS transitions and requestAnimationFrame are competing strategies. They solve the same problem differently. Layering both on the same property means neither works. I ended up with zero CSS transitions on animated properties across all four chart types.

Colors That Follow Position Instead of Identity

Bubble chart. Bubbles sorted by value each frame so the largest renders on top (correct z-order). Colors assigned by array index after sorting.

Frame 1: Python is biggest, gets index 0, gets blue. Frame 2: JavaScript overtakes Python, gets index 0, gets blue. Python drops to index 1, turns orange. Every frame where the lead changes, half the bubbles swap colors.

// before: color by sorted position
const sorted = [...items].sort((a, b) => b.value - a.value);
sorted.forEach((item, i) => {
  item.color = palette[i % palette.length];
});

The fix: build a color map keyed by series name at parse time, before any sorting happens.

const colorMap = useMemo(() => {
  const map = {};
  data.seriesNames.forEach((name, i) => {
    map[name] = palette[i % palette.length];
  });
  return map;
}, [data.seriesNames, palette]);

data.seriesNames preserves the original CSV column order. It never changes during playback. Sorting for z-order still happens, but it only affects render order, not color. Any visualization where items reorder needs visual properties assigned by identity, never by current array position.

Ranks That Re-Sort Every Tick

Same bar chart race. I was re-sorting bars by their interpolated value on every RAF tick. Values cross each other mid-frame constantly — Python at 11.83 overtakes Java at 11.81 for one tick, then Java is back on top the next. The bars flickered between positions 60 times a second.

The fix: compute sort order only at whole frame boundaries, store it in a pre-computed array, then interpolate rank positions as floats between frames.

const frameRanks = useMemo(() =>
  data.frames.map(frame => {
    const sorted = data.seriesNames
      .map(name => ({ name, value: frame[name] }))
      .sort((a, b) => b.value - a.value);
    const ranks = {};
    sorted.forEach((s, i) => { ranks[s.name] = i; });
    return ranks;
  }), [data]);

// interpolate rank as a float
const rank = (currentRanks[name] ?? idx) * (1 - frac)
           + (nextRanks[name] ?? idx) * frac;
const top = rank * barHeight;

A bar sliding from position 3 to position 1 moves smoothly over the full frame duration instead of jumping. No flickering.

Curves That Rewrite Their Own History

The line and area charts used Catmull-Rom splines. The animation draws a line progressively — like a pen moving across the screen. Curves looked great.

The problem showed up immediately: as the animation advanced and new points entered the spline, the entire line wiggled. Segments already "drawn" shifted into new positions on every frame.

Catmull-Rom computes each segment's control points from the tangent at its endpoints, and the tangent at any point depends on its neighbors. Add a new neighbor, all the tangents change. Feed completed points into the spline function as the animation progresses and every frame recalculates every segment. The old part of the curve is never stable.

The fix split the work into two memos with different dependency arrays.

// Phase 1: compute ALL segments from full dataset, once
const stableGeometry = useMemo(() => {
  const allPoints = data.frames.map((f, i) => ({
    x: xScale(i), y: yScale(f.values[seriesName])
  }));
  const segments = precomputeSegments(allPoints);
  return { allPoints, segments };
}, [data, width, height]);

// Phase 2: reveal progressively, split active segment with de Casteljau
const chartData = useMemo(() => {
  const { segments } = stableGeometry;
  const wholeIdx = Math.floor(position);
  const frac = position - wholeIdx;

  let d = segments.slice(0, wholeIdx).map(s => s.pathData).join('');
  if (frac > 0 && segments[wholeIdx]) {
    const partial = splitBezierAt(segments[wholeIdx], frac);
    d += partial.pathData;
  }
  return d;
}, [stableGeometry, position]);

Pre-compute the full curve from all data points. Completed segments render byte-identical every frame. The active segment gets split at the exact fractional position using de Casteljau subdivision. Historical geometry never depends on current playback position.

The Common Assumption

Each problem came from expecting SVG elements to behave like DOM elements when animated. CSS transitions ignore geometric attributes. RAF and transitions fight over the same values. Array indices aren't stable identifiers when sort order changes. Spline algorithms that look local are global.

The fix was the same every time: compute everything yourself, from one source of truth. Zero CSS transitions on animated properties, all positions derived from a single playback float.

The four chart tools are part of the varstatt.com/toolkit — free, browser-based, no sign-up: bar chart race, line chart race, area chart race, bubble chart race. The old repo from 2020 bears no resemblance to what shipped.

45 Tabs I Stopped Opening

Jurij Tokarski — Thu, 09 Apr 2026 14:45:00 +0000

The JWT decoder I used to reach for sent the token to a server. I noticed because I had DevTools open for something else and saw the POST. A JWT often carries user IDs, emails, roles, expiration data. I'd been pasting production tokens into a stranger's endpoint for months.

That was the first tool I built for the toolkit. The rest followed the same pattern: I needed something, the available options were ad-heavy or required sign-up or made network calls that didn't need to happen. A Base64 encoder doesn't need a backend. Neither does a regex tester, a color converter, or a hash generator.

There are 45 tools now. No sign-up, no tracking, no data collection. Most run entirely in the browser — a few like DNS Lookup and SSL Checker need a server call by nature.

The Catalogue

Encoding — Base64, JWT Decoder, Image to Base64, Encrypt / Decrypt, Hash Generator

JSON & YAML — JSON Formatter, JSON ↔ YAML, YAML Validator

Markdown — Markdown Preview, Text Diff, HTML ↔ Markdown, Markdown to PDF, Markdown to DOCX, CSV Editor

Images — QR Code, Barcode, Image Converter, Favicon Generator, SVG Optimizer, Placeholder Images, Aspect Ratio

Design — Mesh Gradient, CSS Cover Art, Color Converter, Text to Gradient

Charts — Bar Chart Race, Line Chart Race, Bubble Chart Race, Area Chart Race

Network — DNS Lookup, CORS Tester, SSL Checker, OG Tag Validator, HTTP Status Codes, Robots.txt Validator, Sitemap Validator, User Agent Parser

Text — Regex Tester, Case Converter, Slug Generator, Word Counter, Copy Paste Characters

Generators — UUID, Password, Crontab, Unix Timestamp

Most are straightforward. Three outgrew the toolkit and became standalone npm packages.

Text to Gradient

The Text to Gradient tool and the Mesh Gradient Generator both needed the same thing: a way to turn an arbitrary input into a unique, stable visual. Same input, same gradient, every time. No database, no storage.

A djb2-style 32-bit hash is all it takes:

function textHash(str) {
  let hash = 5381;
  for (let i = 0; i < str.length; i++) {
    hash = ((hash << 5) + hash) + str.charCodeAt(i);
    hash = hash >>> 0;
  }
  return hash;
}

Everything derives from that number. hash % palettes.length selects the color palette. seededRandom(hash + layerIndex * 1000) generates position and opacity variation per layer. The same string always produces the same gradient — looks hand-crafted, costs nothing to store.

The gradients themselves are layered radial-gradient() calls. There's no mesh-gradient() in CSS. What works is stacking 6-8 radial gradients positioned at organic spots — 15%, 37%, 63%, 82% — not pure corners or centers, which look algorithmic. Each one uses a 0px first stop for a crisp center and transparent at 50% for soft falloff. The browser composites them in layer order.

background:
  radial-gradient(ellipse at 15% 20%, rgba(120, 40, 200, 0.9) 0px, transparent 60%),
  radial-gradient(circle at 80% 10%, rgba(40, 180, 220, 0.8) 0px, transparent 50%),
  radial-gradient(ellipse at 55% 75%, rgba(200, 60, 120, 0.85) 0px, transparent 55%),
  #1a0a2e;

For tinting — hover states, borders, soft fills — color-mix() handles it without any HSL arithmetic:

background-color: color-mix(in srgb, var(--accent) 12%, white);
border-color: color-mix(in srgb, var(--accent) 25%, transparent);

One thing that cost me time: making these dynamic in Tailwind. A template literal like bg-[color-mix(in_srgb,${color}_12%,white)] silently produces nothing. Tailwind's compiler scans source files for complete static strings at build time. A class assembled from a variable doesn't exist as a string when the scanner runs — it gets skipped with no warning. Inline styles are the fallback for truly dynamic values.

Text to Gradient is now an npm package. It powers the default cover images across the site when a page has no custom visual. Those covers are also animated — which is where the next package came from.

Loopkit

Every tool, blog post, landing page, and discovery step on varstatt.com has an animated SVG cover — all powered by Loopkit. I had ~35 cover designs already in JSX when I started building the engine underneath them. The first decision was whether to keep composable React components or switch to schema-driven JSON.

JSON won because of output flexibility. A React component locks you into JSX. A schema is data — it can render to HTML for OG images, to SVG for exports, to CSS for emails, or to React for the live site. The core engine has no React dependency.

const cover = createCover(schema);
cover.html       // full HTML with inline styles
cover.style      // React style objects
cover.innerHtml  // just the elements
cover.hoverCss   // raw CSS rules

Phase ordering. I had the cycle structured as: animate forward, hold final frame, fade out, loop. Loop restarts were smooth, but the first play() call snapped instantly from the held frame to frame 0. Moving the fade to the beginning of the cycle fixed it — every iteration, including the first, starts with a reverse interpolation from wherever the animation sits, then plays forward.

Hover exits. mouseenter called play(), mouseleave called reset(). The reset snapped to the static frame — functional but mechanical. A settle() method reads the live position and interpolates smoothly from there to the end state over a capped duration. The key: tracking currentAnimElapsed during active animation is what makes settle() possible. Without it, mouseleave can only snap.

Stagger math. In a staggered loop where each element has its own delay, the cycle duration isn't animDuration. It's the time until the last element finishes, plus hold time. Using just animDuration cuts off late-starting elements before they complete.

let lastFinish = 0;
for (const el of schema.elements) {
  const delay = computeDelay(el.animate.sequence ?? 0, schema.stagger ?? 0);
  const duration = el.animate.duration ?? schema.duration ?? 1;
  lastFinish = Math.max(lastFinish, delay + duration);
}
const cycleDuration = lastFinish + holdDuration;

Re-centering all 48 schemas programmatically surfaced one more problem. The centering script computes a bounding box, then shifts coordinates to align with the canvas center. Loopkit schemas use [from, to] arrays for animated values — a bar animates with y: [247, 87]. The bbox script was reading [0], the start value. A bar starting at y=247 with height 180 gave a 427px bounding box on a 280px canvas. The fix was one index: read [1], the end state, because that's the visual rest position.

Loopkit is under 5KB with zero dependencies. It's an npm package now.

Markdown Repository

Markdown Repository began as a utility function inside this site. I query .md and .mdx files by frontmatter — filter by tags, sort by date, paginate. The API looks like Firestore's where/orderBy/limit chain. Once three of my projects used the same copy-pasted code, I extracted it into an npm package. The publish pipeline — trusted publishing with OIDC, no stored tokens — turned into its own post.

The Full List

45 tools, three npm packages. The full list is at varstatt.com/toolkit.

npm Publish Without Tokens

Jurij Tokarski — Tue, 07 Apr 2026 10:35:00 +0000

I published an npm package last week — markdown-repository, a Firestore-style query builder for markdown files. The code worked. The tests passed. The release pipeline took longer to get right than the package itself.

The Old Way

The standard npm publishing workflow uses a long-lived access token. You generate it on npmjs.com, store it as a GitHub Actions secret, and reference it in your workflow:

- run: npm publish
  env:
    NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

It works, but the token never expires, has write access to your packages, and lives in plain text in your CI secrets. If it leaks — through a copied workflow file or a careless log — anyone can publish under your name.

npm's granular tokens improved this slightly. You can scope them to specific packages and set a 90-day expiration. But you still have to rotate them manually.

Trusted Publishing

npm now supports trusted publishing with OIDC. Instead of a stored token, your GitHub Actions workflow proves its identity to npm using a short-lived OpenID Connect credential. npm verifies the credential against the workflow you've authorized, and accepts the publish.

No token to store. No token to rotate. No token to leak.

First Publish Is Manual

Before you can configure trusted publishing, the package must already exist on the registry. npm has no "pending publisher" feature — you can't set up OIDC for a package that doesn't exist yet.

For the very first version, publish from your machine:

npm login
npm publish --access public

I spent a while debugging my workflow before realizing trusted publishing only works from the second release onward. Once the package exists on npmjs.com, go to its settings and add a trusted publisher. From that point, the workflow handles everything.

Setting Up the Workflow

The setup has two parts.

On npmjs.com: go to your package settings, add a trusted publisher. Specify the GitHub org/user, repository, workflow filename, and optionally an environment name.

In the workflow: add id-token: write permission and an environment that matches what you configured on npm.

name: Release

on:
  release:
    types: [published]

permissions:
  contents: read
  id-token: write

jobs:
  publish:
    runs-on: ubuntu-latest
    environment: release
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24.x
          registry-url: https://registry.npmjs.org
          cache: npm
      - run: npm ci
      - run: npm test
      - run: npm run build
      - run: npm publish --provenance --access public

Provenance attestation is automatic with trusted publishing. The --provenance flag is redundant but makes the intent explicit.

The Misleading 404

My first three releases failed with this error:

npm error 404 Not Found - PUT https://registry.npmjs.org/markdown-repository
npm error 404 'markdown-repository@1.1.0' is not in this registry.

The package existed. The version was correct. The OIDC token exchange succeeded — I could see the signed provenance statement in Rekor's transparency log. Everything worked except the actual publish.

The problem: Node 22 ships with npm 10.x. Trusted publishing requires npm 11.5.1 or later.

npm's documentation mentions this requirement. The error message doesn't. A 404 on PUT looks like a registry problem or a package name conflict. Nothing points you toward an npm version mismatch.

The Fix

Use Node 24.x in your workflow. On GitHub Actions, node-version: 24.x resolves to a recent patch that includes npm 11.5.1+ — markdown-repository publishes this way without an explicit npm upgrade.

- uses: actions/setup-node@v4
  with:
    node-version: 24.x

If you're stuck on an older Node version, upgrade npm explicitly:

- run: npm install -g npm@latest

With npm 11.5.1+, the same workflow publishes successfully. No tokens needed.

The Environment Mismatch

The same 404 shows up when the environment name on npmjs.com doesn't match the environment field in your workflow job. If your workflow says environment: release but npm has the environment field blank (or vice versa), the OIDC claims don't match and npm rejects the publish — with a 404, not a meaningful error.

What the Pipeline Looks Like Now

The full workflow for markdown-repository runs lint, tests, and build on every commit. On a GitHub release, it publishes to npm with provenance — no secrets configured anywhere in the repository.

Three Ways the Wrong Value Won

Jurij Tokarski — Tue, 31 Mar 2026 00:00:00 +0000

A user created a tender and immediately couldn't edit it. Not after a day, not after some permission change — immediately. They hit "Create," the page loaded, and the edit button was grayed out.

That was the first bug. It took three fixes across two projects before I understood what connected them: in each case, the value that reached the client wasn't the value I'd computed. Something else got there first — by being faster, by being stale, or by being last in the object literal.

The Value That Arrived Too Early

I pulled up the tender document in Firestore. The ai_driver field was missing entirely. The frontend created tenders like this:

const tenderData = {
  title,
  company_id: companyId,
  ...(companyData?.ai_driver && { ai_driver: companyData.ai_driver }),
};

New companies had no ai_driver set. The conditional spread evaluated to falsy, so the field was never written. That was supposed to be fine — a Cloud Function trigger would set the default after creation.

The Firestore snapshot listener had other plans. It fired before the Cloud Function, saw no ai_driver, and ran this check:

const isDiscontinuedDriver =
  !tender.ai_driver || DISCONTINUED_AI_DRIVERS.includes(tender.ai_driver);

Missing field. Falsy. "Discontinued." Read-only. The user just watched their tender lock itself. Every single tender created by a new company since this code shipped had been born locked.

The fix had two parts. The frontend writes every field it reads immediately after creation — no delegating defaults to triggers:

const tenderData = {
  title,
  company_id: companyId,
  ai_driver: companyData?.ai_driver || DEFAULT_AI_DRIVER,
};

And the discontinuation check had to distinguish "missing" from "actively deprecated":

const isDiscontinuedDriver =
  tender.ai_driver && DISCONTINUED_AI_DRIVERS.includes(tender.ai_driver);

Deployed both. Bug reports kept coming.

The Value That Outlived Its Meaning

Different users, same symptom. Tenders locked on creation. But these companies had ai_driver explicitly set in Firestore — set to assistants-api-gpt4o, a driver I'd discontinued months earlier.

I traced it to the organization settings form:

aiDriver: company.ai_driver || "assistants-api-gpt4o",

That hardcoded fallback was a leftover from migration. New companies had no ai_driver in Firestore, so the form loaded with a dead value nobody could see. The field wasn't even visible on the settings page — it was an internal config, not a user-facing dropdown.

The form submitted its entire state on every save. A user enables a jurisdiction toggle, hits save, and the payload includes ai_driver: "assistants-api-gpt4o". The backend guard:

if (payload.ai_driver) {
  update.ai_driver = payload.ai_driver;
}

Truthy string passes. The discontinued driver gets written to Firestore. Every tender created after that inherits it. The user who toggled a jurisdiction setting three weeks ago has no idea they just broke tender creation for their entire organization.

I dropped the hardcoded fallback. Deployed. Reports kept coming — users had the old bundle cached. Every save from a cached session re-wrote the stale value, undoing any Firestore cleanup I ran manually.

The frontend fix wasn't the real fix. The real fix was backend enum validation:

if (payload.ai_driver && Object.values(AIDriver).includes(payload.ai_driver)) {
  update.ai_driver = payload.ai_driver;
}

The backend rejects any value not in the current enum. Cached bundles, stale defaults, garbage input — all dropped. The frontend can send whatever it wants; the backend is the last line, and it has to act like it.

That stopped the bleeding. But the pattern was already in my head when I opened a different codebase weeks later.

The Value That Was Always Last

I was reviewing a feature flag called ai_chat_enabled. The backend computed it from the user's subscription plan — a careful if/else chain that looked up the plan, checked edge cases, and resolved to a boolean. Solid logic. Well-tested in isolation.

Then I looked at the response builder:

return {
  statusCode: 200,
  body: {
    email: email_address,
    name: name,
    plan: plan,
    ai_chat_enabled: ai_chat_enabled,
    ...customerPreferences,
  },
};

customerPreferences came from DynamoDB. It contained its own ai_chat_enabled key — the raw stored preference, not the computed one. The spread came after the explicit assignment.

JavaScript object literals follow last-writer-wins. The spread silently overwrote the computed value with whatever was sitting in the database. The entire plan-based computation — the lookup, the edge cases, the if/else chain — never reached the client. Not once. Not since the day this code shipped.

The tests checked that the computation logic returned the right boolean. They never checked that the response builder actually used it.

The fix was one line — move the spread before the explicit fields:

return {
  statusCode: 200,
  body: {
    ...customerPreferences,
    email: email_address,
    name: name,
    plan: plan,
    ai_chat_enabled: ai_chat_enabled,
  },
};

Computed values last. Raw data first. The spread provides defaults; the explicit fields override them.

The Wrong Value Always Has a Way In

Timing, staleness, ordering. Three mechanisms, same result: the value I intended never made it. If the frontend reads a field, the backend must validate it. If the backend computes a value, nothing downstream should be able to quietly replace it. The wrong value will always find a way in. The only defense is making sure the right value goes last.

An Empty AI Response Corrupted Chat History

Jurij Tokarski — Thu, 26 Mar 2026 00:00:00 +0000

The spinner ran. The stream closed. The chat bubble stayed empty. No error anywhere.

I was building a conversational discovery tool for founders — a multi-step Gemini-powered flow that walked people through product decisions, collected answers, and built a structured brief. Complex setup: long system prompt, tool definitions, large user messages. Genkit's generateStream handling each turn.

Intermittently, a user would send a message and get nothing back. No timeout, no catch block firing, no non-2xx status. Just a clean stream completion with zero content inside.

What the Logs Said When I Added Them

Standard error handling gives you no signal here:

try {
  const { stream, response } = await ai.generateStream({ ... });
  for await (const chunk of stream) {
    // exits immediately — no chunks arrive
  }
  // response.text() returns ''
  // no exception thrown
} catch (err) {
  // never reached
}

Adding chunk-level logging made it visible. The stream was completing, but the one chunk that arrived looked like this:

Chunk #1 has no content.
Keys: [ 'index', 'role', 'content', 'custom', 'previousChunks', 'parser' ]
role: model
content.length: 0

The content property existed. It wasn't null. It was an empty array. The keys custom, previousChunks, and parser are Genkit's internal markers for a thinking chunk. The model had spent the entire response budget on internal reasoning and had nothing left to output. HTTP 200. Genkit reported success.

Two Ways to Get Nothing

Gemini 2.5 Flash ships with thinking mode enabled by default. Under normal inputs that's fine. Under heavy inputs — long system prompt plus tool definitions plus a long user message — it can exhaust the entire token budget on reasoning before producing a single output token.

There's a second cause that produces the same result: silent rate limiting. Rather than returning a 4xx, Gemini returns a valid, complete, empty stream. The observable symptom is identical. The detection is identical: assert that at least one content chunk arrived after the stream closes.

For the thinking mode case, the fix is one line in the Genkit config:

const { stream, response } = ai.generateStream({
  model: MODEL,
  system: systemPrompt,
  messages,
  tools,
  config: {
    thinkingConfig: { thinkingBudget: 0 },
  },
});

thinkingBudget: 0 disables extended thinking. For a conversational flow where latency matters more than deep reasoning, there's no reason to let the model spend the budget on internal traces.

Fix deployed. I moved on.

The Save That Made It Permanent

What I hadn't checked: the database. Every one of those empty responses had already been saved to Firestore. An empty string is a valid string. The save ran. Nothing flagged it.

The stream handler read finalResult.text after generateStream resolved and wrote it as the AI's message. When thinking mode ate the budget, finalResult.text was "". Firestore now held a record of every affected conversation — each one storing a legitimate-looking AI turn with no content.

History as Poison

When those users came back and sent new messages, getChatHistory pulled their messages from Firestore and formatted them for Gemini:

return messages.map((msg) => ({
  role: msg.role === "ai" ? "model" : "user",
  content: [{ text: msg.content }],
}));

When msg.content is "", that produces { role: "model", content: [{ text: "" }] }. A valid-looking empty model turn in the middle of a real conversation. Gemini received it, interpreted it as unfinished context, entered thinking mode to reason about it, exhausted the budget, returned nothing — which got saved as another empty message, which poisoned the next turn.

The conversation was permanently, silently broken. No exception at any layer. No signal the user could act on. Just a chat that would never respond again.

The Fix That Requires Two Places

Fixing only the stream detection isn't enough — the database is already corrupted. Fixing only the history filter isn't enough — new empty responses can still arrive and be saved. Both defenses are required.

Never write an empty AI message:

const finalText = accumulatedText || finalResult.text || "";
if (finalText) {
  await saveAIMessage(chatId, finalText);
} else {
  console.warn("[StreamHandler] Skipping empty AI message save");
}

And filter empty turns before sending history to the model:

return messages
  .filter((msg) => msg.content)
  .map((msg) => ({
    role: msg.role === "ai" ? "model" : "user",
    content: [{ text: msg.content }],
  }));

Miss either one and the loop can restart. The stream guard stops new corruption. The history filter handles the records already in the database.

The Retry That Made It Worse

The first instinct after detecting an empty stream was to retry. The naive retry called the same send function — which re-inserted the user's message into the messages array. The model received the question twice. On an already-stressed conversation with heavy context, this accelerated the problem rather than resolving it.

The fix is an isRetry flag that skips message insertion on retry calls:

async function streamMessage(content, sessionId, token, { isRetry = false } = {}) {
  if (!isRetry) {
    setChatMessages(prev => [
      ...prev,
      { id: userMsgId, role: 'user', content },
      { id: aiMsgId,   role: 'assistant', content: '' },
    ]);
  } else {
    setChatMessages(prev => [
      ...prev.filter(m => m.id !== aiMsgId),
      { id: aiMsgId, role: 'assistant', content: '' },
    ]);
  }

  await streamAIResponse(sessionId, token);
}

The user message stays in history exactly once. Without this, retry logic breaks an already-broken conversation faster.

Why Every Layer Said "Success"

What made this hard to debug: every layer reported success. HTTP 200, no caught exceptions, valid Firestore writes, clean history formatting. The failure was in the semantics, not the mechanics. An empty model turn is not a successful model turn — and asserting that distinction at each boundary is the only thing that stops the loop.