DEV Community: Raju Dandigam

Debug a Flaky CI Test from Structured Test Replay JSON

Raju Dandigam — Sat, 31 Oct 2026 05:07:09 +0000

A flaky test gives you an unusually valuable experiment: the same test failed and then passed inside one recorded run.

Instead of rerunning it until the failure disappears, compare the attempts.

The Cypress Cloud CLI exposes Test Replay commands, network requests, and console logs as JSON. Its local replay viewer requires version 0.4.0 or newer, and the current CLI documentation requires Node.js 22.21.0 or later. Cloud CLI is currently Beta, so use the latest supported version for exploration and pin the validated version in repeatable automation.

I am a Cypress Ambassador. IDs and outputs below are synthetic, and the workflow was checked against the public Cloud CLI documentation.

Find a flaky test

npm install --global @cypress/cloud@latest
cy-cloud version
cy-cloud status

cy-cloud test list \
  --projectId <projectId> \
  --runNumber <runNumber> \
  --status passed \
  | jq '.tests[]
      | select((.attempts | length) > 1)
      | {testId, testName, attempts}'

A passing status with multiple attempts means at least one earlier attempt did not pass.

Pull both timelines

Replay attempts are one-indexed. If attempt one failed and attempt two passed:

cy-cloud replay timeline \
  --testId <testId> \
  --attempt 1 \
  --commands --network --logs \
  > failed-attempt.json

cy-cloud replay timeline \
  --testId <testId> \
  --attempt 2 \
  --commands --network --logs \
  > passing-attempt.json

Look for the first material divergence, not merely the final assertion.

Evidence	Failed	Passed
Discount response	Arrived after assertion	Arrived before assertion
HTTP status	200	200
Console error	None	None
Displayed total	`$50.00`	`$45.00`

The API did not fail. The relative ordering changed. That points toward synchronization or a product race rather than a bad selector.

Use a consistent classification so the investigation does not become “flaky” versus “not flaky”:

Class	Evidence pattern	Typical next step
Product race	DOM or state differs before the assertion	Fix application coordination
Test synchronization	Expected event occurs later, with no product defect	Wait on that observable event
Test data	Attempts receive different fixtures or records	Isolate and seed deterministically
Environment	Browser crash, unavailable dependency, resource pressure	Repair CI/runtime conditions
Unknown	Recording lacks the decisive boundary	Reproduce with more targeted evidence

The label is a hypothesis backed by cited events, not a substitute for reproduction.

Narrow to the failure window

Large timelines produce noise. Ask for only the commands around the failed assertion plus nearby browser traffic:

cy-cloud replay timeline \
  --testId <testId> \
  --attempt 1 \
  --commands \
  --aroundFailure 5 \
  --network \
  --logs

Use cy-cloud replay info --testId <testId> first when you want a summary of what the replay contains.

Inspect recorded DOM state

replay view requires Cloud CLI 0.4.0+ and a Chromium-family browser:

cy-cloud replay view open --testId <testId>
cy-cloud replay view move --attempt 1 --time failure
cy-cloud replay view inspect \
  --testId <testId> \
  --selector '[data-cy="cart-total"]'

You can also pin before and after a specific command ID found in the timeline. This helps determine whether an earlier action changed the DOM, styles, or attributes before the assertion surfaced the problem.

Give an AI agent a bounded question

Do not ask, “Make the test pass.” Ask:

Compare failed-attempt.json with passing-attempt.json.
Identify the first material difference before the assertion.
Cite the event IDs supporting the conclusion.
Classify the cause as product, synchronization, test data,
environment, or unknown.
Propose a fix, but do not edit files.
Do not add fixed waits or weaken assertions.

That prompt makes unsupported guesses visible. If the evidence is insufficient, unknown is an acceptable answer.

Keep the evidence packet smaller than the entire replay. Include the failed and passing windows, run metadata, browser/version information, and the exact test source revision. Exclude unrelated request bodies and console content. A bounded packet reduces both privacy exposure and the chance that an AI assistant anchors on irrelevant details.

Verify locally

After reviewing the proposal:

reproduce the scenario with controlled data;
apply the smallest change;
run the affected spec repeatedly;
introduce a negative control that should fail;
confirm the test still protects the business behavior.

This article deliberately covers recorded CI evidence, not a live local Cypress session. cypress tap is useful for the final local run, but it solves a different part of the workflow.

For CI-only failures, rerun the reviewed fix in a pinned Cypress Docker image before changing the test. A full image tag or digest holds Node and browser versions steady:

export CYPRESS_IMAGE='cypress/browsers@sha256:<approved-digest>'
docker compose run --rm cypress \
  npx cypress run --spec cypress/e2e/checkout.cy.ts --browser chrome

Store the image digest and Cypress/browser versions with the failed and passing JSON. Docker cannot reproduce provider load or every CI resource constraint, but it can remove silent runner upgrades from the hypothesis set.

Security boundaries

Cloud CLI is read-only and follows the authenticated user's Cypress Cloud permissions. OAuth is recommended locally; CYPRESS_CLOUD_TOKEN is intended for non-interactive environments.

Still review:

which projects the identity can read;
whether the agent can access shell secrets;
which request/response bodies Test Replay recorded;
local replay-cache retention;
whether generated JSON is safe to attach to an issue.

Structured evidence is better than a guess. It is not automatically safe evidence.

Cloud CLI requests are rate limited, so CI automation should fetch once, cache the resulting artifact within the job, and avoid letting several agents repeatedly query the same replay.

What would you learn by comparing your most recent flaky test's failed and passing attempts side by side?

References

Test a Streaming React AI Chat Without Calling an LLM

Raju Dandigam — Thu, 29 Oct 2026 17:58:12 +0000

You do not need a live LLM to reproduce most bugs in an AI chat interface.

These failures are ordinary UI-state problems:

the first token arrives but the spinner remains;
a tool finishes but its card still says Running;
Stop aborts the request, yet a buffered token appears afterward;
retry duplicates the partial answer;
an error removes text the user already received.

Make the stream injectable, then test those event sequences deterministically.

Define an application-level event contract

type ChatEvent =
  | { type: 'token'; messageId: string; value: string }
  | { type: 'tool.started'; id: string; name: string }
  | { type: 'tool.completed'; id: string; summary: string }
  | { type: 'completed'; messageId: string }
  | { type: 'failed'; messageId: string; message: string }

type ChatStream = (
  prompt: string,
  signal: AbortSignal,
) => AsyncIterable<ChatEvent>

The React component receives a ChatStream. Production supplies an SSE or fetch-backed implementation. The component test supplies a controlled source.

A small controlled stream

function createTestStream<T>() {
  const queued: T[] = []
  const waiting: Array<(value: IteratorResult<T>) => void> = []
  let closed = false

  const iterator: AsyncIterableIterator<T> = {
    [Symbol.asyncIterator]() { return this },
    next() {
      if (queued.length > 0) {
        return Promise.resolve({ value: queued.shift()!, done: false })
      }
      if (closed) return Promise.resolve({ value: undefined, done: true })
      return new Promise((resolve) => waiting.push(resolve))
    },
  }

  return {
    stream: async function* () { yield* iterator },
    push(value: T) {
      const resolve = waiting.shift()
      resolve ? resolve({ value, done: false }) : queued.push(value)
    },
    close() {
      closed = true
      waiting.splice(0).forEach((resolve) =>
        resolve({ value: undefined, done: true }),
      )
    },
  }
}

Now the test determines exactly when each event arrives.

Test progressive rendering

it('renders one growing message and a completed tool card', () => {
  const source = createTestStream<ChatEvent>()

  cy.mount(<ChatPanel createStream={source.stream} />)
  cy.findByRole('textbox', { name: 'Message' }).type('Find options')
  cy.findByRole('button', { name: 'Send' }).click()

  cy.then(() => source.push({
    type: 'token', messageId: 'm1', value: 'I found ',
  }))
  cy.findByTestId('assistant-m1').should('have.text', 'I found ')

  cy.then(() => source.push({
    type: 'tool.started', id: 't1', name: 'search_options',
  }))
  cy.findByTestId('tool-t1').should('have.attr', 'data-status', 'running')

  cy.then(() => source.push({
    type: 'tool.completed', id: 't1', summary: '3 results',
  }))
  cy.findByTestId('tool-t1').should('have.attr', 'data-status', 'complete')

  cy.then(() => {
    source.push({ type: 'token', messageId: 'm1', value: 'three.' })
    source.push({ type: 'completed', messageId: 'm1' })
    source.close()
  })

  cy.findByTestId('assistant-m1').should('have.text', 'I found three.')
  cy.findByRole('button', { name: 'Stop generating' }).should('not.exist')
})

Cypress queries retry while React processes the events. No fixed sleeps or model calls are needed.

Test late events after cancellation

Cancellation is a good adversarial case because transports can have buffered data.

cy.findByRole('button', { name: 'Stop generating' }).click()
cy.findByRole('status').should('contain.text', 'Stopped')

cy.then(() => source.push({
  type: 'token',
  messageId: 'm1',
  value: 'this must be ignored',
}))

cy.findByTestId('assistant-m1').should('not.contain.text', 'must be ignored')

The reducer or stream consumer should ignore events belonging to an aborted generation. Also test duplicate tool completions, failure after partial text, retry, and unmount while streaming.

Test the failure state without deleting useful evidence

A transport can fail after several tokens have rendered. Unless the product intentionally discards them, the partial answer should remain visible and be marked incomplete:

cy.then(() => {
  source.push({ type: 'token', messageId: 'm1', value: 'Partial answer' })
  source.push({
    type: 'failed',
    messageId: 'm1',
    message: 'Connection lost',
  })
})

cy.findByTestId('assistant-m1').should('have.text', 'Partial answer')
cy.findByRole('alert').should('contain.text', 'Connection lost')
cy.findByRole('button', { name: 'Retry' }).should('be.enabled')

Use accessible roles for status, alert, progress, and controls so the test protects keyboard and screen-reader behavior as well as component state. A streaming interface that visually updates but never announces completion is still broken for part of its audience.

Do not overuse `cy.clock()`

cy.clock() is useful for retry countdowns, debounce, or a delayed “still working” message. It controls timer APIs. It does not automatically control arbitrary Promises, async iterators, or server delivery.

Control the stream for causality; control the clock only for behavior that genuinely depends on time.

Keep a separate provider contract

This component test does not prove that the real provider emits your ChatEvent schema. Keep a smaller integration test for the adapter:

provider frames parse correctly;
error frames map to failed;
abort closes the request;
secrets and raw prompts are not unintentionally logged.

Model-quality evaluation is another suite again. Separating UI behavior, transport compatibility, and semantic quality keeps failures understandable.

The useful test matrix is event-oriented: success with tokens, success with a tool, failure before the first token, failure after partial output, user cancellation, duplicate events, and retry. It is much smaller and more reproducible than sampling many model sentences.

Which of your current “LLM test” failures could be reproduced as a five-event fixture instead?

References

Add On-Device Summarization to React with Chrome's Summarizer API

Raju Dandigam — Sat, 24 Oct 2026 16:44:16 +0000

A travel brief often mixes the trip itself with meeting context, preferences, and unresolved decisions. A short summary can help the person review it before requesting proposed form changes.

That is a useful place for Chrome’s Summarizer API: a bounded reading aid beside an existing text field.

In Trip Workbench, the public demo I built and maintain, the original brief already feeds a separate Gemini feature that proposes trip fields for review. The local summary has a different purpose. It helps the person read the brief; it does not update the trip, replace the source text, or decide which details the cloud model receives.

This walkthrough builds on commit 4b3f370. The demo has the basic API call. The lifecycle extensions below add capability states, download feedback, cancellation, and a boundary that prevents an old summary from appearing beside a changed brief.

Choose a reading task with a clear boundary

Consider this synthetic brief:

I need to attend a customer workshop in New York on November 12–13, 2026. Depart SFO for JFK on November 11. Economy, one traveler. Do not book a hotel because I will stay with family. Travel only if the client confirms the workshop by Friday. Plan to return November 14, but November 15 is still possible if the workshop extends.

A useful summary would preserve the dates, the hotel exclusion, the confirmation condition, and the uncertainty around the return. A fluent summary that converts those details into a confirmed itinerary would be misleading.

Keep the original visible beside the output. Label the result “Summary of this brief” and let the person regenerate it after editing.

Continue sending the original brief to Gemini when the person selects “Suggest trip details.” Using a lossy summary as the extraction input would make any omitted condition harder for the next step to recover.

The feature needs no application API key or @google/genai call. Chrome supplies the on-device model. Chrome documents that model use does not send the input to Google or a third party, while initial model setup can require a download. That statement applies to the local inference path; the application’s separate Gemini request remains a cloud operation. See Chrome’s Summarizer guide.

Treat availability as part of the interface

Chrome documents Summarizer support from version 138, subject to platform, hardware, storage, and model availability. It remains a limited-availability web API, so browser version alone is insufficient. Serve the feature in a secure context and retain the ordinary text workflow.

There are two checks: whether the API exists, and whether it can support the configuration you want.

Use the same core options for checking availability and creating the instance. This example requests English key points in plain text. The existing demo uses a short tldr; medium key points give this review task more room without guaranteeing that every detail survives.

The examples use JavaScript and JSX. For TypeScript, Chrome recommends @types/dom-chromium-ai. Replace the demo’s handwritten Summarizer global declaration if adopting those types.

In summary-client.js:

export const summaryOptions = {
  type: "key-points",
  format: "plain-text",
  length: "medium",
  expectedInputLanguages: ["en"],
  outputLanguage: "en",
};

export async function checkSummaryAvailability(API = globalThis.Summarizer) {
  if (!API) return "unsupported";
  try {
    const value = await API.availability(summaryOptions);
    return ["available", "downloadable", "downloading", "unavailable"].includes(value)
      ? value : "unknown";
  } catch {
    return "error";
  }
}

Map the availability result to a useful UI state:

State	What the interface should do
Checking	Keep the brief editable while checking support
Unsupported or unavailable	Explain that on-device summarization cannot run here
Downloadable	Offer the action and explain that model setup may require a download
Downloading	Offer to wait for readiness and show progress once attached
Available	Offer summarization immediately
Unknown or error	Offer a readiness retry and preserve the brief

Run this capability check in a parent Effect with a cleanup guard so a late result cannot update an abandoned panel. Do not call create() from that Effect.

Chrome’s setup guidance requires user interaction when model downloads are needed. Let the Summarize button call creation directly. Separating the readiness probe also keeps a potentially slow check out of the click path.

Availability can change. A successful probe is not a reservation; creation still needs error handling. When the language or summary configuration changes, check the new options again.

Own the summarizer for one request

For this occasional button action, one instance per request keeps resource ownership straightforward. Create it, generate the summary, and release it in finally.

Add this function to the same module:

export async function summarizeBrief(
  brief,
  { signal, onUpdate = () => {} },
  API = globalThis.Summarizer,
) {
  signal.throwIfAborted();
  if (!API) throw new Error("On-device summarization is unavailable.");
  if (!brief.trim()) throw new Error("Enter a brief first.");

  let session;
  try {
    session = await API.create({
      ...summaryOptions,
      signal,
      monitor(monitor) {
        monitor.addEventListener("downloadprogress", event => {
          if (!signal.aborted) onUpdate({
            phase: "downloading",
            percent: Math.round(event.loaded * 100),
          });
        }, { signal });
      },
    });
    signal.throwIfAborted();
    onUpdate({ phase: "summarizing" });
    const text = await session.summarize(brief, { signal });
    signal.throwIfAborted();
    return text;
  } finally {
    session?.destroy();
  }
}

The signal goes to both create() and summarize(). Cancelling while creation is pending must reach that operation, too. Aborting a creation signal after it resolves also destroys the resulting instance, so this signal belongs to this one request.

The final abort check prevents a late result from being accepted after cancellation. The finally block releases any instance that was acquired, including when generation fails.

Download progress and generation are separate phases. The Writing Assistance API specification defines download progress as a fraction with a total of 1. A displayed 100% means the download stage reached completion; the interface should still wait for instance creation and summarization to finish.

Destroying an instance releases its resources. It does not mean that the browser removes its downloaded model, nor does this code promise to cancel a shared download for every other page.

Batch output keeps this first implementation small. Streaming can improve perceived responsiveness for longer output, but it also requires managing incomplete text and clearing partial results when the request becomes stale.

Bind the result to the brief that produced it

The current demo creates an AbortController inside its click handler but does not retain it for cancellation. It also stores the resulting text without binding it to a brief version.

A person can therefore edit the brief while a summary is running, or start another request before the first finishes. The last response to arrive can overwrite the display even when it belongs to older input.

A dedicated summary component can own one active controller. This hook cancels earlier work and accepts updates only from the currently active request:

import { useEffect, useRef, useState } from "react";
import { summarizeBrief } from "./summary-client.js";

export function useBriefSummary(brief) {
  const active = useRef(null);
  const [view, setView] = useState({ phase: "idle" });

  useEffect(() => () => {
    active.current?.abort();
    active.current = null;
  }, []);

  function cancel() {
    active.current?.abort();
    active.current = null;
    setView({ phase: "cancelled" });
  }

  async function start() {
    active.current?.abort();
    const controller = new AbortController();
    active.current = controller;
    const isCurrent = () =>
      active.current === controller && !controller.signal.aborted;

    setView({ phase: "preparing" });
    try {
      const text = await summarizeBrief(brief, {
        signal: controller.signal,
        onUpdate(update) {
          if (isCurrent()) setView(update);
        },
      });
      if (isCurrent()) setView({ phase: "done", text });
    } catch (error) {
      if (active.current !== controller) return;
      setView({
        phase: controller.signal.aborted ? "cancelled" : "error",
        message: error?.name === "QuotaExceededError"
          ? "This brief is too long. Choose a smaller section."
          : "Could not summarize on this device. Try again or keep reading the brief.",
      });
    } finally {
      if (active.current === controller) active.current = null;
    }
  }

  return { view, start, cancel };
}

The hook controls request ordering within one mounted summary component. Bind that component’s identity to the source so an edit also clears an already completed summary.

Trip Workbench already tracks briefVersion, editor generation, and session generation. Use them with the draft ID:

const summarySourceKey = JSON.stringify([
  state.sessionGeneration,
  state.editorGeneration,
  state.savedDraft?.id,
  state.briefVersion,
]);

<BriefSummary
  key={summarySourceKey}
  brief={state.brief}
  readiness={readiness}
/>

React resets component state when its key changes. That removes the previous output, and the hook’s cleanup aborts its request. Keep the textarea in the parent so typing retains focus.

Using the existing brief version also invalidates a result when the person edits and then restores the same text. If summary options become editable, include their version in this identity as well.

The availability probe can stay in the parent, since changing the brief does not change this example’s fixed language configuration.

Show useful states and preserve the source

Inside BriefSummary, call the hook and derive whether work is in progress:

const { view, start, cancel } = useBriefSummary(brief);
const busy = ["preparing", "downloading", "summarizing"].includes(view.phase);
const canRun = ["available", "downloadable", "downloading"].includes(readiness);

return (
  <section aria-label="Brief summary">
    <button
      type="button"
      disabled={!canRun || busy || !brief.trim()}
      onClick={() => void start()}
    >
      Summarize on this device
    </button>
    {busy && <button type="button" onClick={cancel}>Cancel</button>}
    <p role="status">
      {view.phase === "preparing" && "Preparing on-device summarization…"}
      {view.phase === "downloading" && `Downloading model: ${view.percent}%`}
      {view.phase === "summarizing" && "Summarizing…"}
      {view.phase === "cancelled" && "Summary cancelled."}
      {view.phase === "done" && "Summary ready."}
      {view.phase === "error" && view.message}
    </p>
    {view.phase === "done" && (
      <>
        <h3>Summary of this brief</h3>
        <p style={{ whiteSpace: "pre-wrap" }}>{view.text}</p>
        <p>Generated on this device. Check details against the original brief.</p>
      </>
    )}
  </section>
);

Place the readiness explanation and a readiness-retry button beside this control in the parent. A disabled action should have a reason the person can read.

Render the plain-text result through React text interpolation. The summary does not need HTML insertion.

Long inputs can exceed the model’s input quota. The summarize() API can reject with QuotaExceededError; the hook turns that into a request to choose a smaller section. For a document-oriented feature, measureInputUsage() and inputQuota can support an earlier check. A character count is not an exact token or quota measurement.

The fallback remains useful: the person can read and edit the original brief, then use the rest of the form. A cloud summary could be a separate action with a clear explanation that it sends the text to a server. Failure of the local path should not silently trigger that transfer.

Test lifecycle behavior and summary quality separately

The existing Summarizer unit test checks the missing-API case. That is a useful start, but it does not establish behavior during downloads, cancellation, or editing.

For the adapter shown here, eight controlled checks pass with a fake Summarizer. They cover availability states, availability errors, cancellation before creation, immediate creation in the call path, cancellation during creation, progress and input forwarding, rejected late output, and cleanup after generation errors.

These are lifecycle results. The React wiring has not been mounted in a browser during this walkthrough, and no native model output or performance measurement is claimed.

Use component tests for replacement requests, brief changes, draft changes, unmounting, and Strict Mode cleanup. Assert that an old result never appears beside a new source. Keep those tests deterministic with deferred promises.

Then run native browser checks with a small set of briefs:

Case	What to inspect
First use on a supported device	Readiness, download feedback, cancellation, and eventual completion
Model already ready	Creation and generation latency without mixing in first-download time
Brief edited during generation	Old output disappears and cannot return
Unsupported configuration	A clear explanation and an intact editing workflow
Long brief	A recoverable quota response with the original text retained
Travel brief above	Dates, negation, conditions, and uncertainty preserved without invented commitments

For the travel brief, record whether the output preserves “do not book a hotel,” the client-confirmation condition, and the uncertain return date. These are evaluation criteria, not recorded model results.

Measure correctness alongside usefulness: how long it takes the person to review the summary and source, how often important details are omitted, and whether the summary saves effort at all. Very short briefs may be faster to read directly.

This gives the feature a clear role in the wider assistant: a local reading aid, tied to one version of the source, that leaves the original available for human review and the later Gemini proposal.

A Read-Only WebMCP Tool Erased My React Form State

Raju Dandigam — Fri, 23 Oct 2026 21:51:19 +0000

The read path in Trip Workbench can erase a person’s unsaved purpose without sending a write.

In a deterministic reproduction, the returned saved snapshot still says “Workshop.” The edit buffer contains “Leadership meeting” before the read, then loses that value when the response is applied.

That failure is reproducible in Trip Workbench at commit 4b3f370, the public React and TypeScript demo I built and maintain. Its get_trip_draft tool refreshes saved state through the same coordinator that manages the React form. The refresh path uses a merge rule designed for a different operation.

The previous article exposed three WebMCP tools: read, update, and validate. This article follows the read path through Chrome’s debugging tools, reproduces the state loss with controlled API responses, and develops a repair that preserves unsaved work.

The reproduction and repair results below are coordinator-level tests. The repair is a local candidate; the linked repository still contains the original behavior. Native Chrome and model-driven verification remain separate steps.

A read must preserve unsaved work

Trip Workbench keeps two representations of a draft:

savedDraft contains the last accepted server snapshot and its revision.
localFields contains the values currently displayed in the form.

A person can change localFields.purpose without changing the server. The editor then reports unsaved changes.

The read tool promises to return saved fields, along with revision, status, and hasUnsavedChanges. It deliberately does not expose the unsaved buffer to the agent.

For example, the server can contain “Workshop” while the form displays “Leadership meeting.” A successful read should return “Workshop,” report that unsaved changes exist, and leave “Leadership meeting” on screen.

The WebMCP adapter also declares readOnlyHint: true. An annotation describes intended behavior; application code still has to honor it.

This matters beyond the disappearing text. The update path rejects agent mutations while the editor is dirty. If a read accidentally clears the buffer and marks it clean, a later agent update can pass that check. The read has changed the conditions under which a write is allowed.

The useful invariant is concrete: refreshing saved data must preserve current unsaved values and must never move an accepted saved revision backward.

Trace the read through Chrome

For local development, Chrome documents the flag chrome://flags/#enable-webmcp-testing. Enable it and relaunch the browser. WebMCP is still a proposed standard, so record the browser version alongside any result. See the current setup instructions.

Open a disposable Trip Workbench draft with a saved purpose of “Workshop.” Keep the application’s cloud AI mode disabled if you are testing only the page tools.

In DevTools, open Application → WebMCP. The WebMCP pane lists available tools and records invocations, including their arguments, results, and status. Select get_trip_draft, enter the active draft ID, and use its manual test controls.

Before running it, change the form’s purpose to “Leadership meeting” without saving.

Inspect three things after the call: the returned saved purpose, the unsaved-changes indicator, and the visible input. A successful tool response alone cannot tell you whether the editor preserved the person’s work.

You can also invoke the same tool from the Console:

const context = document.modelContext;
if (!context) throw new Error("WebMCP is unavailable");

const tools = await context.getTools();
const readTool = tools.find(tool => tool.name === "get_trip_draft");
if (!readTool) throw new Error("Open a trip draft first");

const result = await context.executeTool(readTool, {
  draftId: "REPLACE_WITH_THE_OPEN_DRAFT_ID"
});
console.log(typeof result === "string" ? JSON.parse(result) : result);

The current imperative API takes a discovered tool descriptor and an argument object. JSON-stringified input arguments are deprecated from Chrome 155.

Then use the Sources panel to follow get_trip_draft → refreshSavedDraft. Put a breakpoint after the GET resolves and before the coordinator applies its result. Compare:

the saved fields before the request;
the local fields when it started;
the local fields when it completed;
the response fields and revision.

Chrome’s logpoints are useful when pausing would alter the sequence you are investigating. Log scalar snapshots such as revision, edit version, and dirty state.

A paused JavaScript thread cannot process ordinary typing in that same page. To test typing during a request, use a controlled response delay or a deferred promise instead of assuming that interacting while paused creates the intended race.

Reproduce the failure without timing guesses

The first failure does not require concurrent requests. The edit exists before the read starts.

The coordinator accepts a fetchImpl dependency, so the smallest reproduction can supply a stable server response. This test fits the repository’s existing Vitest setup:

import { expect, it } from "vitest";
import { createCoordinator } from "../../src/client/lib/coordinator.js";
import { defaultTripFields, type TripView } from "../../src/domain/types.js";

it("keeps an unsaved edit when refreshing the saved draft", async () => {
  const saved: TripView = {
    id: "draft-a",
    revision: 1,
    status: "draft",
    fields: { ...defaultTripFields(), purpose: "Workshop" },
    createdAt: "2026-10-17T07:00:00.000Z",
    updatedAt: "2026-10-17T07:00:00.000Z"
  };

  const coordinator = createCoordinator({
    fetchImpl: async <T>() => ({
      ok: true as const,
      data: structuredClone(saved) as T
    })
  });

  await coordinator.openDraft("draft-a");
  coordinator.updateLocalField("purpose", "Leadership meeting");

  const result = await coordinator.refreshSavedDraft("draft-a");
  const state = coordinator.getState();

  expect(result.ok).toBe(true);
  expect(state.savedDraft?.fields.purpose).toBe("Workshop");
  expect(state.localFields?.purpose).toBe("Leadership meeting");
  expect(state.dirty).toBe(true);

  if (result.ok) expect(result.data.hasUnsavedChanges).toBe(true);
});

In the pinned implementation, the assertion for the local purpose fails: it contains “Workshop.”

The relevant comparison is in mergeAfterInFlightEdit:

const humanChanged =
  input.localNow[key] !== input.localAtDispatch[key];

That comparison asks whether the person changed a field after dispatch.

For this refresh, localAtDispatch.purpose and localNow.purpose both contain “Leadership meeting.” The comparison returns false. The helper keeps the server value, even though the person still has an unsaved edit relative to the saved draft.

The surrounding hadLocalEdits check correctly notices that the buffer is dirty. It cannot repair the helper’s choice of comparison baseline.

This also explains why the existing test for typing during an agent update can pass. An edit made after dispatch differs from localAtDispatch; an edit already present does not. Those are different cases.

Give refreshes and saves different merge rules

A confirmed save acknowledges values that the request submitted. Its reconciliation logic needs to distinguish that submitted work from newer typing.

A refresh acknowledges no local edits. Its job is to update the saved snapshot while preserving differences that remain in the current edit buffer.

For the scalar fields in this demo, a refresh can use three values for each field:

Value	Meaning
Base	Current accepted saved value
Local	Current value in the form
Incoming	Value in the newly received server snapshot

If local differs from base, keep local in the form. Otherwise, adopt incoming. If both local and incoming differ from base and disagree with each other, retain the local value as unsaved work and surface the overlap for review.

Here is the candidate helper. It uses the repository’s TripFields type and fieldsEqual function:

function mergeSavedRefresh(
  base: TripFields,
  local: TripFields,
  incoming: TripFields
) {
  const localFields = { ...incoming };
  const overlapped: Array<keyof TripFields> = [];

  for (const key of Object.keys(base) as Array<keyof TripFields>) {
    if (local[key] === base[key]) continue;

    localFields[key] = local[key] as never;

    if (incoming[key] !== base[key] && incoming[key] !== local[key]) {
      overlapped.push(key);
    }
  }

  return {
    localFields,
    dirty: !fieldsEqual(localFields, incoming),
    overlapped
  };
}

Keeping a value in the local buffer does not save it or resolve the conflict. The person still needs to review the changed saved value before deciding what to persist. The candidate adds a notice naming overlapping fields; a field-by-field comparison would make that review easier.

Use the current accepted saved snapshot at response application time as the base. Another operation may have completed while the GET was pending. A local value that was unsaved when the read began may already have been confirmed by a manual save.

This rule treats edits as current value differences. An edit that returns a field to its saved value is no longer an unsaved change. Applications that must preserve edit intent even when values match need additional metadata, such as per-field edit versions.

Reject older snapshots before merging

A correct field merge does not prevent the saved revision itself from moving backward.

The original refresh path also accepts this sequence:

Order	Event	Accepted saved revision
1	A read begins and will return revision 1	1
2	An agent update commits and its response is applied	2
3	The earlier read returns revision 1	Must remain 2

The server has already advanced. Applying the late read makes the browser display older data.

Keep the existing session, editor, and draft identity checks. After those checks succeed, reject a response whose revision is lower than the currently accepted one. Then merge without another asynchronous boundary:

// After checking response success and the active editor identity.
if (res.data.revision < state.savedDraft.revision) {
  return fail(
    "STALE_SNAPSHOT",
    "A newer saved revision is already displayed."
  );
}

const merged = mergeSavedRefresh(
  state.savedDraft.fields,
  state.localFields,
  res.data.fields
);

setState({
  savedDraft: res.data,
  localFields: merged.localFields,
  dirty: merged.dirty,
  ...(merged.overlapped.length ? {
    message:
      `Saved values changed for ${merged.overlapped.join(", ")}. ` +
      "Your local values remain unsaved; review before saving."
  } : {})
});

STALE_SNAPSHOT is a new error in this candidate. It tells the caller that the read lost a race; the caller can read again. It is distinct from the server’s REVISION_CONFLICT response to a stale write.

This comparison assumes revisions increase for one draft and identify immutable versions of its persisted state. It does not order different drafts, replace identity checks, or establish that a snapshot is globally current.

The same rule covers two reads that complete out of order. A response at revision 2 cannot replace an already accepted revision 3 simply because it arrived later.

Check the repair against specific sequences

I exercised the pinned coordinator with controlled API responses, using Node’s test runner after stripping TypeScript syntax from the source modules. The candidate changed the refresh path and added the dedicated merge helper.

These nine checks produced four passes and five failures on the original implementation, and nine passes on the local candidate:

Check	Original	Candidate
Preserve an edit made before refresh	Fail	Pass
Preserve an edit made during refresh	Pass	Pass
Combine a remote destination change with a local purpose edit	Fail	Pass
Preserve overlapping local text and show a review notice	Fail	Pass
Ignore a read older than a confirmed mutation	Fail	Pass
Avoid resurrecting an edit already confirmed by a manual save	Pass	Pass
Ignore a response after switching drafts	Pass	Pass
Preserve typing during an agent update	Pass	Pass
Prevent out-of-order reads from lowering the revision	Fail	Pass

The delayed cases use deferred promises: start the operation, perform the intervening state change, and explicitly release the response. That makes the order reproducible without relying on how quickly someone types.

These results establish the coordinator behavior for those sequences. They do not exercise Chrome’s native dispatch, React rendering, the database, or a model choosing tools. The repository’s results document also lists native and model-driven WebMCP as unverified.

Keep cancellation and recovery in the test plan

Tool registration and an individual invocation have separate lifetimes. Chrome’s imperative API supports a registration signal for removing a tool and an execution signal for cancelling a call. Since Chrome 153, unregistering a tool does not cancel its in-flight executions.

That leaves application responsibilities after navigation: a late result must not update a different editor, and cancelling a network request does not establish whether a server mutation committed.

Trip Workbench uses command receipts to resolve uncertain writes. There is a separate issue in recheckOutcome: it creates a new dispatch context from the current buffer when recovery starts. That loses the original comparison baseline. A later receipt can therefore treat edits made while the outcome was uncertain as if they existed at mutation dispatch.

The refresh repair does not address that path. Receipt recovery needs to retain the original mutation context and reconcile against it, with a dedicated test for typing between an uncertain outcome and a recheck.

For React registration, also test mount, cleanup, and remount. Strict Mode’s extra Effect setup and cleanup cycle in development is useful for exposing registration leaks. A clean tool list after remount is one check; rejecting stale in-flight results is another.

Finish with native and model driven checks

Once the coordinator repair is integrated, repeat the disposable-draft exercise through Chrome’s native WebMCP interface. Check the actual input value and unsaved indicator alongside the invocation output. Verify that an attempted agent update still receives USER_EDIT_IN_PROGRESS while the preserved edit remains unsaved.

Then test whether an agent handles that response appropriately. Chrome’s DevTools MCP integration exposes list_webmcp_tools and execute_webmcp_tool when started with --categoryExperimentalWebmcp.

With a draft already containing an unsaved human edit, use a prompt such as:

Inspect the open trip draft. If it has unsaved changes, explain that I need to save or discard them and stop. Otherwise, set the destination to JFK using the returned revision, then validate the updated draft. Stop before submission.

Record whether the agent reads first, recognizes the unsaved state, and stops. For the clean-draft branch, inspect the revision it supplies and the resulting saved fields. A plausible final message is insufficient evidence that the intended sequence ran.

The engineering contract is visible in the form: reading a draft preserves what the person is working on, and a response from earlier work cannot replace a newer accepted state. Those properties have to hold for every caller, including the agent.

Cypress 16.1.1 Upgrade Checklist: Measure Speed, Flake, Memory, and Secret Boundaries

Raju Dandigam — Thu, 22 Oct 2026 19:09:29 +0000

Cypress 16 brings meaningful runtime, network, retry, and security changes. That does not mean every suite becomes faster simply by changing the package version.

A responsible upgrade answers two questions:

What behavior changed?
Did those changes improve this suite under comparable conditions?

The following checklist targets Cypress 16.1.1, released September 29, 2026. It includes the larger Cypress 16 migration, the 16.1.0 reliability work, and the 16.1.1 patch fixes.

Before upgrading: capture a baseline

Run representative specs on the same CI image, browser, machine size, and application build. Keep at least:

Signal	Why it matters
Total and per-spec duration	Detect broad and localized changes
Retry and flake count	Speed can improve while stability worsens
Peak browser/runner memory	Long specs may degrade late in a run
Crash/hang count	Averages hide catastrophic failures
Browser and Node versions	Prevent invalid comparisons
Network protocol-sensitive assertions	HTTP/2 can change observed headers/status behavior

One run is not a baseline. Use enough repeated runs to see ordinary variance, and report the environment with any published result.

Pin the runner image before comparing

Docker helps only if the image is fixed. Use an official cypress/browsers image with a full version tag or a registry digest, then run both baselines under the same resource limits:

export CYPRESS_IMAGE='cypress/browsers@sha256:<approved-digest>'

docker run --rm \
  -v "$PWD:/e2e" \
  -w /e2e \
  "$CYPRESS_IMAGE" \
  npx cypress run --browser chrome

Avoid latest and moving short tags: a browser or Node update can otherwise enter between the “before” and “after” runs. Record the image digest, npx cypress version, node --version, browser version, CPU, and memory. If the application also runs in Compose, add a health check; container startup order alone does not mean the server is ready.

1. Confirm runtime and framework compatibility

Cypress 16 requires supported Node lines beginning with Node 22, 24, or 26; Node 20 and 25 are no longer supported for binary installation. Component Testing minimums also moved: Angular 21, Vite 8, and Next.js 15.0.4.

node --version
npm ls cypress vite next @angular/core
npx cypress verify

Upgrade one major at a time if the project is several releases behind.

2. Review native browser networking

Chrome, Chromium, and Edge now use the native browser network. The application can negotiate HTTP/1.1, HTTP/2, or HTTP/3 as it would outside Cypress. cy.intercept() still works, but tests that assert transport details may need review:

req.httpVersion;
compression headers;
cached 304 responses;
browser-specific protocol assumptions.

Do not permanently enable the deprecated forceHttp1 escape hatch merely to preserve an old assertion. Fix the assertion to protect user-visible or application behavior instead.

3. Find typing-delay assumptions

The default keystroke delay is now zero. Most tests need no change. Tests that depended on the old implicit delay may expose application debounce or input-handling assumptions.

Search for fields whose behavior changes only when typing slowly. If a product deliberately requires a delay, make it explicit in that test rather than restoring a global slowdown.

4. Move retryable storage assertions to `.should()`

Cookie and storage reads are now query commands. Directly chained assertions re-read state until they pass or time out:

cy.getCookie('session_id').should('exist')

An assertion inside .then() remains a one-time snapshot:

cy.getCookie('session_id').then((cookie) => {
  expect(cookie).not.to.be.null
})

Use .should() when you want retry behavior. Keep .then() when a one-time read is intentional.

5. Review visibility assumptions

The modern visibility algorithm is now the default. Tests relying on legacy-only overflow, transform, or coverage behavior may fail. Prefer assertions that describe what a user can do rather than locking the suite to implementation details.

visibilityStrategy: 'legacy' is a deprecated migration aid, not a long-term configuration.

6. Remove obsolete memory configuration

Browser memory management is stable and enabled by default for Chromium-based browsers. Remove:

experimentalMemoryManagement: true

If the suite intentionally disabled it, migrate to manageBrowserMemory: false and document why. Then compare long-running specs and repeated open-mode runs, not just small happy paths.

7. Migrate secret access

Cypress.env() was removed because it exposed configured environment values in browser context. Use:

cy.env() for sensitive values read asynchronously through the Node process;
Cypress.expose() for public, non-sensitive synchronous configuration.

cy.env(['apiToken'], { log: false }).then(({ apiToken }) => {
  cy.request({
    url: '/api/me',
    headers: { authorization: `Bearer ${apiToken}` },
    log: false,
  })
})

Once yielded, the token is an ordinary value. Do not put it in assertions or error messages.

Also replace removed cy.exec() calls with controlled cy.task() implementations and delete cy.end().

8. Include the 16.1 reliability and patch fixes

Cypress 16.1.0 specifically targeted fewer hangs, lower memory growth, and trusted-certificate caching. Cypress 16.1.1 then fixed cy.clock() cancellation behavior, TypeScript option signatures, shadow-DOM .closest() performance, and dependency security issues. If those problems affected your suite, construct a representative reproduction and compare it separately. Do not attribute every duration improvement to HTTP/2 when a hang fix, browser update, cache state, or application change also occurred.

Report the upgrade honestly

A useful upgrade report looks like this:

Environment: ubuntu-latest, Chrome 146, Node 24
Sample: 20 identical recorded runs before and after
Median duration: ...
P95 duration: ...
Retries: ...
Peak memory: ...
Crashes/hangs: ...
Known confounders: ...

The goal is not to prove that a release is universally faster. It is to determine what changed in your system and preserve evidence for any regression.

Which one of your suite's failures is currently being blamed on “Cypress performance” without a controlled measurement?

References

What Should a Self-Healing Cypress Test Be Allowed to Change?

Raju Dandigam — Thu, 22 Oct 2026 00:02:00 +0000

A checkout test can “heal” and still become wrong.

Imagine that [data-cy="purchase"] disappears. The page now contains two visible buttons labeled Continue: one requests a quote; the other places the order. An AI repair clicks the first match and the test passes because both flows eventually show “Thank you.”

The selector recovered. The purchase behavior is no longer tested.

This is why self-healing needs a change boundary.

I am a Cypress Ambassador. This article complements my CypressConf 2026 session, “When Tests Heal Themselves: Building Trust in AI-Generated Cypress Tests.”

Separate three kinds of change

Change	Example	Review level
Selector repair	New role/name for the same action	Focused review
Step rewrite	New click, wait, and assertion sequence	Full test review
Intent reinterpretation	Different action judged “equivalent”	Product decision

Calling all three “healing” hides the risk. A tool should be allowed to perform only the class of change the team intended.

A useful policy separates automatic analysis from automatic mutation:

Stage	Agent may do	Agent may not do
Diagnose	Read failure evidence and rank causes	Change source files
Propose	Produce a bounded patch and evidence summary	Merge or approve its own patch
Verify	Run the affected spec and negative control	Redefine the expected behavior
Release	Attach review evidence	Bypass branch protections

This preserves the speed benefit of AI-assisted triage without making “the suite is green” the only approval criterion.

Define the allowed envelope

A simple review policy can state:

const repairPolicy = {
  editableFiles: ['cypress/e2e/checkout.cy.ts'],
  allowed: ['selector-replacement', 'wait-for-observable-event'],
  forbidden: [
    'assertion-removal',
    'fixed-delay',
    'force-click',
    'skip-test',
  ],
  requiredPostconditions: [
    'POST /api/orders returns 201',
    'confirmation route reached',
    'persisted order status is confirmed',
  ],
  humanApproval: true,
}

This is an application policy, not a built-in Cypress format. Its purpose is to make “fix the test” reviewable.

Protect intent outside the selector

A robust test uses semantic anchors that survive markup changes:

cy.intercept('POST', '/api/orders').as('createOrder')

cy.findByRole('button', { name: 'Place order' }).click()

cy.wait('@createOrder').then(({ request, response }) => {
  expect(request.body.items).to.have.length.greaterThan(0)
  expect(response?.statusCode).to.eq(201)
})

cy.location('pathname').should('match', /\/orders\/[^/]+\/confirmation$/)

Now replacing the selector with Request a quote does not satisfy the route and request postconditions.

Useful anchors include:

accessible role and name;
destination route;
API operation and payload;
durable server state;
audit event;
permission check;
absence of a forbidden side effect.

Choose the smallest combination that uniquely defines the behavior.

Reject suspicious repair patterns

AI-generated diffs deserve extra scrutiny when they add:

cy.wait(5000)
cy.get('button').first().click({ force: true })
it.skip('places an order', () => { /* ... */ })

Other warning signs are removed assertions, broader selectors, increased timeouts, or replacing a domain assertion with visibility alone.

The repair should provide observable evidence:

Failure: old selector no longer matched
Replacement: role=button, name="Place order"
Changed: selector only
Preserved: order request, confirmation route, persisted status
Not changed: assertions, timeout, force options

Treat the source diff and rerun as authoritative. The summary is a review aid, not proof.

Include an environment fingerprint in that evidence: Cypress, Node, browser, application commit, viewport, and CI image or Docker digest. A repair should first reproduce in the original pinned environment. Otherwise an unreviewed browser or image upgrade can be misclassified as a broken selector—and the agent may rewrite a valid test to accommodate runner drift.

Keep the candidate patch narrow enough for a reviewer to understand in one pass. If the proposed fix crosses application source, fixtures, and assertions, split the work: first repair or clarify the product, then update the test against the reviewed behavior. A large cross-layer patch can make a failing product and a weakened assertion cancel each other out.

Run a negative control

After the repaired test passes, introduce a controlled defect that violates the requirement. For example, stub order creation to fail:

cy.intercept('POST', '/api/orders', {
  statusCode: 409,
  body: { code: 'ORDER_NOT_CREATED' },
})

The purchase test must now fail. If it remains green, the repair weakened the intent.

Negative controls are especially important after step regeneration. They answer the question a successful rerun cannot: Can this test still detect the defect it exists to prevent?

Know the Cypress workflow you are using

cy.prompt() supports generating code that you export and commit, as well as continuous self-healing at runtime. Generated-once code fits ordinary review more naturally because the diff exists before merge. Runtime adaptation needs recorded evidence and tighter restrictions on consequential flows.

Use continuous adaptation cautiously for low-risk presentation changes. Payments, permissions, account deletion, privacy controls, and regulated workflows should normally require an explicit reviewed patch because their business meaning cannot be inferred safely from a nearby passing path.

Cypress currently lists cy.prompt() as Generally Available and Studio AI recommendations as Beta. Release maturity matters, but neither status removes the team's responsibility to define test intent.

The approval rule

Allow automatic repair only when:

the change is inside the declared envelope;
independent postconditions preserve the original behavior;
the code and evidence are reviewable;
a relevant negative control still fails.

Otherwise, escalate instead of healing.

What is the most consequential behavior in your suite that a “helpful” broad selector could accidentally replace?

References

Expose a React Workflow to AI Agents with WebMCP

Raju Dandigam — Mon, 19 Oct 2026 20:14:44 +0000

A person opens a trip draft and asks an assistant:

Prepare my SFO to JFK trip for November 12–15, 2026, for a customer workshop. Use economy for one traveler, check what needs attention, and stop before submission.

The application already knows how to edit a draft, validate its fields, and save it. WebMCP gives a compatible agent a structured way to request those operations in the page the person has open.

I first explored this pattern in a local prototype inside an event-draft wizard. We exposed its title and dates through three tools, routing changes through the same React state handler used by the form. That experiment used synthetic data and direct browser calls.

Trip Workbench, which I built and maintain, carries the pattern into a public demo with an authenticated backend and persisted drafts. Its useful scope is still small: read, update, and validate the current draft.

This walkthrough uses commit 4b3f370. The native and model-driven Trip Workbench journeys remain unverified, and its refresh and receipt-recovery paths still have known local-edit reconciliation defects. Use a clean, disposable draft for the exercise below.

What WebMCP adds to an existing API

WebMCP is a proposed web standard for exposing application functionality to agents. The imperative API lets page JavaScript register a name, description, input schema, and executable handler.

The backend API continues to perform the business operation. WebMCP supplies a discoverable entry point into that operation from the active browser workflow.

That distinction matters for a stateful form. The page knows which draft is open, whether the person has unsaved edits, and how the visible controls relate to application state. A handler can consult that context before calling the backend.

Registering tools does not install an agent or select a model. A compatible agent client must discover and invoke them. This demo also does not require a separate remote MCP server for its page tools.

The WebMCP explainer describes this cooperative interaction between the page, person, and agent. Whether it reduces work for a particular user is something to evaluate.

Trip Workbench’s Gemini “Suggest trip details” feature is a separate integration. That button asks the application server to generate a proposal. The WebMCP path allows an external agent to invoke operations registered by the page; it does not depend on the application’s Gemini key.

Choose tools around a complete task

The three tools form one bounded interaction:

Tool	Contract
`get_trip_draft`	Refresh the authorized saved draft and return fields, revision, status, and an unsaved-changes indicator
`update_trip_draft`	Save an allowed patch against an expected revision
`validate_trip_draft`	Evaluate the saved revision and return field issues and policy findings

These are domain operations. The agent can update a trip without learning which dropdown controls the destination or how the date picker is arranged.

The scope also has explicit edges. A person creates and opens the draft. The tool set contains no create, submit, approval, or ticket-purchase operation.

An update saves the draft immediately when accepted by the backend. Unlike the Gemini proposal feature, there is no second Apply button for this path. The tool description should make that consequence clear.

Leaving submission out of the tool set limits this interface. It does not guarantee human-only submission if an agent has additional browser capabilities and can operate the ordinary UI. Any required confirmation must be enforced at the application action itself.

Describe the allowed change precisely

An update needs more than a generic patch: object.

The agent needs to know the editable fields, allowed values, date representation, and behavior of omitted or cleared values. Trip Workbench allows:

Fields	Constraints
Origin and destination	An airport from the demo catalog, or `null`
Departure and return	A real calendar date in `YYYY-MM-DD` format, or `null`
Trip type	`one_way` or `round_trip`
Passenger count	Integer from 1 to 9
Cabin	`economy`, `premium_economy`, or `business`
Purpose	String, at most 500 characters

An omitted field keeps its saved value. Explicit null clears a nullable field. The patch must contain at least one field, and unknown keys are rejected.

The shared schema module combines that patch contract with the operation context:

export function createUpdateTripDraftArgsSchema(activeDraftId: string) {
  return z.object({
    draftId: z.literal(activeDraftId),
    expectedRevision: ExpectedRevisionSchema,
    commandId: CommandIdSchema,
    patch: TripPatchSchema,
  }).strict();
}

The literal draft ID limits this registration to the currently open resource. It is not an authorization check; the server still decides whether the current user may change that draft.

The repository derives the published JSON Schema from its Zod definition and parses incoming arguments again inside the handler. Keep runtime validation even when a browser inspects the schema. Custom checks, including calendar validity and relationships between fields, still need application code.

In the earlier event-draft prototype, an extra undeclared argument reached the handler in the tested Chrome build. The strict parser rejected it. That observation is a practical reason to keep validation at the application boundary rather than assume discovery metadata enforces every rule.

The two operation identifiers have different jobs:

expectedRevision identifies the saved state the caller intends to change.
commandId identifies one logical mutation. Reuse it only for an exact retry with the same arguments.

A new command based on an outdated revision should conflict. An exact retry can return the original command receipt without applying the change again. These behaviors belong to the backend implementation.

Register tools with the editor lifecycle

Chrome’s current imperative API uses document.modelContext. Feature-detect it and preserve the manual workflow when it is unavailable.

In React, register the tools when the relevant editor context is available, then remove them when that context changes. The registration effect in Trip Workbench follows this shape:

useEffect(() => {
  if (!draftId || draftStatus !== "draft") return;

  const controller = new AbortController();

  registerTripDraftTools({
    draftId,
    canMutate: Boolean(canMutate),
    coordinator: props.coordinator,
    signal: controller.signal,
  });

  return () => controller.abort();
}, [draftId, draftStatus, canMutate, props.coordinator]);

The adapter passes that signal to registerTool. An editable traveler gets all three tools. An observer gets read and validate.

Capability-based registration makes the available interface clearer, but backend permission checks remain necessary. Hiding a write tool does not replace authorization on its HTTP endpoint.

Two signals have different responsibilities: the registration signal removes tools; the execution signal requests cancellation of one invocation. Chrome documents that unregistering does not cancel executions already in flight.

The handler therefore needs its own execution checks, and asynchronous results must still belong to the active draft. A registration effect alone does not solve shared-editor concurrency.

Also verify actual discovery. The presence of document.modelContext establishes API availability, not that every registration succeeded.

Chrome’s current documentation also points React developers to the experimental use-webmcp-tool package. A hook can remove registration boilerplate, but it does not replace the contracts in this example: the active draft ID, runtime schema parsing, coordinator guards, backend authorization, and revision-aware persistence still belong to the application. I use the imperative API here so those boundaries remain visible.

Route the handler through application code

The WebMCP adapter should translate and validate arguments, then delegate to the application operation. It should not directly assign DOM input values or maintain a second set of trip rules.

Here is a shortened update handler using the repository’s schema and coordinator helpers. It assumes draftId and coordinator come from the active registration context:

const updateTool = {
  name: "update_trip_draft",
  description:
    "Save allowed changes to the open trip draft. " +
    "Read its revision first. Refuses unsaved local edits. " +
    "Does not submit the request.",
  inputSchema: updateTripDraftJsonSchema(draftId),

  async execute(
    args: Record<string, unknown>,
    opts?: { signal?: AbortSignal },
  ) {
    if (opts?.signal?.aborted) {
      return JSON.stringify({
        ok: false,
        error: { code: "CANCELLED", message: "Cancelled before dispatch." },
      });
    }

    const parsed = createUpdateTripDraftArgsSchema(draftId).safeParse(args);
    if (!parsed.success) {
      return JSON.stringify({
        ok: false,
        error: { code: "INVALID_INPUT", message: "Invalid draft update." },
      });
    }

    const result = await coordinator.applyAgentPatch(
      parsed.data.patch,
      parsed.data.expectedRevision,
      parsed.data.commandId,
      { signal: opts?.signal },
    );

    return JSON.stringify(result);
  },
};

Inside the adapter, connect that definition to the browser. Here, registrationSignal is the signal supplied by the React effect:

const context = document.modelContext;
if (context) {
  await context.registerTool(updateTool, {
    signal: registrationSignal,
  });
}

The full adapter registers all three definitions and handles registration failures, execution exceptions, and field-level input errors.

The coordinator manages the editor’s saved and local state and calls the authenticated API. The server owns access checks, revision checks, domain validation, and persistence. Keeping those responsibilities together means manual saves and agent updates use the same business rules.

The result should describe the operation’s actual outcome. Trip Workbench returns an application envelope with ok, followed by either data or error. A successful update includes its command receipt and resulting revision. Validation returns its own findings.

For example, PREMIUM_CABIN means the request requires review. It does not mean an exception was approved. Likewise, a successful tool invocation can return ok: false; the agent must inspect the application result.

Useful errors explain what can happen next:

Error	Appropriate next step
`USER_EDIT_IN_PROGRESS`	Ask the person to save or discard their local edits
`REVISION_CONFLICT`	Read current state and reconsider the intended change
`OUTCOME_UNKNOWN`	Reconcile the original command before attempting another mutation

The recovery implementation deserves separate testing. A disconnected request does not establish that a server write was rolled back.

Exercise the tools before involving a model

Use a browser build that supports the documented API. Chrome provides a local WebMCP testing flag and an origin-trial path. Record the full browser version because the API is evolving.

Start the app, sign in as an editable demo user, create a fresh draft, and remain on its editor. Save or discard any local edits before starting.

Chrome’s Application → WebMCP pane lists available tools and supports inspecting and invoking them. Navigating to a different application route can unmount the editor and remove its tools.

For a repeatable console exercise, this helper discovers a tool and parses the application’s JSON-string result:

async function callTripTool(name, args) {
  const tools = await document.modelContext.getTools();
  const tool = tools.find((entry) => entry.name === name);
  if (!tool) throw new Error(`Tool unavailable: ${name}`);

  const raw = await document.modelContext.executeTool(tool, args);
  if (raw === null) throw new Error("Tool navigated without a result");

  const result = JSON.parse(raw);
  if (!result.ok) {
    throw new Error(`${result.error.code}: ${result.error.message}`);
  }
  return result.data;
}

const draftId = location.pathname.match(/^\/trips\/([^/]+)$/)?.[1];
if (!draftId) throw new Error("Open a draft editor first");

const before = await callTripTool("get_trip_draft", { draftId });

const updated = await callTripTool("update_trip_draft", {
  draftId,
  expectedRevision: before.revision,
  commandId: crypto.randomUUID(),
  patch: {
    origin: "SFO",
    destination: "JFK",
    departureDate: "2026-11-12",
    returnDate: "2026-11-15",
    tripType: "round_trip",
    passengerCount: 1,
    cabin: "economy",
    purpose: "customer workshop",
  },
});

console.log(await callTripTool("validate_trip_draft", {
  draftId,
  expectedRevision: updated.toRevision,
}));

The current browser API takes a discovered tool descriptor and an arguments object. This adapter returns a JSON string, which is why the helper parses the result. Keep those input and output conventions distinct.

Inspect the visible fields, returned revision, and validation findings. Refresh the page to check persistence. This tests native invocation with arguments you chose.

Let an agent choose the sequence

For a developer-agent test, Chrome documents a DevTools MCP route using --categoryExperimentalWebmcp. It exposes list_webmcp_tools and execute_webmcp_tool to the connected coding agent.

That is a development and debugging connection. It should be described separately from an end-user browser-agent deployment.

With the agent connected to the signed-in browser session, use this prompt:

Use the WebMCP tools on the currently open Trip Workbench draft. Prepare a round trip from SFO to JFK, November 12–15, 2026, for one traveler in economy, with purpose “customer workshop.” Read the draft first, update it, validate the resulting revision, and explain anything requiring my attention. Stop before submission.

Check whether it discovers the tools, uses the actual draft ID and revision, sends valid arguments, and accurately explains the returned findings. Then inspect the saved result yourself.

Keep the evidence levels separate:

Evidence	What it establishes
Handler tests with a browser substitute	Application behavior for controlled inputs
Direct native browser invocation	Registration and execution in the tested browser
Model-driven journey	How that agent selects tools, supplies arguments, and interprets results

The repository results do not yet establish the last two for Trip Workbench. Record the commit, browser, agent, model, prompt, tool calls, persisted state, and any human correction when running them.

For product feedback, ask whether delegating the task was easier than completing the form and whether the person understood what changed. Include review and correction time in that comparison.

The value of these three tools is a small, understandable interface to an existing workflow. They let us investigate whether an agent can prepare useful draft changes while the application retains its state, rules, and visible user experience.

Build a Reviewable Gemini Form Assistant in React

Raju Dandigam — Sat, 17 Oct 2026 23:56:41 +0000

A person filling out a travel request often already has the information in a message:

Prepare a round trip from SFO to JFK, departing November 12, 2026 and returning November 15, 2026, for one traveler in economy. Purpose: customer workshop.

An AI assistant can turn that brief into form fields. The more interesting problem starts when its response arrives. The person might have changed the dates, corrected the purpose, or opened another request.

I built and maintain Trip Workbench, a React and TypeScript demo with synthetic travel data, to explore that interaction. Its assistance flow lets a person request suggestions, inspect proposed fields and clarification questions, and explicitly apply a change.

The useful unit of work is a proposal tied to a particular draft state. That gives the application something it can validate, display, reject when outdated, and save through its existing API.

Keep generation separate from saving

The feature uses two application operations:

Operation	Responsibility
`POST /api/trips/:id/proposal`	Read the authorized draft, ask Gemini for suggested fields, and validate the response
`PATCH /api/trips/:id`	Apply a user-accepted patch after checking authorization, revision, and domain rules

Generating a proposal leaves the saved trip unchanged. Clicking Apply proposed changes sends a separate mutation. In this demo, applying saves the draft; reviewing and submitting the trip request happens afterward.

That distinction should be visible in the interface. A generic “Accept” button is less clear than a label that explains whether the action fills a local form or saves to the server.

The Gemini call runs in the Node backend using @google/genai. The React client sends its brief and draft context to the application API. Google’s API-key guidance recommends keeping production keys out of browser code.

This article follows that embedded assistance flow. Trip Workbench’s separate WebMCP tools provide another entry point for external agents; they are not involved in the Suggest button.

Define what a proposal can contain

A proposal contains two things: a patch and clarification questions.

For the opening brief, this is an illustrative response:

{
  "patch": {
    "origin": "SFO",
    "destination": "JFK",
    "departureDate": "2026-11-12",
    "returnDate": "2026-11-15",
    "tripType": "round_trip",
    "passengerCount": 1,
    "cabin": "economy",
    "purpose": "customer workshop"
  },
  "clarifications": []
}

The patch contains editable trip fields. It has no place for an owner ID, tenant ID, approval decision, or submitted status.

Partial patches are useful. If the brief says “change the destination to JFK,” the assistant should preserve unrelated fields. If the destination is ambiguous, it can return an empty patch and a question.

For example, “Springfield next Friday” needs more context. This demo supports a small airport list. The assistant should explain that limit and ask for a supported destination instead of silently substituting one.

Here is a stricter proposal schema I would use when extending the adapter. It constrains airports at the schema boundary and rejects unknown keys throughout:

import { z } from "zod";
import { AIRPORTS } from "../../domain/types.js";

const Field = z.enum([
  "origin", "destination", "departureDate", "returnDate",
  "tripType", "passengerCount", "cabin", "purpose",
]);

const ProposedTripSchema = z.object({
  patch: z.object({
    origin: z.enum(AIRPORTS).nullable().optional(),
    destination: z.enum(AIRPORTS).nullable().optional(),
    departureDate: z.string().nullable().optional(),
    returnDate: z.string().nullable().optional(),
    tripType: z.enum(["one_way", "round_trip"]).optional(),
    passengerCount: z.number().int().min(1).max(9).optional(),
    cabin: z.enum(["economy", "premium_economy", "business"]).optional(),
    purpose: z.string().max(500).optional(),
  }).strict(),
  clarifications: z.array(z.object({
    field: Field.nullable(),
    kind: z.enum(["missing", "ambiguous", "unsupported"]),
    question: z.string().min(1).max(240),
  }).strict()).max(8),
}).strict();

The date strings still need calendar validation. A string-shaped value does not establish that February 30 exists.

Define patch semantics explicitly, too: omitting a field preserves its value; null clears a nullable field. The model should omit uncertain information and ask a question. It should clear a field only when the requested change calls for that.

Give Gemini the context needed to interpret the brief

The server loads the current saved draft after checking access. It also supplies a reference date, application timezone, and allowed values.

These details affect interpretation. “Next Friday” depends on a reference date and may still require clarification. An airport mentioned in ordinary language might be outside the application’s supported set.

The prompt should describe those limits and ask for only the requested changes. Treat the brief as input data; instructions embedded in it do not grant additional application capabilities.

At commit 4b3f370, the repository pins @google/genai 1.30.0. Its adapter requests JSON using responseMimeType, then parses and validates the result locally. It does not yet pass an output schema to Gemini.

A focused improvement is to supply the schema during generation as well:

const { GoogleGenAI } = await import("@google/genai");
const ai = new GoogleGenAI({ apiKey: config.geminiApiKey! });

// config and prompt are constructed by the server adapter.
// ProposedTripSchema is the schema defined above.
const response = await ai.models.generateContent({
  model: config.geminiModel,
  contents: prompt,
  config: {
    responseMimeType: "application/json",
    responseJsonSchema: z.toJSONSchema(ProposedTripSchema),
  },
});

if (!response.text) {
  throw new Error("AI_INVALID_OUTPUT");
}

const proposal = ProposedTripSchema.parse(
  JSON.parse(response.text),
);

This excerpt belongs inside the existing adapter, after its API-key check. Its route must translate parsing and validation failures into an application error.

The SDK reference documents responseJsonSchema for this API. It supports a subset of JSON Schema, so keep runtime checks such as string limits and calendar rules even when the schema is supplied to the model. Current Google examples also show the newer responseFormat shape; do not copy fields between SDK generations without checking the version you actually ship.

These examples follow the repository’s models.generateContent path. Google now documents that path under Generate Content API Legacy; its newer Interactions API uses a different request shape. Keep the SDK version and API surface consistent when adapting the code.

The next check is application-specific. Merge the proposed patch with the saved fields and run the existing domain validator. A return date can be a real date and still precede departure. A valid cabin can still require policy review.

Google’s structured-output documentation also calls for application validation. Constraining the response shape does not establish that its values match the person’s intent.

Bind the proposal to the state that produced it

Consider this sequence:

A suggestion request starts with the purpose “customer workshop.”
The person changes the purpose to “leadership meeting.”
The original suggestion arrives with “customer workshop.”

Checking only the server revision misses the problem. The person’s newer text may still be unsaved.

Trip Workbench tracks three versions for this reason:

Version	What it tracks
Saved revision	Changes persisted by the backend
Client edit version	Local form edits
Brief version	Changes to the text being interpreted

The server echoes those bindings outside the model-generated proposal. They are application metadata; Gemini does not choose them.

The coordinator’s freshness check includes:

return (
  bound.boundRevision !== state.savedDraft.revision ||
  bound.boundClientEditVersion !== state.editVersion ||
  bound.boundBriefVersion !== state.briefVersion
);

The surrounding implementation also checks the draft identity, session, and editor generation. That prevents a response for one editor context from being treated as a suggestion for another.

Check freshness when the response arrives and again when the person clicks Apply. The second check matters because a proposal can become outdated while someone is reading it.

The simplest policy is to mark the proposal stale and ask for a new suggestion. For eight fields, that is easier to explain than automatically merging an old proposal with new intent. A larger editor might justify more granular conflict handling.

There is another practical detail: this endpoint uses saved fields as model context. Unsaved form edits are not sent to Gemini. The coordinator blocks an agent-style patch while local edits remain unsaved. A clearer UI would ask the person to save or discard those edits before requesting a suggestion.

Version checks protect this particular handoff. They do not, by themselves, prove that every refresh or recovery path in a shared editor preserves local changes.

Make review useful and apply through the existing API

The current React panel lists proposed values and clarification questions. It disables Apply when the proposal is stale or its patch is empty.

For an existing populated form, I would add a current-value column:

Field	Current	Proposed
Destination	BOS	JFK
Departure	November 10, 2026	November 12, 2026
Purpose	Team meeting	Customer workshop

That table is a suggested review improvement, not a screenshot of the current demo. It makes the consequences of applying easier to inspect.

Keep clarification questions visible beside the fields. A partial proposal can be useful without being ready for submission. In the current design, the person can refine the brief and request another suggestion.

The Apply handler checks freshness, then delegates to the existing coordinator:

if (coordinator.isProposalStale(proposal)) {
  // Keep the form unchanged and ask for another suggestion.
  return;
}

const result = await coordinator.applyAgentPatch(
  proposal.patch,
  proposal.boundRevision,
  newCommandId(),
);

The backend checks the user’s authority, validates the patch against the current draft, and enforces the expected revision. If another writer changed the draft, the application should require a fresh review before applying a different change.

This second validation is necessary even though the proposal was already checked. The draft can change between generation and application. Client-side review is also not an authorization boundary.

A business-cabin request illustrates the distinction: the patch can save successfully while the later validation step returns a review requirement. Saving the draft does not approve a policy exception or purchase a ticket.

Keep failures understandable

The manual form should remain usable when assistance fails.

A rejected model response should leave the fields unchanged and explain that no suggestion could be prepared. A stale proposal should explain that the brief or draft changed. A revision conflict should lead to reviewing current data before trying again.

For a production feature, also release the “Suggesting” state in a finally block and distinguish provider failures from invalid output. The current adapter maps otherwise unclassified provider exceptions to AI_TIMEOUT; that label alone does not establish that a timeout occurred.

Generation failures and save failures need different treatment. A lost proposal response has not saved a trip. A lost PATCH response may have followed a successful save. The latter requires reconciling the original command’s outcome before creating another mutation.

Evaluate the proposal and the handoff separately

Trip Workbench has disabled, fixture, and live modes. Fixture mode returns canned examples so the interface can be exercised repeatedly. Its SFO–JFK example uses fixed dates, so it does not measure Gemini’s extraction ability.

Use deterministic checks for the application behavior:

Check	Expected behavior
Unknown patch key or invalid calendar date	Reject the proposal without changing saved fields
Form edit while generation is pending	Mark the returned proposal stale
Brief change after generation	Prevent applying the old suggestion
Navigation to another draft	Discard the old context’s response
Server revision changes before Apply	Reject the stale mutation
Network failure during generation	Restore usable controls and preserve the form

Then evaluate live Gemini with a complete brief, an ambiguous brief, and a partial update to an existing draft. Run each several times and record the model, prompt, proposed fields, clarification questions, human corrections, and final saved state.

For the partial update, explicitly check what stayed unchanged. A correct destination is not enough if the assistant also overwrote a purpose that the user never asked to change.

The repository’s results document records fixture evidence and leaves live Gemini unverified. These exercises define the next evaluation; they are not a reported success rate.

Finally, compare the complete task with manual entry. Include the time spent writing the brief, waiting, reviewing, and correcting. For a short form, manual entry may be faster. Assistance becomes more interesting when the user already has a longer brief containing the required information.

The question I want feedback on is concrete: can a person understand and correct the proposed change with less effort than entering it themselves?

That is the outcome this design lets us evaluate. The model provides a candidate change, the interface makes it reviewable, and the application decides whether it can be saved.

A Replay-Safe Cypress Cloud Webhook Receiver in TypeScript

Raju Dandigam — Mon, 12 Oct 2026 16:32:50 +0000

Cypress Cloud can send a run.completed webhook when a recorded run reaches a final state. The tempting implementation is:

app.post('/webhook', (req, res) => handle(req.body))

That misses the hard parts: authenticity, replay protection, duplicate delivery, partial failure, and the ten-second response deadline.

Here is the safer shape.

1. Verify the exact raw body

Cypress signs ${timestamp}.${rawBody} with HMAC-SHA256. JSON parsing can change the bytes, so register a raw parser on this route before express.json().

import crypto from 'node:crypto'
import express from 'express'

const app = express()
const secret = process.env.CYPRESS_WEBHOOK_SECRET

if (!secret) throw new Error('Missing CYPRESS_WEBHOOK_SECRET')

app.post(
  '/webhooks/cypress',
  express.raw({ type: 'application/json', limit: '256kb' }),
  async (req, res) => {
    const signature = req.get('X-Cypress-Signature') ?? ''
    const timestamp = req.get('X-Cypress-Timestamp') ?? ''
    const eventType = req.get('X-Cypress-Event') ?? ''
    const eventVersion = req.get('X-Cypress-Event-Version') ?? ''
    const eventId = req.get('X-Cypress-Event-Id') ?? ''

    if (!signature || !timestamp || !eventType || !eventVersion || !eventId) {
      return res.status(400).send('Missing headers')
    }

    const age = Math.abs(Date.now() / 1000 - Number(timestamp))
    if (!Number.isFinite(age) || age > 300) {
      return res.status(400).send('Stale request')
    }

    const expected =
      'sha256=' +
      crypto
        .createHmac('sha256', secret)
        .update(`${timestamp}.${req.body.toString('utf8')}`)
        .digest('hex')

    const actualBytes = Buffer.from(signature)
    const expectedBytes = Buffer.from(expected)
    const valid =
      actualBytes.length === expectedBytes.length &&
      crypto.timingSafeEqual(actualBytes, expectedBytes)

    if (!valid) return res.status(401).send('Invalid signature')

    if (eventType !== 'run.completed' || eventVersion !== '1') {
      return res.status(422).send('Unsupported event type or version')
    }

    const event = JSON.parse(req.body.toString('utf8'))
    const inserted = await storeEventOnce(eventId, event)

    return res.status(inserted ? 202 : 200).send(
      inserted ? 'accepted' : 'duplicate',
    )
  },
)

app.use(express.json())

HMAC proves authenticity and integrity; it does not encrypt the payload. The endpoint still needs HTTPS.

The signature check should happen before schema validation or business routing. Otherwise an attacker can use the parser and downstream code even though the request was never authenticated. Keep the route-specific raw parser ahead of the application's general JSON parser, and reject a payload that is too large before doing expensive work.

The relevant headers have distinct jobs:

Header	Use
`X-Cypress-Signature`	Authenticate the raw bytes
`X-Cypress-Timestamp`	Reject an old signed replay
`X-Cypress-Event`	Route the payload to the correct handler
`X-Cypress-Event-Version`	Select and validate the payload schema
`X-Cypress-Event-Id`	Deduplicate the logical event
`X-Cypress-Request-Id`	Correlate one delivery attempt
`X-Cypress-Idempotency-Key`	Stable delivery identity supplied by Cypress

2. Deduplicate on the event ID

X-Cypress-Event-Id remains stable across automatic retries and manual redelivery. X-Cypress-Request-Id changes on every attempt, so it is the wrong deduplication key.

Persist the event and an outbox job atomically:

async function storeEventOnce(eventId: string, payload: unknown) {
  return db.transaction(async (tx) => {
    const created = await tx.webhookEvents.insertIfAbsent({
      eventId,
      payload,
      receivedAt: new Date(),
    })

    if (!created) return false

    await tx.outbox.insert({
      key: eventId,
      topic: 'cypress.run.completed',
      payload,
    })

    return true
  })
}

Without the transaction, a crash between “mark processed” and “enqueue work” can either lose the action or run it twice.

3. Return quickly

Cypress Cloud times out an attempt after ten seconds. It retries network failures, 408, 429, and 5xx responses, up to ten total attempts with exponential backoff and jitter.

Do not create tickets, fetch replay data, call an LLM, or post several notifications in the request handler. Verify, persist, enqueue, and acknowledge. Let workers do slow work with their own retry and idempotency policy.

Monitor the boundary separately from the worker. Useful receiver metrics include signature failures, stale requests, duplicate rate, persistence latency, acknowledgement latency, and queue depth. A 202 means the event was durably accepted for work; it does not mean the ticket, notification, or triage job completed.

Keep container state disposable

Running the receiver and worker as separate Docker services is useful because they scale and fail differently:

services:
  receiver:
    build: .
    command: ["node", "dist/receiver.js"]
  worker:
    build: .
    command: ["node", "dist/worker.js"]
  postgres:
    image: postgres@sha256:<approved-digest>
    volumes:
      - webhook-data:/var/lib/postgresql/data

volumes:
  webhook-data:

Do not keep accepted event IDs or outbox jobs only in container memory or on an ephemeral filesystem. A new replica must recognize events accepted by an old one. Inject the webhook secret at runtime through the deployment platform's secret mechanism rather than baking it into the image.

Containers make the runtime replaceable; the database transaction makes the event durable.

4. Read the final status

run.completed covers more than passing and failing:

type RunStatus =
  | 'passed'
  | 'failed'
  | 'errored'
  | 'timedOut'
  | 'cancelled'

A webhook's arrival does not mean “deploy.” It means Cypress Cloud finalized the run. Your release system still needs policy for branch, commit SHA, required groups, flaky tests, and other gates.

The Run Completion API can reduce the normal completion delay after expected groups finish, but it is currently Experimental and never terminates active work. The Run Cancellation API is also Experimental and represents a different intent: stop and mark the run canceled.

Test the failure cases

Use Cypress Cloud's Send test action, choose run.completed, and exercise the signed delivery path with its synthetic payload. Test sends are delivered once and are not retried. Add local tests for:

invalid signature;
stale timestamp;
duplicate event ID;
unsupported schema version;
persistence failure;
worker retry after acknowledgement.

The highest-value assertion is not “returned 200.” It is “sending the same event twice created one durable job.”

Also test the crash boundary: persist the event and outbox row, terminate the worker before delivery, restart it, and confirm that exactly one downstream consequence is eventually recorded. The receiver is only as reliable as the persistence and worker protocol behind its quick acknowledgement.

That is the property that makes downstream automation safe.

References

The UI Said 'Saved.' Did the Audit Log Agree?

Raju Dandigam — Sat, 10 Oct 2026 15:44:51 +0000

The green Saved toast answers one narrow question: did the React application render a success message?

It does not prove that the database committed the update. It does not prove that the audit event was written. It does not prove that a notification job reached its queue.

For consequential workflows, test three postconditions:

Surface	Evidence
Browser	The user saw the expected state
API	The durable record contains the change
Side-effect sink	The expected asynchronous event appeared

Example: changing an account role

The UI action remains in the browser. Setup and postcondition checks use HTTP.

it('persists a role change and emits an audit event', () => {
  const subjectId = 'user-role-42'

  cy.request('POST', '/test/fixtures/users', {
    id: subjectId,
    role: 'viewer',
  })

  cy.visit(`/admin/users/${subjectId}`)
  cy.findByLabelText('Role').select('editor')
  cy.findByRole('button', { name: 'Save changes' }).click()
  cy.findByRole('status').should('contain.text', 'Saved')

  cy.request(`/api/users/${subjectId}`)
    .its('body.role')
    .should('eq', 'editor')

  waitForEvent(subjectId, 'role.changed').should('deep.include', {
    from: 'viewer',
    to: 'editor',
  })
})

The visible assertion detects broken user feedback. The API assertion detects an optimistic UI that never committed. The audit assertion detects a missing downstream obligation.

Poll with a budget

The audit event may be produced by a queue worker. Checking once immediately creates a race; waiting an arbitrary five seconds makes every run slow.

Use bounded polling:

function waitForEvent(
  subjectId: string,
  action: string,
  attemptsLeft = 8,
): Cypress.Chainable<Record<string, unknown>> {
  if (attemptsLeft === 0) {
    throw new Error(`No ${action} event found for ${subjectId}`)
  }

  return cy
    .request(`/test/audit-events?subjectId=${subjectId}`)
    .then(({ body }) => {
      const event = body.events.find(
        (candidate: { action: string }) => candidate.action === action,
      )

      if (event) return event

      return cy
        .wait(500)
        .then(() => waitForEvent(subjectId, action, attemptsLeft - 1))
    })
}

This example has a four-second observation window. Choose the real value from the workflow's latency expectation. If the product promises that the audit event is available within two seconds, a test that waits thirty seconds hides a regression.

Give every test a correlation key

Parallel tests can create similar events. Querying for “the latest role.changed event” may find another test's data and pass incorrectly.

The fixture should carry a unique subject or correlation value:

const correlationId = `cy-${Cypress.spec.name}-${Date.now()}`

cy.request('POST', '/test/fixtures/users', {
  id: subjectId,
  role: 'viewer',
  correlationId,
})

Then query by both subject and correlation ID. Clean up the fixture afterward or let an isolated ephemeral environment reset it.

Keep the worker visible in Docker

For local and CI reproducibility, run the React application and asynchronous worker as separate services instead of hiding both in one process:

services:
  app:
    build: .
    command: npm run start:test
    healthcheck:
      test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]

  audit-worker:
    build: .
    command: npm run audit-worker:test

  cypress:
    image: "${CYPRESS_IMAGE:?pin a full tag or digest}"
    environment:
      CYPRESS_baseUrl: http://app:3000
    depends_on:
      app:
        condition: service_healthy

Pin CYPRESS_IMAGE to an approved full Cypress image tag or digest. A moving browser or Node version makes before/after comparisons ambiguous. Also remember that ordinary depends_on controls start order, not readiness; use a health check or an explicit wait strategy before Cypress begins.

Observe facts, not implementation accidents

The inspection boundary should express the product promise without coupling the test to a worker's internal table layout.

Fragile observation	More durable observation
Query a queue's private row schema	Ask for the event by correlation ID and type
Assert an exact worker timestamp	Assert it falls inside the agreed window
Read an internal retry counter	Assert one externally visible consequence
Find the latest event globally	Match test-owned identifiers

This is still integration testing, so some coupling is intentional. The goal is to couple to the contract—“a role change produces this audit fact”—rather than to incidental storage details.

Keep inspection endpoints narrow

A /test/audit-events endpoint is useful only when it is intentionally designed:

unavailable in the public production surface;
protected by a scoped test identity;
returns only the fields required for verification;
filters by test-owned identifiers;
has retention and cleanup behavior;
never exposes unrelated customer data.

An alternative is a cy.task() that queries a test database or message sink from the Node process. The principle is the same: observe the consequence through an authorized, controlled boundary.

What this does not test

Finding an audit row does not prove that every eventual consumer processed it. Finding a queued email does not prove that an external provider delivered it. Choose the observation point that matches the promise you want to protect.

For a high-risk action, one test may need two side-effect boundaries: for example, the outbox event was committed and the downstream sandbox provider accepted it. Keep those assertions in a small representative suite; running every downstream integration for every UI scenario would be slow and expensive.

This pattern also does not replace service-level tests. Its purpose is to connect one representative browser action to the system state and side effect that give the action meaning.

The next time a test stops at Saved, ask which other witness could disagree.

Reference

Test REST and GraphQL Contracts Beside Your UI with cy.request()

Raju Dandigam — Fri, 09 Oct 2026 17:06:33 +0000

A broken API contract often surfaces as a vague UI failure:

expected [data-cy="order-total"] to contain "$42.00"

Was the React component wrong? Did the API rename total to amount? Did GraphQL return 200 with an errors array? A focused API test answers those questions closer to the boundary that changed.

Cypress can run REST and GraphQL checks through cy.request() in the same specs, configuration, and CI job as the UI suite.

A REST contract should assert shape and meaning

A status assertion alone is not much of a contract. Check the fields the consumer actually relies on and a few domain constraints:

describe('GET /api/cart', { testIsolation: false }, () => {
  it('returns the fields required by checkout', () => {
    cy.request('/api/cart').then(({ status, body }) => {
      expect(status).to.eq(200)
      expect(body).to.have.all.keys(
        'id',
        'items',
        'subtotal',
        'tax',
        'total',
        'currency',
      )
      expect(body.total).to.be.a('number').and.be.at.least(0)
      expect(body.currency).to.be.oneOf(['USD', 'EUR', 'GBP'])

      body.items.forEach((item: unknown) => {
        expect(item).to.include.all.keys('sku', 'quantity', 'unitPrice')
      })
    })
  })
})

This is intentionally consumer-oriented. It is not a full OpenAPI schema validator, but it catches the contract changes most likely to break the checkout UI.

For expected error responses, disable Cypress's default status-code failure and assert the API's error contract:

cy.request({
  method: 'POST',
  url: '/api/orders',
  body: { items: [] },
  failOnStatusCode: false,
}).then(({ status, body }) => {
  expect(status).to.eq(422)
  expect(body.errors).to.deep.include({
    field: 'items',
    code: 'MIN_ITEMS',
  })
})

GraphQL can fail with HTTP 200

GraphQL usually uses one endpoint. Transport success does not guarantee query success, so assert both data and the absence of errors.

const gql = (query: string, variables: Record<string, unknown>) =>
  cy.request('POST', '/graphql', { query, variables })

it('returns the order fields used by the confirmation page', () => {
  gql(
    `query Order($id: ID!) {
      order(id: $id) {
        id
        status
        total
        currency
      }
    }`,
    { id: 'order-17' },
  ).then(({ status, body }) => {
    expect(status).to.eq(200)
    expect(body).not.to.have.property('errors')
    expect(body.data.order).to.deep.include({
      id: 'order-17',
      status: 'CONFIRMED',
      currency: 'USD',
    })
    expect(body.data.order.total).to.be.a('number')
  })
})

An alternative is to assert errors is either absent or empty, depending on your server's response shape. Be explicit; do not assume 200 means the resolver succeeded.

Control the data before asserting the contract

A contract check built on whatever data happens to exist is difficult to diagnose. Seed a synthetic resource with a stable identifier, verify it, and remove it through a supported test boundary:

beforeEach(() => {
  cy.request('POST', '/test/fixtures/orders', {
    id: 'contract-order-17',
    status: 'CONFIRMED',
    total: 42,
    currency: 'USD',
  })
})

afterEach(() => {
  cy.request('DELETE', '/test/fixtures/orders/contract-order-17')
})

In a shared environment, generate a per-run identifier rather than reusing the literal value. Test-only fixture endpoints should be authenticated, narrowly scoped, unavailable from the public production surface, and prevented from returning unrelated records.

`cy.request()` and `cy.intercept()` solve different problems

Question	Use
Can I call this endpoint directly and inspect the response?	`cy.request()`
Did the application make this request?	`cy.intercept()`
Should I stub or wait for browser traffic?	`cy.intercept()`
Do I need a database query or filesystem operation?	`cy.task()`

cy.request() is handed to the Cypress Node process. The browser does not make the call, so it is not visible in DevTools, is not intercepted by cy.intercept(), and is not constrained by browser CORS.

That makes direct API tests convenient. It also means they cannot prove that browser code uses the endpoint correctly.

Keep API specs substantial

Cypress starts a browser for an end-to-end spec even when that spec only uses cy.request(). Avoid creating one spec per endpoint method. Group related API tests by resource so the startup cost is shared:

cypress/e2e/api/
├── auth.cy.ts
├── carts.cy.ts
├── orders.cy.ts
└── users.cy.ts

For API-only groups, testIsolation: false can avoid browser cleanup between tests, but then your own setup must ensure that one test cannot contaminate another.

Prefer a few meaningful assertions over a response snapshot

Large snapshots tend to fail on harmless fields such as timestamps while hiding why a consumer broke. Assert the required fields, enum values, nullability, and domain relationships instead. If the API has a formal schema, validate that schema in its own pipeline and keep the Cypress assertion focused on the React workflow's dependency.

That division gives two useful signals: the service owns its complete contract, while the UI repository documents the subset it actually consumes.

What these tests do not replace

These examples do not replace:

provider/consumer contract tooling across independently deployed services;
OpenAPI or GraphQL schema validation;
load, security, or resilience testing;
browser integration tests;
service-level unit and integration tests.

They provide fast evidence in the repository where frontend engineers feel the breakage. That is often enough to turn “the page is wrong” into “the response contract changed.”

Which API dependency causes the most ambiguous UI failures in your suite today?

Reference

Cypress API Testing Guide

Stop Logging In Through the UI: cy.request() + cy.session() in Cypress 16

Raju Dandigam — Wed, 07 Oct 2026 20:05:06 +0000

A UI login test should log in through the UI. Most other tests should not.

If 80 tests begin by typing the same username and password, they are repeatedly testing authentication while adding redirects, animations, third-party identity pages, and network timing to unrelated scenarios.

Cypress can establish the session over HTTP with cy.request(), store it with cy.session(), and continue in the real browser with the same cookies.

I am a Cypress Ambassador. This article uses a synthetic application and reflects my own technical evaluation.

The smallest useful command

In Cypress 16, sensitive environment values should be read asynchronously with cy.env(). The old synchronous Cypress.env() API was removed.

// cypress/support/commands.ts
Cypress.Commands.add('loginByApi', () => {
  cy.env(['supportUser', 'supportPassword'], { log: false }).then(
    ({ supportUser, supportPassword }) => {
      cy.session(
        ['support-user', supportUser],
        () => {
          cy.request({
            method: 'POST',
            url: '/api/auth/login',
            body: {
              username: supportUser,
              password: supportPassword,
            },
            log: false,
          })
            .its('status')
            .should('eq', 200)
        },
        {
          cacheAcrossSpecs: true,
          validate() {
            cy.request('/api/auth/me')
              .its('body.role')
              .should('eq', 'support')
          },
        },
      )
    },
  )
})

Then the spec spends its browser time on the feature being tested:

describe('support dashboard', () => {
  beforeEach(() => {
    cy.loginByApi()
    cy.visit('/support')
  })

  it('filters cases assigned to me', () => {
    cy.findByRole('button', { name: 'Assigned to me' }).click()
    cy.findAllByTestId('case-row').should('have.length.greaterThan', 0)
    cy.findAllByTestId('case-owner').each(($owner) => {
      expect($owner).to.contain.text('Me')
    })
  })
})

Why the browser is already authenticated

cy.request() does not maintain a separate HTTP-client cookie jar. Before sending a request, Cypress attaches matching cookies from the browser. When a response contains Set-Cookie, Cypress writes those cookies into the browser jar.

That shared boundary makes this flow possible:

POST /api/auth/login
        ↓ Set-Cookie
browser cookie jar
        ↓
cy.visit('/support')

cy.session() snapshots the resulting cookies, localStorage, and sessionStorage. On later tests, Cypress can restore the snapshot instead of running the setup again.

The session identifier matters. ['support-user', supportUser] prevents Cypress from confusing this session with a different user or login strategy.

Choose the cache boundary deliberately

cacheAcrossSpecs: true can remove repeated authentication work across spec files in the same run. It is appropriate only when the cached identity is safe to reuse. If a test changes that user's role, password, tenant, or server-side session, either give the scenario a different session key or avoid cross-spec caching.

The key should describe every input that can change the authenticated state:

cy.session(
  ['support-user', tenantId, locale, authStrategy],
  establishSession,
  { validate: validateSession, cacheAcrossSpecs: true },
)

Do not include a password in the identifier. Session identifiers can appear in diagnostic output; use non-secret dimensions and let the setup callback read credentials privately.

cacheAcrossSpecs is not a distributed cache. Parallel CI machines or Docker containers do not share the saved browser session. Each runner must be able to establish and validate its own session, and the test account must tolerate that concurrency. If parallel containers mutate one shared user, isolation problems can look like authentication flake.

Always validate the restored session

A cached session can become invalid because the server expired it, the test environment reset, or the authorization changed. Without validate(), the next failure may appear far from authentication.

The validation request should check something meaningful. A generic 200 from /health proves nothing about the user. /api/auth/me can verify both authentication and the expected role.

If validation fails, Cypress reruns the session setup.

Secrets are safer, not magically safe

cy.env() keeps configured values in the Node process until the requested keys are yielded. Cypress logs key names, not values. Once your callback receives the object, however, those strings are ordinary JavaScript values.

Avoid this:

cy.env(['supportPassword']).its('supportPassword').should('not.be.empty')

A failed assertion can print the value. Use { log: false }, avoid assertions on secrets, do not interpolate them into error messages, and scope test credentials narrowly.

Public, non-sensitive settings such as an environment label can use Cypress.expose() and synchronous access. Credentials should not.

What API login does not test

This pattern intentionally skips:

form validation;
password-manager behavior;
identity-provider redirects;
browser CORS behavior for the login call;
visual and accessibility behavior of the login page.

Keep a small dedicated authentication suite for those responsibilities. Use API login for tests whose subject is something else.

This also makes triage clearer:

Failure	Likely boundary
Login API rejects credentials	Fixture, secret, or authentication service
Session validation fails after restore	Expired or mutated server-side session
`/support` redirects after validation passed	Browser routing or authorization integration
Filter assertion fails	The support-dashboard behavior under test

Also remember that cy.request() originates in the Cypress Node process. It does not appear in the browser Network panel, and cy.intercept() cannot spy on or stub it. Use cy.intercept() for requests made by your application.

The decision rule

Use the UI when login is the behavior under test. Otherwise, establish the prerequisite through the narrowest supported API, validate the identity, and spend the browser run on the feature you actually want to evaluate.

That is faster, but more importantly, it makes failures easier to classify.

Would a failed login in your current suite identify an authentication regression—or merely prevent fifty unrelated tests from starting?

DEV Community: Raju Dandigam

Debug a Flaky CI Test from Structured Test Replay JSON

Find a flaky test

Pull both timelines

Narrow to the failure window

Inspect recorded DOM state

Give an AI agent a bounded question

Verify locally

Security boundaries

References

Test a Streaming React AI Chat Without Calling an LLM

Define an application-level event contract

A small controlled stream

Test progressive rendering

Test late events after cancellation

Test the failure state without deleting useful evidence

Do not overuse cy.clock()

Keep a separate provider contract

References

Add On-Device Summarization to React with Chrome's Summarizer API

Choose a reading task with a clear boundary

Treat availability as part of the interface

Own the summarizer for one request

Bind the result to the brief that produced it

Show useful states and preserve the source

Test lifecycle behavior and summary quality separately

A Read-Only WebMCP Tool Erased My React Form State

A read must preserve unsaved work

Trace the read through Chrome

Reproduce the failure without timing guesses

Give refreshes and saves different merge rules

Reject older snapshots before merging

Check the repair against specific sequences

Keep cancellation and recovery in the test plan

Finish with native and model driven checks

Cypress 16.1.1 Upgrade Checklist: Measure Speed, Flake, Memory, and Secret Boundaries

Before upgrading: capture a baseline

Pin the runner image before comparing

1. Confirm runtime and framework compatibility

2. Review native browser networking

3. Find typing-delay assumptions

4. Move retryable storage assertions to .should()

5. Review visibility assumptions

6. Remove obsolete memory configuration

7. Migrate secret access

8. Include the 16.1 reliability and patch fixes

Report the upgrade honestly

References

What Should a Self-Healing Cypress Test Be Allowed to Change?

Separate three kinds of change

Define the allowed envelope

Protect intent outside the selector

Reject suspicious repair patterns

Run a negative control

Know the Cypress workflow you are using

The approval rule

References

Expose a React Workflow to AI Agents with WebMCP

What WebMCP adds to an existing API

Choose tools around a complete task

Describe the allowed change precisely

Register tools with the editor lifecycle

Route the handler through application code

Exercise the tools before involving a model

Let an agent choose the sequence

Build a Reviewable Gemini Form Assistant in React

Keep generation separate from saving

Define what a proposal can contain

Give Gemini the context needed to interpret the brief

Bind the proposal to the state that produced it

Make review useful and apply through the existing API

Keep failures understandable

Evaluate the proposal and the handoff separately

A Replay-Safe Cypress Cloud Webhook Receiver in TypeScript

1. Verify the exact raw body

2. Deduplicate on the event ID

3. Return quickly

Keep container state disposable

4. Read the final status

Test the failure cases

Do not overuse `cy.clock()`

4. Move retryable storage assertions to `.should()`

`cy.request()` and `cy.intercept()` solve different problems