DEV Community

Keith Azodeh
Keith Azodeh

Posted on Originally published at Medium Fully Autonomous

Cloudflare’s Local JSON Action Returns Placeholders. What Did Your Test Prove?

A supplier page says $19 per case. Your test passes, the JSON parses, and the application shows a price. Did the extractor read the page? With Cloudflare’s newly supported local Browser Run Quick Actions, that depends on which action you called: local json returns placeholders.

AI disclosure: This article was generated with AI assistance from public sources. The proposed tests below are illustrative; they are not reported test results from Keith’s projects or Cloudflare.

On November 13, 2026, Cloudflare announced local Quick Actions support. Calling .quickAction() under npx wrangler dev now uses Chrome on your machine. The announcement explicitly identifies the JSON action as emulated because its AI model is unavailable locally.

That is a useful development boundary. It becomes a release problem when a passing test is described as evidence about a component that never ran.

A valid response can answer the wrong question

Cloudflare’s local testing documentation explains the placeholder behavior: schema fields with const, enum, or default receive the specified value, or the first enum entry. Other fields receive empty values such as an empty string, zero, false, or an empty array. Wrangler logs a warning.

Suppose an illustrative pricing schema gives currency a default of USD and constrains unit to an enum beginning with “case.” A local response containing those values does not establish that the page quoted dollars or cases. The schema supplied those answers.

A test can still prove something worthwhile: the caller handles the response, validation runs, and the interface renders the resulting state. It cannot establish that an extraction model identified the correct offer.

The assertion decides what “passed” means. Checking that a number exists answers a different question from checking that $19 came from the current supplier offer.

Why the distinction matters after extraction

The public TMC Price Intelligence case study describes a delivered 2023–2025 engagement that turned supplier messages and supporting files into structured pricing information. The workflow extracted product, size, price, region, and supplier, then connected those records to reviewable reports, quotes, and invoice workflows. An early prototype used DeepSeek R1 for interpretation.

That history does not establish that TMC used Browser Run. It supplies a concrete reason to care about the distinction: extracted fields travel into operational decisions.

For a similar workflow, a schema-shaped zero is not automatically a free offer. An empty array is not automatically evidence that a supplier has nothing available. A default unit is not evidence that two offers are comparable. The receiving application needs deliberate behavior for each of those states.

Local emulation is a good place to exercise that behavior. Treating its outputs as measured extraction accuracy would skip the very stage that interprets the source.

Give the suites different jobs

I would label an offline suite “response handling” and keep its assertions narrow. Feed the application explicit fixtures for a usable offer, an incomplete record, and a failed request. Confirm that incomplete records stay out of quote-ready results and that the review screen makes the missing information visible.

Those fixtures should be under the test author’s control. A vendor’s placeholder generator is useful for checking an integration, but its default values should not quietly become the entire catalog of business edge cases.

Then create a separate extraction suite whose expected answers come from the source material. Use synthetic supplier pages with known offers, keeping those pages separate from the application’s expected-output fixtures. Run the real extraction path against them.

One small test would be particularly revealing: publish two fixture pages that differ only in price, $12 versus $19. Expect the extracted price to follow that change. Keep the schema identical. This tests whether the answer responds to source evidence rather than merely satisfying the same output shape.

Add a page with two offers and an explicit instruction about which one to extract. Inspect the selected offer against the page. A successful parse cannot tell you whether the application chose the intended supplier or accidentally combined fields from different rows.

These are proposed acceptance checks, not a claim that this test set is sufficient for every pricing system.

Make remote execution a visible choice

Cloudflare documents setting "remote": true on the browser binding to obtain real JSON results during local development. That makes “running the Worker locally” an incomplete description of the test environment: the browser operation can be remote.

Record that choice alongside the fixture revision and expected answers. Keep any separately configured model or prompt settings with the result, where applicable. A teammate reading the report should be able to tell whether it exercised response handling, a real extraction request, or both.

The tradeoff is practical. Offline tests can run frequently with controlled responses. Real extraction tests introduce service dependencies and usage, so a team may run a smaller source-grounded set at a release boundary. Neither suite has to pretend to cover the other’s job.

For teams building pricing, procurement, or document-ingestion software, this changes how a green check should be read. Ask which component produced the answer before using that check to authorize the next operational step.

Cloudflare has documented the local boundary clearly. Put the same boundary in the test report.

Top comments (0)