DEV Community: xiaoxu

Testing Binary Image Responses Beyond HTTP 200

xiaoxu — Sun, 09 Aug 2026 14:22:23 +0000

Testing Binary Image Responses Beyond HTTP 200

Why this matters

HTTP 200 proves that a handler returned successfully. It does not prove that a downloaded image is decodable, uses the requested format, has the declared dimensions, carries the right filename, or can be represented correctly by the client.

Binary transformation APIs often split their result across two channels: bytes in the body and metadata in headers. If those channels drift, the image can still open while the UI reports the wrong size or filename.

I tested a production Next.js image route as one body-plus-headers contract instead of treating status as the outcome.

What I built or tested

The route accepts multipart input, validates the file, resolves output format and quality, and passes bytes to a Sharp-based processor. On success it returns the encoded body plus:

Content-Type and Content-Disposition;
Cache-Control: no-store;
original and processed byte counts;
original and output formats;
a sanitized output filename;
decoded output width and height.

The browser adapter consumes those headers to construct its result object. They are not decorative diagnostics; they drive the download and result UI.

Setup

The private experiment generated a synthetic 40-by-24 PNG in memory, wrapped it in a File, and sent it to the real route handler through NextRequest:

const form = new FormData();
form.append("file", new File([input], "fixture.png", {
  type: "image/png",
}));
form.append("outputFormat", "webp");
form.append("quality", "80");

const response = await POST(new NextRequest(url, {
  method: "POST",
  body: form,
}));

Direct handler invocation avoids starting a server while still exercising multipart parsing, validation, format resolution, processing, and response construction.

Step-by-step walkthrough

Validate declarations against independently decoded bytes:

The test accepts success only when encoded bytes and declared metadata agree.

Decode the body

Do not infer format from Content-Type. Pass the returned bytes to an independent decoder:

const bytes = Buffer.from(await response.arrayBuffer());
const metadata = await sharp(bytes).metadata();

expect(metadata).toMatchObject({
  format: "webp",
  width: 40,
  height: 24,
});

This catches a handler that sends PNG bytes with a WebP header or reports stale dimensions after resizing.

Reconcile byte counts

The original-size header should equal the uploaded file size. The processed-size header should equal the response body's actual byte length:

expect(response.headers.get("x-original-size"))
  .toBe(String(input.byteLength));
expect(response.headers.get("x-processed-size"))
  .toBe(String(bytes.byteLength));

Avoid asserting that processed bytes must be smaller. Conversion and compression settings can legitimately increase a tiny or already optimized image.

Verify filename and policy

The response should declare both an attachment filename and the custom filename consumed by the client. Those values should agree with the requested output format.

Cache-Control: no-store is a separate contract. Correct image bytes do not prove that an intermediary or browser receives the intended retention instruction.

Exercise a non-binary error

A missing-file request returns status 400 with a JSON error. Test content type and message before attempting binary decoding. Clients need distinct parsing paths for structured errors and successful Blobs.

What went wrong

A status-only assertion would have passed even if every custom header were empty. The browser adapter would then use local fallbacks for filename, size, format, and dimensions, potentially hiding a server regression.

The reverse is also possible: headers can look correct while the body is corrupt or encoded in another format. Neither channel can validate the other unless the test decodes and reconciles them.

The experiment verified direct handler behavior, not XMLHttpRequest upload progress or a deployed proxy. An intermediary could alter Content-Disposition, expose fewer headers cross-origin, or change caching behavior. Keep one transport-level test for the deployed shape.

Fix or mitigation

Use a contract matrix:

Concern	Assertion source
status	response
encoded format	independent body decoder
dimensions	decoder compared with headers
original bytes	uploaded fixture compared with header
processed bytes	response buffer compared with header
filename	attachment and custom filename headers
retention	cache-control header
failure	non-2xx content type and structured payload

Keep fixture dimensions small and deterministic. Assert relationships rather than hardcoded compression ratios. For resize routes, calculate expected dimensions from a known input and requested options.

On the client, retain sensible fallbacks for resilience, but test the server contract strictly so fallbacks do not mask regressions in CI.

Trade-offs

Direct route tests are fast and focused, but they bypass a real HTTP server, reverse proxy, CORS configuration, and browser download behavior. Pair them with a smaller end-to-end suite.

Custom headers make the client simple, yet every added field becomes versioned API surface. Prefer a minimal set that the UI actually consumes.

Independent image decoding adds native-library cost to tests. It is worthwhile at the binary boundary because string or snapshot assertions cannot validate encoded artifacts.

Exact encoded byte lengths can vary across library versions. Compare each header with the body generated in the same run rather than pinning an absolute size.

How I verified it

The production route returned a 124-byte WebP from a 150-byte generated PNG. Independent decoding reported WebP, 40-by-24 pixels.

Every metadata field matched the same artifact: content type image/webp, filename fixture.webp, original format PNG, output format WebP, width 40, height 24, and byte-count headers equal to their buffers. The cache policy was no-store.

The negative request returned 400, JSON content type, and No image file received.

Existing browser tests provide the outer evidence: they save actual download URLs and decode output format and dimensions. The direct experiment adds exact route-response reconciliation without claiming deployed proxy behavior.

Conclusion

A binary API response is not just its body, and success is not just its status.

Decode the returned artifact, compare it with every client-visible metadata field, verify filename and cache policy, and test a structured error branch. Keep browser coverage for transport-specific behavior.

That gives one answer to the only question users care about: does the downloaded file—and everything the UI says about it—describe the same real result?

Preventing Silent File Loss from Duplicate ZIP Entry Names

xiaoxu — Thu, 06 Aug 2026 04:55:30 +0000

Preventing Silent File Loss from Duplicate ZIP Entry Names

Why this matters

A batch converter can report that two files finished, generate a valid ZIP, and still deliver only one file.

The failure appears when distinct inputs converge on the same output name. Convert photo.jpg and photo.png to WebP and both become photo.webp. If the archive builder uses that name as a key twice, the second payload replaces the first. There is no malformed archive and no required exception—just silent data loss at the final download boundary.

I reproduced that behavior with the same filename rule and JSZip insertion pattern used by a browser image converter, then tested a deterministic naming strategy that preserves both payloads.

What I built or tested

The converter constructs an output filename by removing the source extension, sanitizing the basename, and appending the selected output extension:

function buildFileName(originalName: string, outputFormat: OutputFormat) {
  const base = originalName
    .replace(/\.[^.]+$/, "")
    .replace(/[^a-zA-Z0-9-_]+/g, "-") || "image";
  const extension = outputFormat === "jpeg" ? "jpg" : outputFormat;
  return `${base}.${extension}`;
}

That is reasonable for individual downloads. It is not sufficient as a unique archive key.

The batch path fetches each result Blob and calls archive.file(result.filename, blob). The same pattern appears in both the general image workspace and the dedicated batch converter. Neither path allocates a collision-safe entry name before insertion.

Setup

The minimal failure needs only two synthetic source names and two distinguishable payloads:

const inputs = ["photo.jpg", "photo.png"];
const outputNames = inputs.map((name) => buildFileName(name, "webp"));

// ["photo.webp", "photo.webp"]

I used the source repository's installed JSZip version and text payloads named from-jpg and from-png. Images are unnecessary for this experiment because the bug is in archive identity, after processing has already produced valid Blobs.

Step-by-step walkthrough

The important distinction is between a display filename and a unique archive entry name:

Name allocation happens before insertion so every successful result owns one archive entry.

Reproduce the collision

Adding the same path twice is syntactically valid:

const archive = new JSZip();
archive.file("photo.webp", "from-jpg");
archive.file("photo.webp", "from-png");

After generating and reopening the ZIP, the experiment found one entry named photo.webp. Its content was from-png, proving that the later write replaced the earlier payload.

This is why checking only that ZIP generation succeeds is weak. The archive is readable and its surviving file is valid.

Allocate a unique name

Track normalized entry names and add a suffix before the extension:

function allocateUniqueName(requested: string, used: Set<string>) {
  const dot = requested.lastIndexOf(".");
  const base = dot > 0 ? requested.slice(0, dot) : requested;
  const extension = dot > 0 ? requested.slice(dot) : "";

  for (let suffix = 1; ; suffix += 1) {
    const candidate = suffix === 1
      ? requested
      : `${base} (${suffix})${extension}`;
    const key = candidate.toLowerCase();

    if (!used.has(key)) {
      used.add(key);
      return candidate;
    }
  }
}

With that allocator, the inputs become photo.webp and photo (2).webp. Reopening the generated ZIP showed two entries, with each original payload attached to the expected name.

Case normalization matters because archive consumers can differ in case sensitivity. Treating Photo.webp and photo.webp as a collision avoids creating an archive that behaves differently after extraction on another filesystem.

What went wrong

The existing integration coverage tests a mixed compression batch with one JPEG and one PNG. Compression preserves their formats, so the resulting names still have different extensions. The test correctly expects two entries, but it does not exercise name convergence.

A separate batch conversion test uses mixed inputs and confirms that two conversions complete, yet it does not download and inspect the ZIP. Even an entry-count assertion would need collision-producing fixtures: two arbitrary input files can accidentally keep unique basenames.

The mistaken assumption is that a successful processing result already has a filename safe for every container. A filename can be individually valid while remaining non-unique inside a batch.

Fix or mitigation

Make archive naming an explicit stage with these properties:

Allocate names in stable result order.
Compare names using a documented normalization rule.
Put the suffix before the extension.
Reserve each name before asynchronous Blob fetching or ZIP insertion.
Keep the original source-to-entry mapping for tests and diagnostics.

Reserve names synchronously before Promise.all. Otherwise, concurrent tasks can observe the same unused name before either records it.

Then strengthen the integration test with photo.jpg and photo.png, convert both to the same format, download the archive, reopen it, and assert:

the archive has two files;
the names are deterministic and unique;
each entry decodes to the expected image dimensions or known payload;
the UI's successful-result count equals the archive entry count.

Trade-offs

Numeric suffixes are predictable and preserve familiar filenames, but they do not reveal the original format. A policy such as photo-from-jpg.webp is more descriptive but can expose source-format details and produce longer names.

Stable ordering matters. Parallel processing completion order can vary, so assigning the unsuffixed name to whichever task finishes first would make archives nondeterministic. Allocate from the selected-file or result order instead.

Unicode normalization and platform-specific reserved names add complexity if arbitrary international filenames must round-trip exactly. A conservative sanitizer plus case-insensitive uniqueness is simpler, but document that it changes names.

Finally, a correct entry count does not prove correct association. Tests should inspect content or decoded metadata, not only keys.

How I verified it

The private experiment ran the production-equivalent basename rule against photo.jpg and photo.png, producing photo.webp twice. JSZip generated a readable archive containing one entry; reopening that entry returned only the second payload.

The mitigation reserved collision-safe names before insertion. The generated archive contained exactly photo.webp and photo (2).webp, and both payload assertions passed.

Source inspection confirmed that both browser batch workspaces currently pass result.filename directly to JSZip. The existing end-to-end ZIP test proves the ordinary two-entry path works, while the controlled experiment isolates the missing collision case.

Conclusion

Batch success is not complete until every successful result has a distinct place in the downloaded archive.

Treat output filenames as requests, not unique identifiers. Allocate deterministic archive names before concurrent work, generate the ZIP, reopen it in a test, and verify both entry identity and content association.

That turns a valid-looking archive with silent replacement into a boundary the test suite can actually defend.

Verifying Image Metadata Removal by Decoding the Download

xiaoxu — Tue, 04 Aug 2026 09:19:54 +0000

Verifying Image Metadata Removal by Decoding the Download

Why this matters

A button labeled "Remove metadata" is not evidence that metadata was removed.

Neither is a successful HTTP response, a new filename, or a smaller file. The output must be decoded and inspected. Otherwise a refactor can accidentally preserve EXIF while every visible UI assertion remains green.

I tested an image cleanup flow with a synthetic JPEG that definitely contains EXIF data. The production path re-encoded it, and an independent decoder inspected the returned bytes. A control path that explicitly preserved metadata made the failure condition observable.

The reusable idea is simple: prove the precondition, run the real path, and assert the postcondition on the downloadable artifact.

What I built or tested

The fixture generator creates a small image and embeds synthetic metadata:

await source
  .jpeg({ quality: 82 })
  .withMetadata({
    exif: {
      IFD0: { Copyright: "Synthetic fixture" },
      IFD3: { GPSLatitudeRef: "N" },
    },
  })
  .toBuffer();

The strings are intentionally artificial. Tests should never need a real user's photograph or location to exercise the privacy boundary.

There are two production implementations. The common browser path decodes the source and draws it onto Canvas before exporting a new Blob. The server compatibility path decodes, rotates, and encodes through Sharp without requesting metadata preservation.

Both are re-encoding paths, but they still need artifact-level tests.

Setup

Separate the test into four checkpoints:

Decode the input and prove EXIF exists.
Run the product operation.
Decode the exact output offered for download.
Assert metadata absence plus basic image validity.

The last step should include format and dimensions. An empty or corrupt Blob has no EXIF too, but it is not a successful cleanup result.

Use an independent inspection call rather than reading a flag returned by the processor. If the processor reports metadataRemoved: true, that value only describes intent. Decoding the result describes the artifact.

Step-by-step walkthrough

The verification flow crosses several boundaries:

The assertion targets the bytes a user receives, not the UI action that produced them.

Prove the fixture is meaningful

Start by inspecting the fixture:

const before = await sharp(input).metadata();
expect(before.exif).toBeDefined();

Without that assertion, a broken fixture can make the test pass vacuously. "EXIF is absent after processing" means little if it was absent before processing too.

Execute the actual browser workflow

The integration test opens the metadata-removal route, uploads the EXIF-bearing JPEG, and waits for a browser-engine result. It then saves the actual download to the test output directory.

Canvas constructs a fresh raster surface, draws decoded pixels, and exports a new image. The code does not copy the source metadata container into the export.

Cover the server compatibility path

The server adapter creates a Sharp pipeline, reads dimensions, optionally resizes, and invokes the selected encoder. It does not call withMetadata before writing the buffer.

The private experiment passed the same EXIF-bearing input through this production function. This confirms that compatibility processing also satisfies the cleanup contract.

Decode the result again

The assertion does not trust the output filename or MIME header:

const after = await sharp(downloadedBytes).metadata();

expect(after.format).toBe("jpeg");
expect(after.width).toBe(64);
expect(after.height).toBe(48);
expect(after.exif).toBeUndefined();

Those checks establish that the result is still the expected image shape while the tested metadata block is absent.

What went wrong

"Re-encoding strips metadata" was too broad as an explanation. I added a control that decoded the same input, called Sharp's explicit metadata-preservation method, and encoded it again.

The control output still contained EXIF.

That failure case matters because it proves the fixture and decoder can detect retention. It also shows that a pipeline can re-encode pixels and still carry metadata when configured to do so.

Another limitation is scope. The assertion covers the EXIF field exposed by the decoder. It does not automatically prove that every possible metadata family, sidecar, container extension, or application-specific payload is absent. The public claim should match the fields actually inspected.

Fix or mitigation

Build the test around a positive control and a negative postcondition:

input: metadata must be present
preserving control: metadata must remain present
cleanup output: metadata must be absent
cleanup output: format and dimensions must remain valid

Run artifact checks for every processing engine that can serve the route. A browser-only test will not catch a server fallback that later starts preserving metadata. A server unit test will not catch a Canvas path that returns the original Blob by mistake.

Keep fixture metadata synthetic and recognizable. If a failure prints the value, it should never disclose real user information.

Finally, document the exact guarantee. "EXIF absent from the re-encoded output" is testable. "All private information removed" is a much larger claim that needs a broader threat model and additional scanners.

Trade-offs

Re-encoding changes more than metadata. Encoders can alter file size, compression artifacts, color profiles, orientation representation, and other characteristics. Removing metadata by re-encoding is not a byte-preserving operation.

Orientation deserves special attention. The server path autorotates before encoding, which can turn an orientation tag into transformed pixels. A test should verify displayed orientation if that matters to the product.

Canvas and Sharp may emit different bytes even with similar quality settings. The cleanup contract should focus on properties users need: decodability, dimensions, expected format, visual orientation, and absence of specified metadata.

Independent decoding adds test cost, but it is much stronger than asserting internal flags. Small synthetic fixtures keep that cost low.

How I verified it

The experiment created a 64-by-48 JPEG with EXIF copyright and GPS-related fields. Sharp metadata inspection confirmed the input contained an EXIF buffer.

A preservation control called withMetadata() and confirmed EXIF remained after encoding. The production server processor then encoded the same input to JPEG at quality 78. Decoding its output found no EXIF while retaining the 64-by-48 dimensions and JPEG format.

The repository's browser integration test provides the complementary path. It proves its input has EXIF, processes through the browser-first cleanup route, downloads the result, and asserts that an independent Sharp decode has no EXIF field.

Together, those checks cover both a real browser download and the server compatibility implementation without using private photos or network services.

Conclusion

Test privacy transformations at the artifact boundary.

Create data that definitely contains the property being removed, prove the test can detect a preserving failure, run every production path, and inspect the exact returned bytes with an independent decoder. Add validity assertions so corrupt output cannot pass by merely lacking metadata.

That turns "the cleanup button worked" into a narrow, reproducible statement: this valid downloaded image no longer contains the metadata field we inserted.

Routing Image Work Between Canvas and Sharp by Capability

xiaoxu — Mon, 03 Aug 2026 06:09:52 +0000

Routing Image Work Between Canvas and Sharp by Capability

Why this matters

"Browser first" sounds like one Boolean. A real image pipeline has at least four questions:

Can the browser decode this input format?
Can Canvas encode the requested output format?
Does the product accept the browser encoder for this operation?
What happens when an apparently supported operation fails at runtime?

If those decisions leak into UI components, batch loops, and API callers, the two processing paths drift. The interface may say an image stayed local while it actually uploaded, or a supported browser silently returns a different output format.

I tested a small capability router that keeps those questions in one place. Common work goes through Canvas. Sharp remains the compatibility path. The result records which engine actually produced each file.

What I built or tested

The browser adapter starts with two explicit sets:

const browserInputs = new Set([
  "image/jpeg",
  "image/png",
  "image/webp",
]);

const browserOutputs = new Set([
  "jpeg",
  "png",
  "webp",
]);

That excludes AVIF, HEIC, and HEIF inputs from the browser path. It also excludes AVIF output. These are capability decisions for this implementation, not claims about every browser.

The router then adds runtime and product constraints. It returns false during server-side rendering, probes WebP Canvas export, and keeps selected PNG compression operations on Sharp even though Canvas can emit PNG.

Setup

Define the routing input independently of either processor:

type ProcessingRequest = {
  file: File;
  outputFormat: "jpeg" | "png" | "webp" | "avif";
  toolSlug?: string;
};

The predicate needs no pixels. It only decides whether the browser path is worth attempting.

JPEG and PNG output are treated as supported once the runtime is a browser. WebP is probed with a one-pixel Canvas because some implementations can accept an output MIME argument yet return a different data URL. The probe verifies that the returned URL begins with the requested MIME type, then caches the Boolean.

This is capability detection rather than user-agent detection. The code asks the current runtime what it can produce instead of maintaining a browser-version table.

Step-by-step walkthrough

The decision order is intentionally cheap:

The predicate chooses an attempt; successful processing determines the recorded engine.

Reject non-browser runtimes

The same module can be imported during server rendering. Checking typeof window first prevents DOM capability code from running there. The server path is the safe default.

Check the input before the output

Canvas output support does not imply that the browser adapter can decode every accepted upload. An AVIF input requesting PNG therefore goes to Sharp even though PNG output is available.

Preserve product exceptions

The router keeps PNG compression on the server for three tool routes: the dedicated PNG compressor, the universal compressor producing PNG, and batch compression producing PNG.

This is the subtle part. A generic canCanvasEncode("png") check would say yes. The product has chosen a different implementation for compression behavior, so the exception belongs beside capability routing and needs a test of its own.

Probe uncertain exports once

For WebP, the adapter creates a tiny Canvas and calls toDataURL("image/webp"). It accepts the path only when the returned data URL declares WebP. The result is cached by output format, avoiding a probe for every file.

Fall back after a real browser failure

Passing the predicate does not guarantee that decoding, allocation, drawing, or encoding will succeed. The client orchestration therefore wraps the complete browser operation in try/catch. On failure it logs a warning and calls the server adapter with the original request.

This keeps the predicate simple. It does not try to predict available memory or every decoder edge case.

What went wrong

The first tempting model was "JPEG, PNG, or WebP means browser." The experiment disproved that shortcut in several ways.

A PNG input on the dedicated PNG compression route went to the server even though both its input and output appeared in the browser sets. An AVIF input requesting PNG also went to the server, but for a different reason: unsupported input decoding. JPEG requesting AVIF failed on output capability. Removing the browser runtime failed before any format test.

The cases look identical from the final Boolean, but they represent different constraints. That is why the routing predicate should remain readable rather than collapse into one opaque capability flag.

There is another limitation: the runtime fallback catches every browser-processing exception. A decode failure and a transient Canvas allocation failure both attempt the server. That is useful for compatibility, but it can add an upload after local work has already begun. The UI and telemetry must report the engine that completed, not merely the engine attempted first.

Fix or mitigation

Return an engine on the result:

type ProcessedResult = {
  processor: "browser" | "server";
  blob: Blob;
  width: number | null;
  height: number | null;
};

For a batch, reduce the recorded engines into browser, server, or mixed. A mixed label is more honest than assigning the whole batch to whichever path processed the first item.

Keep the server API behavior compatible with the browser request. It should accept the same quality, output-format, tool, and resize intent. Otherwise fallback changes semantics instead of only changing execution location.

Finally, make product exceptions named functions. A helper such as shouldKeepPngOnServer explains intent in review and gives future changes one obvious location.

Trade-offs

Browser processing avoids an upload for eligible files, but it consumes client memory and CPU. Large inputs can still fail after passing the format predicate. The server fallback improves completion rate at the cost of network transfer and server resources.

Canvas and Sharp also do not promise byte-identical output. The router aligns product intent and engine selection; it does not make two encoders equivalent. Verify format, dimensions, alpha handling, and other required properties rather than comparing hashes.

The WebP probe detects output support in the current runtime, but it does not prove acceptable quality for every image. A successful one-pixel export is a capability signal, not a visual benchmark.

Catch-all fallback can hide browser regressions if warnings are ignored. Count fallback reasons in observability, and fail tests when a route expected to stay local begins uploading.

How I verified it

I imported the production router into a Node 22 experiment with minimal window, document, and Canvas probe stubs. The assertions observed:

JPEG to JPEG selected the browser;
JPEG to a successfully probed WebP selected the browser;
AVIF input selected the server;
AVIF output selected the server;
PNG compression selected the server; and
removing window selected the server.

The production batch summarizer also returned mixed for one browser and one server result.

Repository end-to-end tests provide the next evidence layer. They download and decode a browser-produced JPEG-to-PNG result, a server-produced AVIF-to-PNG result, and both outputs from a mixed batch. That verifies the engines beyond the pure routing predicate.

Conclusion

A browser-first pipeline is safest when "first" means "attempted under an explicit contract," not "always used for familiar extensions."

List supported inputs and outputs, probe uncertain runtime behavior, preserve named product exceptions, fall back around the complete local operation, and record the engine that actually completed. The resulting capability table is small, testable, and far easier to evolve than format checks scattered across the interface.

Keeping Before-and-After Image Examples Verifiable

xiaoxu — Mon, 03 Aug 2026 06:09:36 +0000

Keeping Before-and-After Image Examples Verifiable

Why this matters

Before-and-after images look like content, but they behave like compiled artifacts. A page can keep rendering while its caption reports an old byte count, an "after" file was generated with different settings, or one asset quietly changed dimensions.

I encountered that boundary in an image-tool project with six example pages. Each page showed two cases, and every case had a before and an after asset. That is 24 files whose bytes, dimensions, and descriptive metadata need to agree.

The useful fix was not another screenshot review. It was to make the examples executable: generate them from controlled inputs, describe their expected properties in code, and fail a verifier when disk and metadata diverge.

What I built or tested

The page data uses a typed record for each case. The important fields are not presentation-only:

type ExampleAsset = {
  alt: string;
  bytes: number;
  format: string;
  src: string;
};

type ExampleCase = {
  id: string;
  before: ExampleAsset;
  after: ExampleAsset;
  outputNote: string;
};

The byte count is deliberately exact. It connects the label shown to a reader with the file committed to the site. The source path connects the same record to the asset the browser requests. The format and output note make the transformation understandable, while decoded dimensions are enforced by the verifier.

This contract is small enough to review. It is also strict enough to answer a practical question: does the page still describe the files that ship?

Setup

The project keeps source images outside the public example directory. An input-generation script prepares ten named inputs, copying PNG sources when no conversion is needed and using explicit JPEG or WebP settings for the others.

A second script finds every -before asset for six tool slugs and creates the matching -after file. The transformations are explicit:

JPEG uses quality 78, MozJPEG, and progressive output.
WebP uses quality 78 and smart subsampling.
PNG uses quality 78, compression level 9, and progressive output.
Every input is autorotated before encoding.

The generator then reports the observed input and output byte counts. Those observations are what should be reviewed and transferred into the typed records. Do not compute a reduction percentage once and leave it disconnected from the files.

The generator produces artifacts; the contract and verifier make them reviewable.

Step-by-step walkthrough

The verifier iterates through every before and after record. For each asset, it performs three checks:

The referenced file is accessible.
Its actual byte count equals the declared count.
Sharp can decode it and reports the expected 1536 by 1024 dimensions.

A compact implementation looks like this:

const file = await stat(filename);
const metadata = await sharp(filename).metadata();

if (file.size !== asset.bytes) {
  failures.push(`${asset.src}: byte count changed`);
}

if (metadata.width !== 1536 || metadata.height !== 1024) {
  failures.push(`${asset.src}: dimensions changed`);
}

Collecting failures is better than throwing on the first mismatch. A regeneration can affect several records, and one verifier run should give the author the complete repair list.

The command exits nonzero when any failure exists. That detail turns a useful local report into a CI gate.

What went wrong

A clean run proved that all 24 assets across the six pages matched their records. That only tested the success path, so I built an isolated failure experiment.

The experiment copied the public example directory to a temporary location and ran the real repository verifier against it. It then appended one byte to a copied JPEG and ran the verifier again.

Why append a byte? The image remained decodable, and its width and height stayed correct. A verifier that only opened the image or checked dimensions would accept it. The byte contract should reject it.

The second run exited with status 1 and reported that the file contained 145155 bytes instead of the expected 145154. The temporary fixture was deleted afterward, and the source repository was never modified.

How I verified it

This is a useful pattern for repository checks in general: create a valid baseline, introduce one controlled violation, and assert both the failure status and the diagnostic. A green check is more credible after its red path has been observed.

The experiment asserted the clean verifier's asset and page counts, then asserted both the nonzero failure status and the precise byte-mismatch diagnostic. This distinguished "the script ran" from "the script enforced the intended contract."

Fix or mitigation

The repeatable sequence is:

1. Change source images or transformation settings.
2. Regenerate before and after assets.
3. Review the generated byte summary and visual output.
4. Update the typed records in the same change.
5. Run the asset verifier.
6. Let CI reject any unreviewed drift.

Keep the generator and verifier separate. Generation is an intentional write; verification is a read-only check. If a CI check silently regenerated files, it could hide the fact that committed artifacts were stale.

Also pin the runtime and image library versions used for generation. Exact encoded bytes can change across encoder versions even when quality settings are unchanged. That is not a reason to weaken the check; it is a reason to make toolchain upgrades explicit and review their output.

Trade-offs

Exact bytes are not a perceptual-quality metric. Two outputs can be visually equivalent with different bytes, and identical dimensions say nothing about whether important texture survived compression. Visual review or perceptual comparisons are a separate gate.

The record also does not prove that a marketing claim is fair. It proves that the declared size and dimensions match the shipped sample under the documented settings.

Finally, strict byte checks are a poor fit when files are intentionally recompressed after the build by an uncontrolled pipeline. In that system, verify the pre-upload artifact and separately test delivery invariants such as dimensions, format, or a content digest exposed by the asset service.

Conclusion

Treat demo assets the way you treat generated code:

preserve their inputs,
make transformations explicit,
store reviewable expected metadata,
verify the committed outputs without rewriting them, and
exercise a real failure case.

That changes before-and-after examples from manually trusted decoration into a small, enforceable build contract. The page can still tell a visual story, but CI now checks that the story matches the files.

Updating Sitemap Dates Only for Routes That Actually Changed

xiaoxu — Thu, 30 Jul 2026 05:37:57 +0000

Updating Sitemap Dates Only for Routes That Actually Changed

Why this matters

A dynamic sitemap makes it tempting to write:

lastModified: new Date()

That is convenient and usually wrong. It tells crawlers that every page changed
whenever the sitemap was generated, even when the only event was a deployment.

The
Sitemaps protocol
defines lastmod as the linked page's modification date, not the sitemap
generation date. Google goes further in its current
sitemap guidance:
it says the value is used when it is consistently and verifiably accurate and
should reflect a significant page update, such as main content, structured
data, or links.

The difficult part is not formatting an ISO timestamp. It is answering:

Which public routes did this source-code change significantly affect?

I traced and tested a Git-aware workflow for a Next.js site that answers that
question before touching any dates. It combines semantic metadata comparison,
an explicit route dependency map, an AST-based stamper, and a CI check that
fails when an affected route still has its previous timestamp.

It does not claim an SEO ranking improvement. Its goal is narrower: make the
sitemap metadata truthful and reviewable.

What I built or tested

The site persists lastModified beside each route definition. Its Next.js
sitemap handler simply emits those values:

export default function sitemap(): MetadataRoute.Sitemap {
  return sitemapEntries.map((entry) => ({
    url: `${siteConfig.siteUrl}${entry.path}`,
    lastModified: entry.lastModified,
  }));
}

A separate Node.js command has three modes:

npm run seo:lastmod:list
npm run seo:lastmod:stamp
npm run seo:lastmod:check

list shows routes affected by the current diff.
stamp replaces only those routes' stored dates.
check fails when an affected existing route still has its previous date.

I ran the repository's integration test. It created a temporary Git repository,
committed a small route matrix, and exercised this sequence:

change one tool definition;
observe the tool route and homepage as affected;
observe the pre-stamp check fail;
stamp a fixed ISO date;
observe the check pass;
change one route-specific SEO record;
observe only that route;
change one static page;
observe only its matching route.

The test passed one of one. It created and removed its own temporary repository;
the evidence repository's existing worktree state was identical before and
after.

Setup

The implementation uses Node.js, Git, TypeScript's compiler API, and Next.js
route metadata. It models four route families:

Source definition	Route shape	Family
Tool definitions	`/${slug}`	`tool`
Converter navigation	configured `href`	`convert`
Guide definitions	`/guides/${slug}`	`guide`
Static sitemap entries	configured `path`	`static`

Each descriptor states:

the source file;
the variable containing the records;
the record's stable key;
a function that converts the record into a public route; and
the dependency family.

A reduced descriptor looks like this:

{
  file: "lib/tool-config.ts",
  variable: "toolDefinitions",
  keyProperty: "slug",
  routeFromValues: (values) => `/${values.slug}`,
  family: "tool",
}

The workflow also needs a comparison base. With tracked working-tree changes,
it compares against HEAD. In a clean checkout, it uses HEAD^ when
available. CI can pass an explicit base branch:

npm run seo:lastmod:check -- --base origin/main

That distinction keeps local pre-commit checks and clean CI checkouts useful
without silently choosing the same reference in every environment.

Step-by-step walkthrough

1. Parse route metadata without executing the application

The script reads designated TypeScript source files and parses them with the
TypeScript compiler API. It locates a named variable initializer and expects an
array literal for route definitions.

Each object literal is serialized into a semantic value. A reduced
property-assignment-only version is:

function serializeSemanticNode(node, sourceFile) {
  if (ts.isStringLiteral(node)) return node.text;

  if (ts.isObjectLiteralExpression(node)) {
    return Object.fromEntries(
      node.properties
        .filter((property) => {
          return getPropertyName(property.name, sourceFile) !== "lastModified";
        })
        .map((property) => [
          getPropertyName(property.name, sourceFile),
          serializeSemanticNode(property.initializer, sourceFile),
        ]),
    );
  }

  // Arrays, numbers, booleans, identifiers, and fallback source text...
}

The important line excludes lastModified from semantic comparison. Otherwise
stamping a timestamp would itself look like another route-content change,
creating a self-perpetuating diff.

This is more robust than a regular expression because it works with typed
object syntax, as const, satisfies, parentheses, and nested arrays or
objects. It is deliberately less general than executing the module. The
designated metadata variables must remain statically inspectable array or
object literals.

2. Compare route records against Git

The tool reads current sources from disk and base sources with:

git show <base>:<file>

It also collects changed paths from the tracked diff and untracked files. A
record is affected when it is new or when its semantic serialization differs
from the matching base record.

For a direct tool-definition change, the mapping adds:

the tool route; and
/, because the homepage consumes the tool list.

A converter-definition change follows the same rule. A guide record maps to
its guide route. A static route record maps to its configured path.

This is not based only on filenames. Two records can live in one configuration
file while only one public route changes. Comparing records preserves that
granularity.

3. Propagate shared dependencies explicitly

Route records are not the whole page. A page can change because a component it
uses changed.

The script therefore maintains dependency sets:

const familyDependencyFiles = {
  tool: new Set([
    "app/[tool]/page.tsx",
    "components/image-tool-workspace.tsx",
    "components/related-tool-link.tsx",
  ]),
  convert: new Set([
    "app/convert/[format]/page.tsx",
    "components/convert-workspace.tsx",
  ]),
  guide: new Set([
    "app/guides/[guide]/page.tsx",
    "components/guide-page-shell.tsx",
  ]),
};

When one of those files changes significantly, every route in that family is
affected. Site-wide components such as the header, footer, or structured-data
component can mark all routes.

Route-specific SEO content gets a narrower treatment. The tool parses the SEO
content object and compares records by slug. Changing one record affects only
its matching tool or converter. Changing code outside that object in the same
component can affect the full tool and converter families.

Static pages map directly from a route to their app/.../page.tsx file.

The resulting policy is:

Change	Affected routes
One tool record	Tool route and homepage
One converter record	Converter route and homepage
One guide record	Guide route
One route-specific SEO record	Matching tool or converter
Shared family component	Every route in that family
Header, footer, or shared structured data	All sitemap routes
One static page	Matching static route

4. Validate before writing

The command first rejects duplicate routes and any route with a missing or
invalid lastModified. New pages therefore cannot enter the sitemap without a
valid initial date.

For existing affected routes, check compares the current date with the base
date:

const staleRoutes = affectedRoutes.filter((route) => {
  const current = currentMetadata.routeRecords.get(route);
  const previous = baseMetadata.routeRecords.get(route);
  return previous && current.lastModified === previous.lastModified;
});

if (staleRoutes.length) {
  throw new Error(
    `${staleRoutes.length} affected route(s) still use the previous date`,
  );
}

This is the real enforcement boundary. Route detection alone is only a report;
a non-zero check makes stale metadata visible to local hooks or CI.

5. Stamp only selected AST ranges

For every affected route, the parser retains the AST node for the existing date
initializer. The stamper groups replacements by source file and applies them
from the end of the file toward the beginning:

for (const replacement of replacements.sort(
  (left, right) => right.start - left.start,
)) {
  source =
    source.slice(0, replacement.start) +
    JSON.stringify(date) +
    source.slice(replacement.end);
}

Reverse-order edits prevent an earlier replacement from shifting the offsets
of later nodes.

--dry-run computes and reports the same route set without writing. --route
adds an explicit route, while --all is reserved for a genuine site-wide
significant change.

The complete decision flow is:

Semantic record changes and explicit dependencies converge on one reviewable
route set.

What went wrong

The integration test intentionally started with a real failure.

It changed one tool description but left all sitemap dates untouched. The route
detector returned:

/
/compress-jpg

The checker then exited with status 1 because both affected routes still used
their previous dates. That failure is desirable: the tool page changed
directly, and the homepage changed indirectly because it consumes the tool
list.

After stamping both routes with a fixed timestamp, the same check passed. The
test then changed only the compress-jpg SEO record and correctly returned one
route rather than the whole tool family.

The experiment also exposed the central limitation: the dependency map is
policy, not omniscience. If a developer adds a significant shared component
outside the known sets, the script cannot infer that architectural edge. A
false negative is possible until the map is updated or the developer supplies
an explicit route.

There is a second boundary. The AST reader expects known variables to use
literal arrays or objects. Refactoring a route definition into a database call,
factory function, or generated import would break this inspection strategy
even if the runtime sitemap still worked.

Fix or mitigation

Treat the tool as an enforced workflow, not a magical detector.

Use list, stamp, check

For a normal page change:

npm run seo:lastmod:list
npm run seo:lastmod:stamp
npm run seo:lastmod:check

Review the list before stamping. The affected set is a content decision, not
just a code result.

For a significant dependency outside the map:

npm run seo:lastmod:stamp -- \
  --route /compress-jpg \
  --route /jpg-to-png

Use --all only when every indexable page truly received a significant
update.

Keep the dependency map beside architecture changes

When adding a shared route shell or moving content between components:

update the route dependency map;
add an isolated-Git test case for the new edge;
run list against the intended base;
inspect false positives and false negatives; and
stamp only after the mapping is correct.

Formatting, dependency upgrades, analytics wiring, CSS-only changes, and
copyright-year changes should not automatically move every timestamp.

Test the failure before the success

The integration test is reusable:

create a temporary Git repository;
write and commit minimal route metadata;
make one semantic content change;
assert the exact affected route list;
assert that check fails;
stamp a deterministic date;
assert the edited metadata; and
assert that check now passes.

Testing the pre-stamp failure proves the checker can reject stale metadata. A
test that runs only after stamping could pass even if enforcement were missing.

Trade-offs

Persisted dates create reviewable diffs, but developers must maintain them.
Record-level AST comparison is more precise than file-level matching, but it constrains how route metadata is authored.
An explicit dependency map is understandable and testable, but incomplete until humans add every meaningful shared edge.
Family-wide propagation avoids false negatives for shared components, but can update more routes than a deeper component analysis would.
Git makes local and CI comparisons reproducible, but shallow CI clones need enough history for the chosen base.
A route date is evidence of a significant content change, not a promise that a crawler will revisit the page or that rankings will change.

How I verified it

I used five evidence layers:

Protocol check: confirmed that lastmod represents the linked page's modification date, not sitemap generation time.
Current Google guidance: confirmed the value should be consistently accurate and tied to significant page updates.
Source trace: followed descriptors, AST serialization, Git diff collection, dependency propagation, validation, range edits, and final sitemap emission.
Isolated integration test: ran the real command against a temporary Git repository and observed the expected pre-stamp failure, targeted writes, and successful post-stamp check.
Release gates: ran article validation, Mermaid rendering, publisher dry-run, TypeScript checking, and the publisher repository's complete test suite before the authorized public write.

The focused sitemap test passed one test with zero failures. Running list
against the evidence repository's current HEAD returned no affected routes
for its unrelated working-tree files, and the repository status was unchanged
after the experiment.

Conclusion

Accurate sitemap dates require a route-impact model, not a clock in the sitemap
generator.

Persist each route's date, compare semantic route records against Git, propagate
significant shared dependencies explicitly, and fail CI when an affected route
still has its previous timestamp. Use AST ranges to update only the selected
fields and isolated Git fixtures to test both the failure and success paths.

The design will never infer every architectural dependency automatically. That
is acceptable when the dependency map is visible, reviewable, and backed by
manual route overrides.

The result is a sitemap that changes when pages change—not merely when the site
builds.

AI assistance disclosure

AI assisted with outlining and drafting. All implementation claims were
checked against the evidence repository, the route-impact behavior was
verified with its isolated Git integration test, and external behavior claims
were checked against primary sitemap documentation before publication.

Why a Batch Publisher Should Isolate Article Failures

xiaoxu — Wed, 29 Jul 2026 05:35:10 +0000

Why a Batch Publisher Should Isolate Article Failures

Why this matters

A batch job has two audiences.

Each item needs an independent result: one malformed article should not prevent
an unrelated article from being prepared. The scheduler or CI job needs an
honest aggregate result: if any item failed, the batch must not report success.

The naive implementations satisfy only one side:

for (const article of articles) {
  await publish(article); // first rejection aborts the loop
}

This is fail-fast. It is simple, but one bad file blocks every later file.

The opposite mistake catches everything and exits successfully:

for (const article of articles) {
  try {
    await publish(article);
  } catch {
    // keep going
  }
}

This keeps the batch moving but lies to automation. CI turns green even though
some work was lost.

The useful middle is collect and continue: isolate each item, retain its
failure, process the rest, and return a non-zero final result.

What I built or tested

I traced a TypeScript CLI with a publish-all command and exercised it using a
no-network fixture:

two valid Markdown articles;
one Markdown article missing its required title;
one non-Markdown file; and
one valid article inside a nested directory.

The observed result was:

{
  "exitStatus": 1,
  "completed": ["batch-valid-one", "batch-valid-two"],
  "failedArticles": 1,
  "aggregateFailure": "1 article(s) failed",
  "ignoredNonMarkdown": true
}

Both valid articles completed even though the invalid article failed. The
non-Markdown file was ignored. After every discovered Markdown file had been
considered, the command exited with status 1.

The experiment ran with --dry-run --platform devto. It used in-memory
publication state and preview storage, so it made no platform or asset-storage
write.

Setup

The project requires Node.js 22 or later and uses a TypeScript CLI. The relevant
command shape is:

blog-publisher publish-all ./ready-articles \
  --dry-run \
  --platform devto

Use a deliberately scoped input directory. The discovery function recursively
accepts every filename ending in .md; it does not know that README.md,
private experiment notes, an authoring source.md, and a rendered index.md
have different roles.

A safe input boundary might look like this:

ready-articles/
├── article-a.md
├── article-b.md
└── release-3/
    └── article-c.md

Do not point the command at a repository root or an article workspace
containing multiple Markdown artifacts unless every matching file is truly a
publishable input.

The experiment used dry-run intentionally. Testing failure isolation does not
require three real publication attempts.

Step-by-step walkthrough

1. Discover candidates recursively

The CLI walks a directory using readdir() with directory entries. A directory
recurses; a file is included only when its name ends with .md:

async function articleFiles(directory: string): Promise<string[]> {
  const entries = await readdir(directory, { withFileTypes: true });
  const nested = await Promise.all(
    entries.map(async (entry) => {
      const entryPath = path.join(directory, entry.name);
      if (entry.isDirectory()) return articleFiles(entryPath);
      return entry.isFile() && entry.name.endsWith(".md")
        ? [entryPath]
        : [];
    }),
  );
  return nested.flat();
}

This explains two experiment results: the nested valid article was found, and
the .txt fixture was ignored.

It also exposes a constraint. The implementation does not filter by a canonical
filename such as index.md, does not exclude private directories, and does not
explicitly sort the resulting paths. Treat the directory contents as the
manifest; do not make ordering a business dependency.

2. Await one article at a time

The batch loop is sequential:

const failures: string[] = [];

for (const article of files) {
  try {
    const output = await publishArticle(article, config, options);
    logger.info({ article, slug: output.slug }, "article completed");
  } catch (error) {
    failures.push(article);
    logger.error({ err: error, article }, "article failed");
  }
}

The await inside for...of matters. The next article does not begin until
the current promise settles.

Sequential execution is not a throughput claim. It is a conservative default
for a workflow with shared credentials, platform rate limits, image state, and
publication records. Parallelism may be appropriate later, but only after those
shared boundaries have explicit concurrency guarantees.

3. Catch at the item boundary

The try block covers one call to publishArticle, not the whole loop. That is
the isolation boundary.

In the experiment, the invalid fixture failed front matter validation because
it lacked a title. The catch block recorded that file and returned control to
the loop. The later valid article still ran.

This is different from hiding the error. The logger retains the original error
beside the article path, and the path is added to the failure collection.

4. Fail the batch after processing the remainder

After the loop:

if (failures.length) {
  throw new Error(`${failures.length} article(s) failed`);
}

The command's top-level error handler sets a non-zero process exit code. CI
therefore receives a failed job, even though successful items were not
discarded.

The full control flow is:

Per-item isolation preserves progress; the final branch preserves batch
truthfulness.

What went wrong

The intentionally malformed file demonstrated the desired item isolation, but
source inspection revealed three broader limits.

First, the final thrown error contains only a count. The detailed per-item
errors exist in logs, while the in-memory failures array stores only paths.
A machine that receives only the process result knows that one item failed but
does not receive a structured outcome manifest.

Second, the batch has no durable checkpoint. If the process crashes halfway
through, publish-all itself cannot resume at item four. A later invocation
rediscovers the directory. The lower publication layer retains platform
records, which helps individual create/update safety, but that is not a batch
run ledger.

Third, recursive “all Markdown” discovery is broader than many article
repositories expect. A generated article workspace can contain:

source.md
index.md
published.md
.work/experiment.md

All four names end in .md. Pointing a generic recursive batch at that
workspace can attempt multiple representations of the same article and private
notes that were never intended for publication. Failure isolation limits the
blast radius of bad input; it does not make input selection correct.

Fix or mitigation

Keep the simple loop for small, carefully scoped batches, but strengthen its
contract.

Return item-level outcomes

Represent every attempted item explicitly:

type BatchOutcome =
  | { article: string; status: "succeeded"; slug: string }
  | { article: string; status: "failed"; error: string }
  | { article: string; status: "unknown"; error: string };

const outcomes: BatchOutcome[] = [];

for (const article of files) {
  try {
    const output = await publishArticle(article, config, options);
    outcomes.push({ article, status: "succeeded", slug: output.slug });
  } catch (error) {
    outcomes.push({
      article,
      status: "failed",
      error: error instanceof Error ? error.message : String(error),
    });
  }
}

await writeBatchReport(outcomes);

if (outcomes.some((item) => item.status !== "succeeded")) {
  process.exitCode = 1;
}

Persisting the report makes the partial result usable by CI, operators, and a
future resume command.

Do not flatten an ambiguous remote write into ordinary failed. If an
individual publisher cannot prove whether a create succeeded, retain
unknown and require reconciliation before another create attempt.

Make discovery a contract

Choose one of these input policies:

accept an explicit manifest of article paths;
include only a canonical filename such as index.md;
exclude private and generated directories by rule; or
copy approved inputs into a clean staging directory.

An explicit manifest is the strongest option when order, review, or per-item
metadata matters.

Add a focused regression test

The current repository has no focused publish-all test. A regression test
should inject or spawn a publisher with three items:

success;
failure;
success.

Assert that all three were attempted in sequence, both successes were retained,
and the aggregate result was non-zero. Add separate cases for an empty
directory, nested discovery, and excluded files.

Trade-offs

Sequential processing is easy to reason about and gentler on shared systems, but total duration grows with every article.
Continuing after failure preserves independent progress, but the batch is no longer atomic. Some items may be public while the final command fails.
A structured outcome manifest improves recovery and observability, but it introduces another durable artifact with retention and redaction concerns.
A broad recursive scanner is convenient for simple staging directories but dangerous in repositories containing multiple Markdown representations.
An explicit manifest adds preparation work, but it makes the intended set and order reviewable.
Dry-run testing verifies local control flow without publication risk. It cannot prove remote rate limits, credentials, or delivery.

How I verified it

I used four checks:

Source trace: inspected recursive discovery, the awaited for...of loop, the per-item catch, and the aggregate throw.
Isolated experiment: spawned the real CLI in DEV.to dry-run mode against two valid articles, one invalid article, a nested directory, and an ignored non-Markdown file.
Assertions: required both valid slugs, exactly one recorded failure, exit status 1, and the aggregate failure message.
Release gates: ran the article validator, Mermaid render, publisher dry-run, TypeScript check, and complete repository test suite before the authorized public write.

The experiment verified local discovery and error-control semantics. It did
not contact DEV.to, upload an image, benchmark batch speed, or simulate a
process crash.

Conclusion

A resilient batch publisher should not choose between “stop at the first bad
article” and “pretend every article succeeded.”

Catch failures at the item boundary, continue independent work, and return a
non-zero aggregate result. Then make the input set explicit and persist
item-level outcomes when the job must survive restarts.

That pattern is small enough for a CLI and strong enough to provide the two
truths a batch needs: which items completed, and whether the batch as a whole
was clean.

AI assistance disclosure

AI assisted with outlining and drafting. Every implementation claim was
checked against the repository, and the failure-isolation behavior was
verified with a local no-network CLI experiment before publication.

Why a GraphQL Publisher Must Inspect Errors on HTTP 200

xiaoxu — Tue, 28 Jul 2026 05:37:01 +0000

Why a GraphQL Publisher Must Inspect Errors on HTTP 200

Why this matters

An HTTP 200 proves that a server returned a successful HTTP response. It does
not prove that the GraphQL operation inside that response succeeded.

That distinction is easy to miss in a publishing pipeline. The transport can
work perfectly while the mutation is rejected because of permissions, invalid
input, or a resolver failure. If the client checks only response.ok, it may
read missing fields as if they were a successful post and persist a result that
never existed.

The inverse is also possible: a GraphQL response can contain both data and
errors. The
GraphQL response specification
allows partial data when field errors occur, and the current
GraphQL-over-HTTP working draft
explains why field errors can still travel in a successful HTTP response.

I traced this boundary in a TypeScript publisher and tested it with a mocked
fetch. The useful result is a three-gate rule:

Check the HTTP status.
Inspect the GraphQL errors list.
Require the expected mutation data before recording success.

What I built or tested

The publisher has a small Hashnode adapter. Its shared HTTP helper rejects
non-2xx responses and attaches the status to the thrown error. After that
transport gate passes, the adapter parses a GraphQL envelope shaped like this:

interface GraphqlResponse<T> {
  data?: T;
  errors?: Array<{ message: string }>;
}

I exercised the adapter through its public createPost() method with five
synthetic responses:

Response	Observed result
HTTP 200 with the expected post data	Returned the post ID and URL
HTTP 200 with two GraphQL errors	Rejected with both messages
HTTP 200 with `{}`	Rejected because data was missing
HTTP 200 with post data and one error	Rejected under the adapter's all-or-nothing policy
HTTP 403 with an error body	Rejected with status `403`

The experiment replaced globalThis.fetch, used fake configuration, and made
no network request. It verified adapter behavior, not live Hashnode
availability or credentials.

Setup

The relevant environment is Node.js 22 or later with TypeScript. A minimal
mutation client needs a target endpoint and token, but the token belongs in an
environment variable rather than source code:

const response = await fetch("https://example.invalid/graphql", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: process.env.GRAPHQL_TOKEN!,
  },
  body: JSON.stringify({ query, variables }),
});

For Hashnode specifically, API writes currently require an eligible Pro
publication. Hashnode's official
API access announcement
and
GraphQL agent guidance
also make an operational point that matters here: access errors should stop
the workflow instead of entering a blind retry loop.

The experiment did not require that plan or a real token because it intercepted
the request locally.

Step-by-step walkthrough

1. Separate HTTP failure from GraphQL failure

The shared HTTP helper reads the response body, then rejects a non-success
status before returning parsed JSON:

const body = await response.text();

if (!response.ok) {
  const error = new Error(
    `${response.status} ${response.statusText}: ${body}`,
  ) as Error & { status?: number };
  error.status = response.status;
  throw error;
}

return JSON.parse(body);

Preserving the status is important. A definite 403 has different recovery
semantics from a timeout where the client cannot prove whether the operation
ran.

But this check is only the first gate. Calling JSON.parse() after a 200 does
not validate the GraphQL operation.

2. Treat a non-empty `errors` list as an operation result

The adapter checks the body immediately after the HTTP helper returns:

if (response.errors?.length) {
  throw new Error(
    response.errors.map((item) => item.message).join("; "),
  );
}

In the isolated experiment, this HTTP 200 response:

{
  "errors": [
    { "message": "permission denied" },
    { "message": "publication unavailable" }
  ]
}

rejected with both messages. No code attempted to read
data.publishPost.post.id.

Joining messages is a modest but useful diagnostic improvement over throwing a
generic “GraphQL failed.” A production client may also retain structured
fields such as path and safe values from extensions, subject to redaction.

3. Require the expected data boundary

An empty error list is still not proof of usable mutation data. The adapter
also checks:

if (!response.data) {
  throw new Error("GraphQL mutation returned no data");
}

My mocked HTTP 200 response containing {} reached this branch. This guards
against a malformed envelope and prevents the caller from treating undefined
as a publication result.

For stronger runtime validation, the next step would be to validate the
operation-specific shape too:

function requirePublishedPost(
  envelope: GraphqlResponse<{
    publishPost?: { post?: { id?: string; url?: string } };
  }>,
) {
  if (envelope.errors?.length) {
    throw new Error(envelope.errors.map(({ message }) => message).join("; "));
  }

  const post = envelope.data?.publishPost?.post;
  if (!post?.id || !post.url) {
    throw new Error("Mutation returned no post ID or URL");
  }

  return post;
}

The repository's current generic check proves that top-level data exists; this
operation-specific helper would make nested shape failures explicit rather
than allowing a property-access exception.

4. Persist success only after all three gates

The full response path is:

Transport, operation, and data-shape checks form separate success gates.

Only the final branch has enough evidence to store a confirmed publication.
This ordering also keeps a scheduler from mistaking “the HTTP request
completed” for “the article was published.”

What went wrong

The subtle failure was not in the GraphQL envelope check itself. It appeared
one layer later, when the publication state machine classified the thrown
error.

The adapter creates a plain Error for GraphQL messages. That error has no
HTTP status. The outer publication loop currently maps a missing status to
unknown, the same conservative state used for transport uncertainty.

That is safe for duplicate prevention, but imprecise for recovery. An explicit
permission rejection is not the same as “the request may have succeeded but
the response was lost.” Treating both as unknown means a deterministic
configuration problem can require reconciliation rather than a direct fix.

The current test suite reveals a second gap. It verifies Hashnode publication
action selection—for example, that an unknown result must be reconciled—but it
does not directly test HTTP 200 GraphQL error envelopes. The mocked experiment
covered the behavior for this article, yet turning those cases into permanent
unit tests would better protect the boundary.

There is one more important case: data and errors together. My experiment
returned a valid post plus secondary field failed. The adapter rejected it
because it checks errors first. For a mutation, that all-or-nothing policy is
conservative: a side effect may have happened, so the caller should reconcile
instead of assuming nothing changed.

Fix or mitigation

Keep the three gates, then make the error category explicit.

A reusable result type can preserve the distinction without coupling the state
machine to message text:

type PublicationFailure =
  | { kind: "transport"; status?: number; message: string }
  | { kind: "graphql"; codes: string[]; message: string }
  | { kind: "shape"; message: string };

The GraphQL adapter can collect safe extensions.code values when the API
provides them. The publication layer can then apply deliberate recovery rules:

a definite authorization or validation rejection becomes failed;
a timeout or ambiguous server failure remains unknown;
partial mutation data plus errors remains unknown until the remote state is reconciled;
a malformed success envelope stops the run and raises an integration alert.

Do not classify by searching human-readable messages if structured codes are
available. Message wording can change, may be localized, and can contain
details that should not be persisted.

A compact regression matrix is also worth keeping:

it.each([
  ["data only", dataEnvelope, "success"],
  ["errors only", errorsEnvelope, "graphql-error"],
  ["data and errors", partialEnvelope, "graphql-error"],
  ["empty envelope", {}, "shape-error"],
])("%s", async (_name, body, expected) => {
  mockFetch(200, body);
  await expect(classifyMutation()).resolves.toBe(expected);
});

This pattern verifies the protocol boundary without a production publication
or credential.

Trade-offs

Rejecting any GraphQL error is easy to reason about, but it discards usable partial query data. That may be appropriate for publication mutations and too strict for read-only dashboards.
Preserving structured error codes adds types and mapping logic. It also makes retries and operator messages substantially safer.
Requiring operation-specific data validation adds code for every mutation. A schema-generated client or runtime validator can reduce repetition, but neither removes the need for recovery policy.
Mocked transport tests are deterministic and safe. They do not prove the current remote schema, account permissions, or service availability.
Conservative reconciliation can delay automation after a partial mutation, but it is preferable to publishing a duplicate.

How I verified it

I used four layers of evidence:

Source trace: followed the request from the HTTP helper through the Hashnode envelope check and into publication failure persistence.
Isolated experiment: mocked five response combinations and asserted the returned ID, joined GraphQL messages, missing-data error, partial-response policy, and preserved HTTP status.
Test audit: searched the repository for focused Hashnode adapter tests and ran the existing state-invariant tests. The focused run passed two tests, while confirming the envelope coverage gap.
Release gates: ran the TypeScript check, complete test suite, Mermaid render, asset preparation, and publisher dry run before the authorized public write.

The experiment also captured the request method and endpoint, confirmed that
the mutation text was present, and verified that the synthetic token was not
placed in the JSON body.

These checks establish the local client's behavior. They do not claim that a
live Hashnode mutation succeeded.

Conclusion

GraphQL clients need a richer definition of success than response.ok.

Check the transport, inspect the GraphQL error envelope, and validate the
operation-specific data before persisting a remote ID. Then carry enough error
structure into the state machine to distinguish a definite rejection from an
ambiguous outcome.

That boundary is small, easy to test without network access, and reusable
across publishing, billing, provisioning, and every other automation where a
200 response can still contain a failed operation.

AI assistance disclosure

AI assisted with outlining and drafting. Every implementation claim was
checked against the repository, and all response cases were exercised with a
local mocked-fetch experiment before publication.

Keeping Asset URLs Deterministic Across ImageKit and R2

xiaoxu — Mon, 27 Jul 2026 05:36:38 +0000

Keeping Asset URLs Deterministic Across ImageKit and R2

Why this matters

A Markdown publisher should not care whether an optimized diagram lands in
ImageKit or Cloudflare R2. It should hand a storage adapter one object key and
receive one durable public URL.

That small interface hides meaningful differences.

ImageKit has an upload API that can overwrite a known path when unique
filenames are disabled. R2 exposes an S3-compatible API, so a publisher can
check an object with HeadObject before deciding whether to upload it. Their
authentication, folder semantics, public delivery configuration, and failure
modes are not interchangeable.

The goal is therefore not to pretend the providers are identical. It is to
keep the provider-neutral identity stable while making provider-specific
recovery behavior explicit.

I traced both adapters in a real TypeScript publisher and ran an isolated
experiment that captured the ImageKit form and R2 S3 commands without sending
anything over the network.

What I built or tested

The common contract is deliberately small:

interface StorageProvider {
  readonly name: string;
  exists(objectKey: string): Promise<boolean>;
  upload(params: {
    localFile: string;
    objectKey: string;
    contentType: string;
  }): Promise<string>;
  publicUrl(objectKey: string): string;
}

I tested:

default and explicit provider selection;
required configuration for each provider;
public URL construction with spaces in an object key;
the multipart fields sent to ImageKit;
R2 HeadObject and PutObject command construction;
R2 behavior for found, 404, and non-404 outcomes; and
the repository's focused ImageKit URL test.

The experiment used fake configuration, an in-memory fetch replacement, and
a local S3 command recorder. It did not contact ImageKit or R2, so it does not
claim that credentials, bucket policy, caching, or remote delivery were
verified.

Setup

Provider choice is configuration-driven. ImageKit is the default:

STORAGE_PROVIDER=imagekit
IMAGEKIT_URL_ENDPOINT=https://ik.imagekit.io/example
IMAGEKIT_PRIVATE_KEY=...
IMAGEKIT_FOLDER=/blog_img

R2 must be selected explicitly:

STORAGE_PROVIDER=r2
R2_ACCOUNT_ID=...
R2_ACCESS_KEY_ID=...
R2_SECRET_ACCESS_KEY=...
R2_BUCKET=assets
R2_PUBLIC_BASE_URL=https://assets.example.com

The factory validates provider-specific requirements before constructing an
adapter. That keeps a missing public base URL or credential from surfacing
halfway through image processing.

Notice that R2 needs two different endpoint concepts:

the account-specific S3 API endpoint for authenticated object operations;
a configured public base URL for readers.

Successful PutObject does not, by itself, prove that the resulting public
URL is reachable.

Step-by-step walkthrough

1. Keep one provider-neutral object key

The upstream image pipeline supplies keys shaped like:

blog/article-slug/diagram-contenthash.webp

Both adapters consume that same key. Their hostnames and API calls differ, but
the path identity does not. This is the most useful portability boundary:
switching providers changes where the bytes live, not how the article refers
to the logical asset.

Both publicUrl() methods encode each path segment independently. In the
experiment, a key containing diagram 1.webp became
diagram%201.webp for both providers. Encoding segments instead of an entire
URL preserves / as hierarchy while protecting spaces and other path
characters.

2. Route through a strict factory

The storage factory first checks configuration, then chooses an adapter:

return config.STORAGE_PROVIDER === "r2"
  ? new R2Storage(config)
  : new ImageKitStorage(config);

My experiment observed ImageKit for default configuration and R2 only when
STORAGE_PROVIDER=r2 was explicit. It also observed the complete missing-field
list for each provider.

This is more useful than allowing a half-configured fallback. If CI intends to
write to R2, silently selecting ImageKit would create valid URLs in the wrong
account.

3. Make ImageKit overwrite semantics deliberate

The ImageKit adapter constructs a multipart upload with:

fileName=diagram.webp
folder=/blog_img/blog/demo
useUniqueFileName=false

The current
ImageKit upload documentation
states that useUniqueFileName=false keeps the supplied filename and replaces
an existing file at the same name. That behavior is important because the
adapter deliberately returns false from exists().

Its recovery model is:

trust the local image table as the durable deduplication index;
when that record is unavailable, upload the deterministic path again;
overwrite that path instead of creating a suffixed duplicate.

The successful upload response can supply the delivery URL. If it omits one,
the adapter can derive the URL from its configured endpoint, folder, and
object key.

4. Let R2 check the object remotely

R2 takes a different path:

One object identity, two explicit recovery strategies.

The R2 adapter sends HeadObject to its configured bucket. A 404 means the
object is absent. Other errors are rethrown rather than treated as a cache
miss.

That distinction prevents an authorization failure or service problem from
silently becoming a write. “I could not check” is not the same as “the object
does not exist.”

When upload is needed, the adapter sends PutObject with the key, content
type, and:

Cache-Control: public, max-age=31536000, immutable

Cloudflare documents R2's
S3-compatible endpoint and operations,
including HeadObject, and provides
JavaScript PutObject examples.

What went wrong

The provider interface initially suggests symmetric behavior, but exists()
does not mean the same thing in both adapters.

For R2, it is a real remote existence check. For ImageKit, it always returns
false; the comment explains that the local database is the durable index and
deterministic overwrite is the fallback.

That asymmetry is intentional, but it is easy to miss in a code review. A
generic caller that assumes every exists() asks the remote provider could
draw the wrong conclusions about traffic, cost, or failure recovery.

The test gap makes the risk larger. The repository has one direct storage test
for ImageKit folder placement and encoded URLs. It has no direct R2 adapter
test for 404 classification, non-404 rethrow, upload metadata, or URL encoding.

There is also a deployment boundary that local command construction cannot
verify. R2 buckets are private by default. Cloudflare's
public bucket documentation
describes custom domains and the r2.dev development endpoint, and recommends
custom domains for production features. A correct R2_PUBLIC_BASE_URL still
requires the matching bucket exposure to be configured.

Fix or mitigation

I would keep the small interface, but add an adapter contract suite with
provider-specific cases.

Contract case	ImageKit expectation	R2 expectation
encoded public path	endpoint + folder + encoded key	public base + encoded key
known local record	caller skips provider work	caller skips provider work
storage existence check	returns false by design	sends `HeadObject`
missing remote object	deterministic overwrite upload	404 then `PutObject`
authorization/service error	upload response must fail	non-404 HEAD error rethrows
upload result	response URL or derived fallback	derived public base URL

The suite should use fake transports, as my experiment did, so it can inspect
requests without credentials. A separate integration check can then verify one
real upload in a disposable prefix for each configured provider.

I would also rename or document the existence capability more explicitly if
more adapters are added. A method such as remoteExists() communicates more
than a generic exists(), while an adapter capability flag could make
database-led recovery visible to the caller.

Trade-offs

ImageKit's deterministic overwrite path is simple and works even when the
local deduplication record is lost. The cost is that recovery performs an
upload rather than a remote existence query.

R2's HEAD-before-PUT path can recover an existing object without uploading it.
It adds another remote operation and requires careful error classification. It
also separates S3 write access from public delivery configuration.

Immutable cache headers fit content-addressed object keys, because changed
bytes should produce a changed key. They are dangerous if mutable content is
reused at the same key. The cache policy and naming policy must therefore be
reviewed together.

Finally, deterministic does not mean identical across providers. The hostname
and provider folder prefix differ. The invariant is the encoded logical object
path and repeatable mapping within each configured provider.

How I verified it

The isolated experiment produced these key observations:

{
  "factoryDefaultsTo": "imagekit",
  "factorySelectsR2": "r2",
  "imageKitExists": false,
  "imageKitUseUniqueFileName": "false",
  "r2Exists": true,
  "r2NotFound": false,
  "r2ServerErrorRethrown": true,
  "r2Commands": [
    "HeadObjectCommand",
    "PutObjectCommand"
  ]
}

It also verified that both adapters encoded a space as %20, that the
ImageKit folder contained the configured prefix exactly once, and that the R2
put command carried the expected bucket, key, content type, and immutable
cache header.

The focused ImageKit test passed. Before release, I additionally ran the
article validator, Mermaid rendering and visual inspection, publisher dry-run,
TypeScript typecheck, and the full repository test suite.

No remote provider was contacted during the experiment. A real R2 public
domain and real ImageKit authorization remain integration concerns, not
claims made by this article.

Conclusion

A useful storage abstraction does not erase provider differences. It gives
those differences a narrow place to live.

Keep one object key across adapters. Validate configuration before processing.
Encode public paths consistently. Then test each provider's actual recovery
model: deterministic overwrite for ImageKit, HEAD-before-PUT for R2, and hard
failure when an existence result is ambiguous.

That is enough to let the Markdown publisher stay provider-neutral without
pretending its storage systems behave the same.

AI assistance disclosure

I used an AI coding assistant to trace the storage adapters, prepare the
isolated transport experiment, compare provider behavior with official
ImageKit and Cloudflare documentation, and edit the draft. I reviewed the
cited source, executed the reported checks, inspected the rendered diagram,
and kept unexecuted remote behavior explicitly labeled as a limitation.

Using Front Matter as a Multi-Platform Publishing Contract

xiaoxu — Fri, 24 Jul 2026 05:38:12 +0000

Using Front Matter as a Multi-Platform Publishing Contract

Why this matters

A Markdown article often looks self-contained: front matter at the top, prose
below it, and perhaps a few local images. But once a publisher targets more
than one platform, those first lines stop being editorial decoration. They
decide where a network write goes, whether it creates a draft or a public
article, which tags cross the API boundary, and whether a canonical URL or
cover image is attached.

That makes front matter deployment configuration.

The dangerous failure mode is not always invalid YAML. A syntactically valid
file can still express a surprising operation. In the publisher I examined,
omitting platforms enables both supported platforms by default. Omitting
published is safer—it defaults to false—but setting it to true changes the
default operation to public unless the command explicitly overrides the mode.

I traced the contract from parser to adapter, exercised its defaults and
rejections with temporary fixtures, and captured the outgoing DEV.to payload
without making a network request.

What I built or tested

The publisher uses a typed metadata shape:

interface ArticleMeta {
  title: string;
  slug: string;
  description?: string;
  tags: string[];
  cover?: string;
  canonical_url?: string;
  published: boolean;
  platforms: Partial<Record<"devto" | "hashnode", boolean>>;
}

I tested four boundaries:

parsing a fully explicit article;
parsing a minimal article and observing defaults;
deriving and overriding the effective release mode; and
mapping metadata into a captured DEV.to request.

I also supplied three malformed values: a non-URL canonical value,
comma-separated tags instead of an array, and a string where a platform
boolean was required. All three failed locally with ZodError.

The experiment did not contact DEV.to or Hashnode. It verifies the local
contract and payload construction, not remote acceptance.

Setup

The source article uses this shape:

---
title: "A descriptive title"
slug: "a-descriptive-title"
description: "A short platform summary."
tags:
  - typescript
  - testing
published: false
platforms:
  devto: true
  hashnode: false
---

Two rules are worth making explicit in a review checklist:

tags is a YAML array, not a comma-separated string.
Every production article should state its platform booleans even though the parser provides defaults.

The parser accepts an optional local cover path and an optional
canonical_url. The canonical value must be a URL. A cover takes a separate
asset path through image processing before it becomes a platform-facing URL.

Step-by-step walkthrough

1. Parse once into a narrow metadata type

The Markdown loader separates YAML from body content, then validates the
metadata with Zod. title and slug must be non-empty strings. Tags default
to an empty array, published defaults to false, and platforms defaults to
this:

{ devto: true, hashnode: true }

Defaults make small examples convenient, but the platform default is broad.
If both credentials exist, an omitted field can expand the blast radius of a
later publish command. My mitigation is simple: allow the parser default for
backward compatibility, but require explicit platform values in generated and
reviewed production content.

2. Turn metadata into an operation

The publisher derives the effective mode from the article when no command
option is present:

const mode = options.mode ?? (article.meta.published ? "public" : "draft");

The CLI can supply --draft or --public, and rejects using both. This creates
a useful precedence rule:

explicit command mode > front-matter published flag > safe false default

Keeping published: false in the canonical source is therefore compatible
with a reviewed public release: the automation can run all checks first, then
make the final command's --public flag the release decision.

3. Intersect article targets with command scope

Front matter says which platforms an article permits. A repeated
--platform option can narrow that list further. The publisher collects
enabled metadata targets and intersects them with the requested platforms.

That distinction matters:

front matter is the article's durable allowlist;
the command is the current operation's requested scope.

A command should be able to narrow an allowlist, not silently widen it.

Durable article policy and one-run command scope meet before any adapter is
called.

4. Map fields at the adapter boundary

After local images are processed, the publisher constructs a platform-neutral
PublishInput containing:

title and slug;
transformed Markdown;
description and tags;
canonical and cover URLs; and
the effective public/draft boolean.

The DEV.to adapter translates that input into its API object. It converts the
tag array into a comma-separated value and takes at most four tags:

tags: input.tags.slice(0, 4).join(",")

The current
Forem API documentation
documents the same create fields—title, Markdown body, published state, tags,
main image, canonical URL, and description—and limits an article to four
tags.

My intercepted request started with five tags and contained only:

typescript,testing,automation,devops

That proves the local truncation behavior. It does not prove that the fifth
tag is unimportant. Silent truncation can hide a content mistake, so the
better generator-side rule is to reject more than four tags before the
adapter.

What went wrong

The most surprising result came from the minimal valid fixture:

---
title: "Minimal article"
slug: "minimal-article"
---

It parsed successfully with no tags, draft mode, and both platforms
enabled. That is valid according to the repository schema, but it is a risky
default for unattended publishing. The meaning of “field absent” is not “do
nothing”; it is “allow both targets.”

There is a second asymmetry. The parser accepts any number of tags, while the
DEV.to adapter sends only the first four. Validation and transport therefore
do not express the same limit. The payload is legal, but an author can believe
five tags were selected when one was silently discarded.

Finally, the test suite has focused DEV.to adapter coverage but no direct
table-driven test for front-matter defaults and malformed values. TypeScript
interfaces cannot close that gap because YAML arrives as runtime data.

Fix or mitigation

I would enforce the contract at three levels.

First, keep runtime schema validation. It correctly rejected:

canonical_url: "not-a-url";
tags: "typescript,testing"; and
devto: "yes".

Second, make production policy stricter than parser compatibility:

published: false
platforms:
  devto: true
  hashnode: false

Require one to four lowercase tags for DEV.to and omit canonical_url unless
the article actually has an original publication URL. Require a local relative
cover path only when that file exists.

Third, add table-driven contract tests. A small matrix should assert:

Case	Expected result
explicit single target	only that target is enabled
omitted platforms	documented default is returned
public flag with draft override	effective mode is draft
five DEV.to tags	validation fails before truncation
malformed canonical URL	parser rejects
string platform flag	parser rejects

The key is to test both valid defaults and invalid inputs. Testing only the
happy-path YAML leaves the most operationally important semantics implicit.

Trade-offs

Front matter keeps article policy beside the content. It is reviewable in Git,
portable across environments, and easy for a generator to produce. A
platform-neutral PublishInput also prevents every adapter from reparsing
Markdown metadata independently.

The same convenience creates coupling:

changing a parser default changes old articles that omitted the field;
platform-specific limits can be hidden behind a generic type;
a boolean published flag mixes authoring intent with release behavior;
canonical URLs and covers require validation beyond TypeScript's compile time; and
multi-platform metadata tends toward the least common denominator.

For a larger system, I would separate compatible parsing defaults from a
stricter release policy validator. The parser answers “can I understand this
file?” The release gate answers “is this exact operation allowed now?”

How I verified it

The isolated experiment observed:

{
  "minimalDefaults": {
    "tags": [],
    "published": false,
    "platforms": {
      "devto": true,
      "hashnode": true
    }
  },
  "modeFromPublishedFlag": "public",
  "explicitModeOverride": "draft",
  "capturedDevtoTags": "typescript,testing,automation,devops",
  "invalidCanonicalUrl": "ZodError",
  "commaSeparatedTags": "ZodError",
  "nonBooleanPlatform": "ZodError"
}

The adapter call used an in-memory replacement for fetch, so no API received
the fixture or fake credential. The repository's focused DEV.to test file also
passed all three tests.

Before public release, I additionally ran the article validator, Mermaid
rendering and visual inspection, the publisher dry-run, the TypeScript
typecheck, and the complete repository test suite.

Conclusion

Front matter is a compact publishing contract only when its semantics are
explicit and tested. The most important fields are not the visible title and
description; they are the ones that control side effects: targets, mode,
canonical identity, cover processing, and platform limits.

Parse runtime data into a narrow type, intersect durable article permissions
with one-run command scope, and validate platform constraints before the
adapter silently normalizes them. Keep the source in draft mode and make the
final public flag a reviewed release decision.

That turns a few lines of YAML from hopeful metadata into an auditable safety
boundary.

AI assistance disclosure

I used an AI coding assistant to trace metadata from parser to platform
adapter, create the isolated fixture experiment, compare the payload with
official Forem documentation, and edit the draft. I reviewed the cited source,
executed the reported checks, inspected the rendered diagram, and made the
final publication decision only after the automated gates passed.

Using SQLite Locally and MySQL in CI for Publication State

xiaoxu — Thu, 23 Jul 2026 05:37:02 +0000

Using SQLite Locally and MySQL in CI for Publication State

Why this matters

Publishing is not a fire-and-forget HTTP request. A practical publisher needs
to remember which source hash it sent, which remote ID and URL came back, and
which rendered images were already uploaded. That state prevents duplicate
posts and unnecessary asset uploads.

For a command-line tool, SQLite is a great local default: it requires no
service and keeps state beside the project. CI or a shared runner may need
MySQL instead. The tempting design is to put both behind one TypeScript
interface and assume the backends are interchangeable.

The interface is necessary, but it is not proof of parity. SQL dialects,
timestamp types, JSON decoding, and concurrency semantics still cross the
boundary. I tested the local path, compared both implementations line by line,
and found one MySQL-specific risk that the existing unit test cannot see.

What I built or tested

The repository already defines a small DatabaseProvider contract:

interface DatabaseProvider {
  initialize(): Promise<void>;
  findPublishRecord(slug: string): Promise<PublishRecord | null>;
  savePublishRecord(record: PublishRecord): Promise<void>;
  findImage(provider: string, objectKey: string): Promise<ImageRecord | null>;
  saveImage(record: ImageRecord): Promise<void>;
  close(): Promise<void>;
}

Both implementations store two kinds of idempotency state:

a publication record keyed by article slug; and
an image record keyed by storage provider plus object key.

I exercised SQLite with repeated writes and a reopen, checked provider
selection and configuration failure, ran the focused test, and inspected the
MySQL SQL and installed driver. I did not run a MySQL server, so this is not
an end-to-end MySQL compatibility claim.

Setup

The project runs on Node.js 22 and TypeScript. Its configuration defaults to:

DATABASE_DRIVER=sqlite
DATABASE_SQLITE_PATH=.publish/blog-publisher.db

Selecting MySQL is explicit:

DATABASE_DRIVER=mysql
DATABASE_HOST=mysql
DATABASE_PORT=3306
DATABASE_NAME=publisher
DATABASE_USERNAME=publisher
DATABASE_PASSWORD=...

The factory rejects the MySQL choice when host, database name, or username is
missing. That early failure matters: silently falling back to a local file in
CI would split state between runs.

Step-by-step walkthrough

1. Keep the logical keys identical

SQLite creates a publish_records table with slug as its primary key and an
images table with a composite (provider, object_key) primary key. MySQL
uses the same logical keys with bounded VARCHAR columns.

That gives both implementations the same answer to “is this the record I
already know?” even though their physical types differ.

2. Express an upsert in each dialect

SQLite writes a publication like this:

INSERT INTO publish_records
  (slug, source_hash, platforms_json, updated_at)
VALUES (?, ?, ?, ?)
ON CONFLICT(slug) DO UPDATE SET
  source_hash = excluded.source_hash,
  platforms_json = excluded.platforms_json,
  updated_at = excluded.updated_at

MySQL uses its corresponding form:

INSERT INTO publish_records
  (slug, source_hash, platforms_json, updated_at)
VALUES (?, ?, ?, UTC_TIMESTAMP(3))
ON DUPLICATE KEY UPDATE
  source_hash = VALUES(source_hash),
  platforms_json = VALUES(platforms_json),
  updated_at = VALUES(updated_at)

The syntax differs, but the intended invariant is the same: a second write for
the same logical key updates one row instead of creating another. The
MySQL upsert documentation
is the source of truth for that server-side behavior.

3. Normalize records at the provider boundary

The rest of the publisher should not know whether SQLite stores platform state
as text or MySQL stores it in a native JSON column. Both providers serialize a
platforms object on write and should return the same JavaScript shape on
read.

One application contract, two storage adapters, one logical state model.

4. Test replacement and persistence, not just insertion

My experiment wrote one slug first as a draft with remote ID 42, then as
published with ID 99. It also wrote the same image key twice with different
hashes and URLs. After closing and reopening SQLite, I queried the records and
counted the rows.

The result contained one publication and one image—the updated values survived
the reopen. The database also reported wal journal mode. The official
SQLite WAL documentation explains why WAL
can improve reader/writer concurrency, while still allowing only one writer at
a time. It also documents a crucial boundary: WAL relies on shared memory and
is not a network-filesystem design.

What went wrong

The schemas look parallel, but the MySQL JSON read boundary is not.

The provider declares platforms_json as a string and does this:

platforms: JSON.parse(row.platforms_json)

However, the installed mysql2 3.22.6 driver defaults jsonStrings to
false. Its result parser decodes a MySQL JSON column with JSON.parse before
returning the row. The pool configuration does not override that default.

That means the provider may receive an object and attempt to parse it again.
In plain JavaScript, JSON.parse({ devto: { status: "published" } }) throws a
SyntaxError because the object is coerced to "[object Object]".

This is a source-grounded compatibility risk, not a live-server reproduction.
The existing focused test only instantiates SqliteDatabaseProvider, so it
cannot catch the mismatch. MySQL's
native JSON documentation
confirms that the column validates and stores JSON documents; how a Node.js
driver returns that value remains a client-boundary concern.

Fix or mitigation

There are two defensible fixes:

Set jsonStrings: true in the MySQL pool and keep the provider's explicit JSON.parse.
Let mysql2 decode JSON, type the returned field as the platform-state object, and normalize string-or-object input in one helper if mixed drivers must be supported.

I prefer the second approach when the application controls its driver version:
the provider's runtime type matches the driver's default behavior, and parsing
is not duplicated. The first approach is smaller and preserves the existing
provider code. Either choice is incomplete without a MySQL integration test.

That test should start a disposable MySQL instance, initialize the schema,
write each logical key twice, reconnect, and assert:

exactly one row per key;
the second hash, status, remote ID, and URL are returned;
the platform state has the same JavaScript shape as SQLite; and
incomplete configuration fails before a connection attempt.

Trade-offs

SQLite keeps local development delightfully small, but the database file and
its WAL-related files are persistent state and should stay on one host. It is
not a shortcut to a shared network database.

MySQL supports a service-oriented CI topology and native JSON, but adds
credentials, lifecycle management, migrations, and a real network failure
surface. It also makes driver behavior part of the persistence contract.

There are subtler parity differences too:

SQLite writes ISO-8601 timestamp text from the Node.js process; MySQL writes DATETIME(3) using UTC_TIMESTAMP(3).
SQLite's text columns do not impose the same length limits as MySQL's VARCHAR keys.
Matching method signatures do not guarantee matching transaction, contention, collation, or error semantics.

The useful abstraction is therefore a promise backed by contract tests, not a
claim that both databases are identical.

How I verified it

The local experiment produced:

{
  "journalMode": "wal",
  "publishCount": 1,
  "imageCount": 1,
  "reopenedPublishHash": "hash-v2",
  "reopenedPublishStatus": "published",
  "reopenedPublishRemoteId": "99",
  "reopenedImageHash": "image-v2",
  "defaultDatabaseDriver": "sqlite",
  "defaultProviderName": "SqliteDatabaseProvider",
  "missingMysqlConfigRejected": true
}

The focused SQLite Vitest test passed. I also ran the repository typecheck and
full test suite after drafting the article and rendering its diagram.

For MySQL, verification was deliberately narrower: I compared the provider
code, official server documentation, and the installed driver's configuration
and parser source. A live MySQL round trip remains the next required gate
before calling the two implementations operationally equivalent.

Conclusion

A provider interface can keep publication logic clean, and SQLite can give
local runs durable idempotency state with almost no setup. Moving the same
contract to MySQL requires more than translating ON CONFLICT into ON DUPLICATE KEY UPDATE.

Test the observable contract across both adapters: replacement, reopen,
decoded value shapes, and failure behavior. The JSON double-parse risk here is
exactly the kind of boundary bug that strong TypeScript types can hide when a
driver's runtime value disagrees with a handwritten row interface.

AI assistance disclosure

I used an AI coding assistant to trace the two provider implementations,
prepare the isolated SQLite experiment, compare the installed driver's JSON
path, and edit the draft. I reviewed the cited source lines, executed the
reported commands, inspected the rendered diagram, and kept the unexecuted
MySQL path explicitly labeled as a limitation.

Designing a No-Network Dry Run for an Image-Aware Publisher

xiaoxu — Wed, 22 Jul 2026 05:37:26 +0000

Designing a No-Network Dry Run for an Image-Aware Publisher

Why this matters

A useful dry run should do more than print “would publish.” It should parse the real article, process the real local images, build the real platform payload, and expose failures before credentials can create a remote artifact.

That requirement gets harder when a Markdown publisher owns images as well as text. The normal path may optimize files, query persistent upload records, contact object storage, replace Markdown URLs, and finally call a publishing API. Simply skipping the last HTTP request leaves several earlier network writes intact.

The focused question is:

How can an image-aware publisher exercise its real transformation path while making storage and platform writes impossible?

I traced and tested a TypeScript publisher that answers with dependency substitution plus an early return: use an in-memory database, use a preview storage provider, process the article normally, then stop before persistent publication state and platform clients are touched.

What I built or tested

I ran the publisher against a generated SVG fixture with all of these flags together:

npm run publish -- fixture/index.md \
  --public \
  --platform devto \
  --dry-run \
  --output-markdown fixture/preview.md

Using --public was deliberate. It exercised construction of a public payload while --dry-run remained the higher-level safety boundary.

The observed result was:

Check	Result
Reported mode	`public`
Platform results	Empty
Image URL	`https://preview.invalid/...`
Persistent SQLite checksum	Unchanged
Local processed-image cache	Changed

That last row is important. This implementation provides a no-network dry run, not a zero-side-effect sandbox.

Setup

The experiment used Node.js 22, the repository's TypeScript CLI, a small local SVG, and an existing SQLite publication database whose checksum was recorded before and after the run. It used no production images and no remote requests.

The real DEV/Forem create operation is POST /api/articles; the official API contract uses the published field to choose draft or immediate publication. The current contract is documented in the Forem v1 API reference. A trustworthy dry run must stop before this operation regardless of whether the requested payload says published: true.

The architecture already exposes two interfaces that make this possible:

interface DatabaseProvider {
  findPublishRecord(slug: string): Promise<PublishRecord | null>;
  savePublishRecord(record: PublishRecord): Promise<void>;
  findImage(provider: string, objectKey: string): Promise<ImageRecord | null>;
  saveImage(record: ImageRecord): Promise<void>;
}

interface StorageProvider {
  readonly name: string;
  exists(objectKey: string): Promise<boolean>;
  upload(input: UploadInput): Promise<string>;
  publicUrl(objectKey: string): string;
}

The dry-run path can therefore replace behaviors at the provider boundary instead of sprinkling if (dryRun) around every network call.

Step-by-step walkthrough

1. Choose safe dependencies before processing

Immediately after parsing the article, the publisher selects its database and storage implementations:

const database = options.dryRun
  ? new MemoryDatabaseProvider()
  : createDatabaseProvider(config);

const storage = options.dryRun
  ? createPreviewStorage()
  : createStorageProvider(config);

The in-memory provider implements the same publication and image-record operations as SQLite or MySQL, but its maps disappear when the process ends. Preview storage implements the same object methods as ImageKit or R2 without importing a network client.

Choosing these dependencies at the top is the critical safety property. Every later image operation receives the preview providers through the normal call chain.

2. Process local assets through the real pipeline

Dry run does not bypass article parsing or image replacement. The publisher still resolves local paths, optimizes images, computes content hashes, builds object keys, and replaces Markdown image nodes.

Preview storage intentionally reports that objects do not already exist. Its upload method returns the same URL shape its publicUrl method would produce:

publicUrl(objectKey: string): string {
  return `https://preview.invalid/${encodePath(objectKey)}`;
}

async upload({ objectKey }: { objectKey: string }): Promise<string> {
  return this.publicUrl(objectKey);
}

The reserved .invalid top-level domain makes the artifact visibly non-production. The preview Markdown shows the transformed path without creating an address that could be mistaken for a successful upload.

3. Build the actual publication input

After image replacement, the publisher constructs the same PublishInput used by a real run: title, rendered Markdown, description, tags, canonical URL, cover URL, and the requested published state.

This is why --public --dry-run is a useful test. It proves that public-mode payload construction works while still preventing a platform call.

4. Return before persistent publication and platform work

The decisive guard appears after payload construction and before persistent state lookup:

if (options.dryRun || options.prepareOnly) return output;

Only code after this point loads an existing publication record, calculates platform actions, constructs the DEV.to publisher, and sends a create or update request.

The diagram highlights both safety layers: external providers are replaced before transformation, and the platform path is cut off after the payload exists.

5. Write an inspectable artifact

The CLI can write the transformed Markdown to a requested output path. That artifact includes preview image URLs and preserves the article metadata, so a reviewer can inspect exactly what would be passed downstream.

This is a local write, but it is intentional and named by the caller. It is different from a remote publication or durable publication-state change.

What went wrong

My initial mental model was “dry run means no writes.” The experiment disproved that wording.

The persistent SQLite checksum stayed unchanged, and no platform result appeared, but the processed-image cache changed. The reason is visible in control flow: the publisher uses a fixed local cache directory, and image optimization happens before the dry-run early return. Preview storage prevents a remote upload; it does not prevent Sharp or file-copy output.

This distinction matters in CI and agent workflows. A supposedly clean preview can leave an untracked cache object, affect later file counts, or make a workspace appear dirty even though no network write occurred.

There are also boundaries the dry run cannot verify:

whether the API key is accepted by DEV.to;
whether ImageKit or R2 permissions are valid remotely;
how the published page renders on the platform;
rate limits, moderation, or remote validation behavior;
whether a real create returns a stable public URL.

The repository currently lacks a focused unit test that calls the publication function with dryRun: true. Source tracing and the CLI experiment establish the current behavior, but a regression test would make the no-network boundary durable.

Fix or mitigation

The immediate fix is precise language: promise no storage-provider or publishing-platform network writes, not “no side effects.” Then make local mutations explicit.

A reusable dry-run checklist is:

Select fake or in-memory external dependencies before business logic begins.
Exercise the same parser, validator, image transformer, and payload builder as production.
Use an unmistakably non-production URL domain for preview assets.
Return before persistent publication lookup and every platform client.
Write preview artifacts only to caller-selected paths.
Record or isolate any local cache directory used during transformation.
Test dry run with the most dangerous requested mode, such as --public.
Assert that persistent state is unchanged and platform results are empty.

For a truly clean workspace, inject the cache directory too. A dry run could use a temporary directory that is removed afterward, while real preparation retains the durable cache. That change would preserve transformation fidelity without leaving new files in the repository.

Trade-offs

Dependency substitution adds implementations that must remain behaviorally compatible with production providers. Preview storage can model URL construction and image replacement, but it cannot reproduce authentication failures, latency, provider-specific validation, or eventual consistency.

Running real image processing improves confidence but costs CPU and creates temporary or cached files. Skipping image work would be faster and cleaner, but it would miss broken paths, unsupported formats, optimizer failures, and incorrect Markdown replacement—the failures a useful preview is meant to catch.

An in-memory database avoids persistent state changes, but it also starts empty on every run. It cannot reveal a conflict with an existing remote ID or an unknown publication state. Reconciliation and update behavior need separate state-machine tests.

The design therefore optimizes for payload fidelity and network safety, not complete production simulation.

How I verified it

I recorded the SQLite file checksum before and after a public-mode dry run; it was unchanged. The CLI reported dryRun: true, returned no platform result, and wrote one preview.invalid image reference into the preview artifact.

I also inspected the local cache and found the generated SVG copy, confirming the local-side-effect limitation rather than hiding it. Source tracing tied each observation to the provider selection, image-processing sequence, and early return.

Finally, I checked the official Forem v1 API reference for the actual article-create boundary. The repository's DEV.to client sends that create only after the dry-run return point.

Before publication, the TypeScript check passed and the full Vitest suite passed all 12 tests across seven test files. The article validator also rendered and previewed the Mermaid diagram without a network write.

Conclusion

A trustworthy dry run is not a parallel toy implementation. It is the real transformation pipeline with dangerous dependencies replaced and a hard stop before irreversible work.

For an image-aware publisher, that means using in-memory state, producing clearly fake asset URLs, processing local files normally, constructing the requested payload, and returning before persistent publication state or platform APIs.

Just name the guarantee honestly. This design is no-network, not no-filesystem. Once that boundary is explicit, it becomes straightforward to test, document, and tighten with a temporary cache when a completely clean preview is required.

AI assistance disclosure

AI assisted with outlining and drafting. Technical claims were checked against repository source, a recorded local experiment, and the official Forem API reference. No credentials, private paths, or production data are included.