Conditional Writes Across S3, Azure Blob and Google Cloud Storage – Without Provider SDKs

Alexander Kaufmann — Fri, 06 Nov 2026 14:00:00 +0000

How to prevent lost updates and race conditions when working with object storage in TypeScript.

The problem: concurrent writes

Imagine two services accessing the same object:

config.json

Both services download the current version and start modifying it.

Service A updates the configuration.
Service B updates the configuration.
Both upload their changes.

Without additional protection, the last write wins.

Service B might silently overwrite Service A's changes.

This is commonly known as the lost update problem.

For applications using object storage as a persistence layer, this can become a serious consistency issue.

Enter conditional writes

HTTP already provides mechanisms to solve this problem.

Two particularly useful headers are:

If-None-Match
If-Match

They allow clients to express conditions that must be satisfied before a write is accepted.

1. Create an object only if it doesn't exist

Imagine uploading a new file that must never overwrite an existing object.

PUT /documents/report.pdf HTTP/1.1
If-None-Match: *
Content-Type: application/pdf

The storage provider should accept the upload only if the object does not already exist.

If it does exist, the request is rejected.

The exact HTTP response depends on the provider. For example:

AWS S3 can return 412 Precondition Failed.
Azure Blob Storage can return 409 BlobAlreadyExists.

This is useful for:

Preventing accidental overwrites
Idempotent upload workflows
Distributed job processing
Creating objects safely from multiple workers

2. Update an object only if it hasn't changed

Now consider an existing object.

First, retrieve its metadata:

HEAD /documents/config.json HTTP/1.1

The response typically includes an ETag that can be used as a validator:

HTTP/1.1 200 OK
ETag: "abc123"

Use that ETag when updating the object:

PUT /documents/config.json HTTP/1.1
If-Match: "abc123"
Content-Type: application/json

The update succeeds only if the current representation still matches the supplied ETag.

If another process modified the object in the meantime, the write is rejected.

This is essentially optimistic concurrency control without requiring distributed locks.

The challenge: object storage providers aren't identical

This is where things become interesting.

Although conditional requests are based on established HTTP concepts, cloud storage providers expose different APIs and capabilities.

For example:

Provider	Relevant mechanisms
AWS S3	Conditional `PutObject` requests
Azure Blob Storage	Conditional request headers and access conditions
Google Cloud Storage	Generation-match preconditions

There are important differences in semantics, supported operations and error handling.

For example, an ETag is not necessarily a content hash. Multipart uploads and provider-specific implementations can also affect how validators behave.

Google Cloud Storage's JSON API, for example, does not use If-Match as an upload precondition. It uses generation-match parameters instead.

A portable abstraction therefore needs to do more than simply forward HTTP headers.

It needs to preserve the actual semantics of each provider.

Building provider-agnostic storage with stowage

This is one of the problems I care about when building stowage: one typed object storage API for AWS S3, Cloudflare R2, Azure Blob Storage, Google Cloud Storage and the local file system, without provider SDKs underneath.

Since v0.7, writes and deletes can carry a precondition.

Create only if absent

import { isStorageError, type Storage } from "@stowage/core";

export async function createReport(
  storage: Storage,
  key: string,
  pdf: Uint8Array,
): Promise<"created" | "exists"> {
  try {
    await storage.put(key, pdf, {
      contentType: "application/pdf",
      onlyIf: "absent",
    });

    return "created";
  } catch (error) {
    if (
      isStorageError(error) &&
      error.code === "PreconditionFailed"
    ) {
      return "exists";
    }

    throw error;
  }
}

The API expresses the intention rather than exposing provider-specific HTTP headers.

Update only if unchanged

The etag comes from the object you read, and the write only succeeds if the key still holds that object.

If another writer got there first, read again and reapply your change:

The change callback should be pure and safe to execute multiple times, since a failed precondition can cause it to run again.

interface Config {
  readonly flags: Readonly<Record<string, boolean>>;
}

export async function updateConfig(
  storage: Storage,
  change: (config: Config) => Config,
): Promise<Config> {
  for (let attempt = 1; attempt <= 5; attempt++) {
    const object = await storage.get("config.json");
    const { etag } = object.stat;

    if (etag === undefined) {
      throw new Error(`${storage.provider} reports no etag`);
    }

    const updated = change(await object.json<Config>());

    try {
      await storage.put(
        "config.json",
        JSON.stringify(updated),
        {
          contentType: "application/json",
          onlyIf: { etag },
        },
      );

      return updated;
    } catch (error) {
      if (
        isStorageError(error) &&
        error.code === "PreconditionFailed"
      ) {
        continue;
      }

      throw error;
    }
  }

  throw new Error(
    "config.json kept changing; gave up after 5 attempts",
  );
}

Run two of these concurrently and both changes can end up in config.json, as long as each change is reapplied to the latest version after a failed precondition.

No distributed lock required.

Same API, different providers

Neither function knows which provider it talks to.

Only the construction differs:

import {
  fromEnv as awsEnv,
  s3Storage,
} from "@stowage/adapter-s3";

import {
  azureBlobStorage,
  fromEnv as azureEnv,
} from "@stowage/adapter-azure-blob";

import { gcsStorage } from "@stowage/adapter-gcs";

declare function accessToken(): Promise<string>;

const s3 = s3Storage({
  bucket: "my-app",
  region: "eu-north-1",
  credentials: awsEnv,
});

const azure = azureBlobStorage({
  account: "myapp",
  container: "documents",
  credentials: azureEnv,
});

const gcs = gcsStorage({
  bucket: "my-app",
  credentials: async () => ({
    accessToken: await accessToken(),
  }),
});

await updateConfig(gcs, (config) => ({
  flags: {
    ...config.flags,
    darkMode: true,
  },
}));

The same updateConfig implementation works across providers that support the required conditional write capability.

What the abstraction hides

Measured against the real providers, the "same" conditional write is not the same at all:

Situation	Provider behavior	What your code sees
`If-None-Match: *` on an existing Azure blob	`409 BlobAlreadyExists`	`PreconditionFailed`
`If-Match` on a key that no longer exists	AWS returns `404 NoSuchKey`	`PreconditionFailed`
`If-Match` on a GCS upload	JSON API ignores it and would overwrite the object	stowage uses `ifGenerationMatch`
Conditional copy on R2	R2 ignores `If-Match` on the copy destination	stowage uses R2-specific `cf-copy-destination-if-*` headers

The GCS adapter uses generation-based conditions for uploads and maps the generation information to the ETag exposed by stowage.

Cloudflare R2 has another interesting limitation: conditional destination headers are required for conditional copy operations.

Where a provider cannot enforce a precondition, stowage doesn't pretend it can.

For example, R2 silently ignores If-Match on DeleteObject. The S3 adapter therefore does not advertise conditional delete as a supported capability.

capabilities tells you what the provider actually supports.

Passing onlyIf for an unsupported operation fails with Unsupported before a request is sent.

A precondition is never silently dropped.

Tested against real providers

Every row in that table is covered by a test case in stowage's conformance suite.

On every pull request, the suite runs against emulators (SeaweedFS, Azurite and fake-gcs-server). Every day and before each release, it runs against real AWS S3, Cloudflare R2, Azure Blob Storage and Google Cloud Storage buckets, on Node 24, Node 26 and workerd.

These provider quirks can only be verified against the actual services: no emulator stands in for R2, and the emulators don't reproduce every quirk of the services they imitate.

I also tested concurrent operations against the in-memory adapter:

Concurrent create operations:

Concurrent createReport calls:

Request 1 → created
Request 2 → exists

Both requests attempted to create the same object. Only one succeeded.

Concurrent update operations:

Starting with an empty configuration:

{
  "flags": {}
}

Two concurrent updates enabled darkMode and betaSearch.

The final result was:

{
  "flags": {
    "darkMode": true,
    "betaSearch": true
  }
}

Neither update was lost.

The examples also pass TypeScript's strict type checking (--strict).

Why this matters

Object storage is increasingly used as more than just a place to upload files.

It is often part of:

Distributed application architectures
Background processing pipelines
Content management systems
Data ingestion workflows
Serverless applications

As soon as multiple workers can modify the same object, concurrency becomes important.

Conditional writes are a relatively small feature with a significant impact on data integrity.

And they are a good example of why a storage abstraction needs to understand the underlying providers rather than simply wrap their APIs.

Building something similar?

I'd love to hear how others approach provider-agnostic object storage, conditional writes and concurrency across cloud storage backends.

Check out stowage on GitHub if you're interested in following the project or contributing.

Feedback and contributions are welcome!

DEV Community: Alexander Kaufmann