DEV Community: Daniel Neata

Failure Recovery for Publishing APIs: Reconcile Before Retrying

Daniel Neata — Sat, 17 Oct 2026 14:02:54 +0000

Failure Recovery for Publishing APIs: Reconcile Before Retrying

A network timeout does not tell you whether a publishing request failed. The remote service may have rejected the request, accepted it and lost the response, or created the record while your process was already disconnecting. Retrying immediately can turn one uncertain delivery into two public posts.

This guide presents a limited recovery pattern for workers that publish approved content to remote APIs. The design combines stable operation keys, explicit delivery states and reconciliation against remote history. It is intended for systems that can search recent records or attach a client-supplied identifier. If the destination provides neither capability, the worker must remain conservative and escalate uncertain writes instead of guessing.

Model uncertainty as a real state

A delivery table often begins with pending, published and failed. That model is incomplete because a transport error can leave the remote result unknown.

Add an uncertain state. Enter it when the request may have reached the destination but the response cannot be trusted. Do not automatically return an uncertain item to the normal retry queue.

A useful record contains the internal content ID, destination, operation key, request start time, response status when known, remote ID, public URL and last reconciliation time. Store error summaries, but never store access tokens or complete authorization headers.

Create a stable operation key

Generate the operation key before the first network request and persist it with the delivery attempt. It must remain stable across process restarts and operator-approved retries.

For example:

import { createHash } from "node:crypto";

function operationKey({ destination, contentId, revision }) {
  return createHash("sha256")
    .update(`${destination}:${contentId}:${revision}`)
    .digest("hex");
}

The revision matters. A corrected article can be a legitimate new operation, while restarting the worker for the same revision should not create another delivery.

If the API supports an idempotency header, send this key in the documented field. That is the strongest option because the destination can reject duplicate requests. Keep local reconciliation anyway; providers can have retention windows or endpoint-specific behaviour.

Separate selection from delivery

Claim one approved item transactionally before making a remote call. The claim prevents two runners from selecting the same item at the same time.

The worker should then use a small state machine:

async function deliver(item, adapter, store) {
  const attempt = await store.beginAttempt(item);

  try {
    const result = await adapter.publish(item, attempt.operationKey);
    await store.markPublished(attempt.id, {
      remoteId: result.id,
      publicUrl: result.url
    });
  } catch (error) {
    if (isDefiniteRejection(error)) {
      await store.markRejected(attempt.id, safeSummary(error));
      return;
    }

    await store.markUncertain(attempt.id, safeSummary(error));
  }
}

A validated 400 response is normally a definite rejection. A timeout, connection reset or invalid response body after the request was sent is not. A 500 response may also be uncertain if the provider could have completed the write before returning the error.

Reconcile before any retry

A reconciliation worker reads uncertain attempts and searches the destination using the strongest available evidence.

First, search by operation key when the platform exposes custom fields or request metadata. Otherwise, compare a bounded time window using destination account, normalized title or text, source URL and creation time. Require more than one weak attribute before declaring a match.

Do not compare only the first few recent posts. Paginate until the search crosses the attempt time plus a documented safety margin. Stop at a hard page limit and report that history verification was incomplete.

When exactly one credible match exists, save its remote ID and URL and mark the attempt published. When no match exists and the provider's consistency delay has passed, move the attempt to retry-approved rather than publishing immediately. When several matches exist, flag the attempt for review because a duplicate may already exist.

Make retry a deliberate transition

Only attempts in retry-approved may return to the delivery worker. Reuse the original operation key. Record who or what approved the retry and the evidence produced during reconciliation.

Limit attempts with a small retry budget and exponential backoff. Backoff protects the provider, but it does not solve duplication by itself. The safety property comes from stable identity and reconciliation.

For systems without remote search or idempotency support, the correct automatic action after an uncertain write may be no action. Manual inspection is slower than a retry, but safer than publishing repeated content under an account you cannot reconcile.

Verify the response before reporting success

A successful HTTP status is not enough. Require the documented identifier field and validate its type. Prefer the public URL supplied by the API. If the service returns only an ID, perform a follow-up read and use the verified URL from that response.

Keep publication counts tied to these receipts. Authentication checks, queue polling and dry runs should never increment a published counter.

Return a structured result from every adapter:

{
  "outcome": "published",
  "operationKey": "stable-key",
  "remoteId": "remote-record-id",
  "publicUrl": "https://example.test/posts/remote-record-id"
}

The URL is illustrative. A production adapter must use the actual destination response or a verified lookup.

Test the difficult boundaries

Use a fake adapter that records requests and can simulate a timeout after creating a remote record. The first run should leave the attempt uncertain. Reconciliation should discover the fake record, and the retry worker should make no second publish call.

Add tests for a definite rejection, an uncertain attempt with no remote match, paginated history, multiple candidate matches and a process crash after the remote write but before the local commit.

Also test two concurrent workers. Only one should acquire the claim. A reliable single-worker test does not prove that scheduled jobs cannot overlap in production.

Operate with evidence

A useful operational report separates four facts: the scheduler ran, a candidate was approved, the remote write was attempted, and a public record was verified. A green workflow badge proves only the first fact.

Log the operation key, internal content ID, outcome and remote receipt. Keep credentials redacted. Alert when uncertain attempts exceed their reconciliation window or when remote history cannot be read completely.

Recovery is not the same as repeating the request. The safe order is identify the intended operation, inspect the destination, reconcile the result and only then decide whether a retry is justified.

API Publishing Deduplication with Failure Recovery: Reconcile Before You Retry

Daniel Neata — Fri, 16 Oct 2026 15:40:38 +0000

API Publishing Deduplication with Failure Recovery: Reconcile Before You Retry

A publishing job sends a request. The connection closes before a response arrives. Should the job send the same request again?

The answer depends on something the worker does not yet know: whether the remote platform accepted the first request. A network timeout describes the worker's observation. It does not prove that publication failed. Treating every timeout as permission to retry is an easy way to create duplicate posts.

This guide describes a small worker design. It is an implementation pattern, not a claim that every publishing platform offers the same API features. Adapt the remote lookup and idempotency mechanism to the platform you actually use.

Separate content identity from delivery attempts

Give each reviewed content item a stable internal identifier. Then create a separate delivery record for each destination account. The combination of content ID, platform and account identifies the intended publication; an attempt ID identifies one execution of the worker.

Do not generate a new content ID merely because a request failed. That would make a retry look like a new publication. Likewise, do not use the current timestamp as the only identity: each execution would receive a different one.

A minimal delivery record might look like this:

{
  "contentId": "guide-2026-10-16-01",
  "platform": "example-network",
  "accountId": "brand-account",
  "state": "approved",
  "externalId": null,
  "lastAttemptId": null
}

Enforce uniqueness for that destination identity in the database. Application-level checks alone can race when two workers read the same queue.

Claim work atomically

The worker should change an eligible delivery from approved to claimed in one atomic operation. If another worker has already claimed it, stop processing that delivery. A read followed by an unrelated write leaves a gap in which both workers can believe they own the item.

Keep the claim's timestamp and worker identifier. A crashed worker can leave a claim behind, so a recovery process needs to identify stale claims. However, an expired claim is not itself permission to publish again. The first worker might have reached the remote platform before it crashed.

When recovering an expired claim, move it into reconciliation. Check remote state before creating another public post.

Model uncertainty as its own state

Useful states include approved, claimed, published, rejected and uncertain. Define their transitions explicitly. In particular, do not collapse uncertain into rejected simply to make the retry queue easier to implement.

Examples help distinguish them:

A local validation failure before any network request is a known failure.
A confirmed remote rejection can be recorded as rejected, following that API's documented response semantics.
A timeout after sending a publication request is uncertain.
A remote ID followed by a failed database update means the remote publication succeeded while local recording failed.

That last case deserves special handling. Record the external ID in durable logs or another recovery store before acknowledging the job. Do not issue a second publication request just because the final database write needs retrying.

Reconcile using the strongest available evidence

If the platform provides an idempotency key for this exact operation, reuse a stable key according to its documented scope and retention rules. Do not assume an arbitrary request header creates idempotency.

If you have an external ID, query that resource directly. Otherwise, use an authenticated listing for the correct account and compare the intended publication with remote records. A stable marker can help where supported. Exact text alone is weaker evidence: legitimate posts can share wording, and platforms can transform URLs or formatting.

Read all pages needed to cover the relevant history. If pagination stops unexpectedly, authentication fails or a safety limit is reached, preserve the uncertain state. An incomplete search cannot establish absence. It is safer to surface that delivery for investigation than to silently duplicate it.

Retry the failed stage

Structure the implementation around stages rather than wrapping the entire worker in a broad retry loop:

async function recover(delivery, remote, store) {
  if (delivery.externalId) {
    const post = await remote.get(delivery.externalId);
    return store.confirmPublished(delivery.id, post.id);
  }

  const result = await remote.findDelivery(delivery);
  if (!result.complete) return store.markUncertain(delivery.id);
  if (result.match) {
    return store.confirmPublished(delivery.id, result.match.id);
  }

  return store.markForReviewedRetry(delivery.id);
}

The example deliberately does not publish. It resolves what is known and hands a confirmed absence to the retry policy. A production adapter must define what a complete search means and how it handles eventual visibility of newly created posts.

Use bounded backoff for transient read failures. Avoid repeatedly querying a platform in a tight loop. Authentication problems need credential recovery, not more publication attempts.

Test the failure between each pair of stages

The most useful tests interrupt the worker at meaningful boundaries: after the claim, after the remote write, and before local confirmation. Simulate a remote service that creates the post but throws a timeout to the caller. Recovery should discover the existing publication and avoid a second write.

Also test concurrent claims, multi-page history, an incomplete history response and a failed local confirmation. Count remote writes in the test, not just successful worker exits. A worker can exit successfully after publishing nothing, or fail after publishing something; neither exit code alone tells the full story.

Report publication separately from job health

For each run, report selected, published, rejected, uncertain and skipped counts. Include the external ID and public URL when available. A green scheduler badge means the job completed according to its exit policy. It should never be presented as proof that a new post exists.

Keep credentials out of logs, and retain enough non-secret delivery metadata to reconcile an interrupted attempt. This approach makes recovery a controlled state transition instead of a guess. It also makes a quiet day explainable: an empty reviewed queue is a content-supply issue, while an uncertain publication is a delivery issue. Each needs a different fix.

API Authentication Across Multiple Platforms: A Practical Implementation Guide

Daniel Neata — Thu, 15 Oct 2026 14:16:53 +0000

API Authentication Across Multiple Platforms: A Practical Implementation Guide

Publishing to several platforms from one GitHub Actions repository sounds simple: store tokens, call APIs, and schedule a job. The difficult part is not sending the request. It is proving that a post was created exactly once, recovering when authentication expires, and keeping credentials out of logs. This guide walks through a small Node.js publisher design suitable for Bluesky, Blogger, and other OAuth-backed APIs. The examples are illustrative; endpoint contracts differ by provider.

Separate credentials by destination

Treat every platform as its own security boundary. A Bluesky app password is not an OAuth refresh token, and a Blogger refresh token cannot be handled as a short-lived access token. Use individual GitHub Actions secrets for each credential. Give each token the minimum scopes needed to create posts. Do not embed tokens in article content, URLs, build artifacts, or debug output.

Use environment variables only at the step that needs them. Avoid printing the complete environment. If an API returns an authentication error, log the HTTP status and a redacted provider error code rather than the raw response body, which can occasionally contain sensitive values.

Build one timeout-aware request helper

Network requests can hang or fail after the remote service has accepted a write. Always set a timeout, distinguish read operations from writes, and do not automatically retry a timed-out create request. Node.js 20+ provides AbortSignal.timeout, which keeps the example dependency-free.


async function apiRequest(url, { method = 'GET', token, body } = {}) {

  const response = await fetch(url, {

    method,

    headers: {

      Authorization: `Bearer ${token}`,

      ...(body ? { 'Content-Type': 'application/json' } : {}),

    },

    body: body ? JSON.stringify(body) : undefined,

    signal: AbortSignal.timeout(15_000),

  });

  if (response.status === 401) throw new Error('AUTH_REQUIRED');

  if (response.status === 429) throw new Error('RATE_LIMITED');

  if (!response.ok) throw new Error(`API_HTTP_${response.status}`);

  return response.json();

}

This helper deliberately avoids logging the request URL because some legacy APIs accept credentials in query parameters. Prefer Authorization headers whenever supported. Rate-limit responses should be handled according to the provider's Retry-After guidance, with a bounded retry policy for safe reads.

Keep authentication lifecycle separate from publication

For OAuth providers, access tokens typically expire before refresh tokens. A refresh operation is a credential-management step, not a reason to retry a post blindly. Refresh securely, then inspect remote publication history before deciding whether a previous attempt needs to be repeated. For app-password systems, create a session for each short-lived job and discard the session afterward.

Record the destination, account identifier, and token expiry where appropriate, but never the token itself. If authentication fails, stop the workflow with a nonzero exit code. A green workflow that silently skipped a broken publisher is a misleading operational signal.

Make the queue claim atomic

Two GitHub scheduled jobs may overlap because cron runs can be delayed. The database should offer an atomic transition from approved to claimed, so that only one worker owns a post. A plain GET followed by POST is not sufficient protection against a race condition. If the queue is empty, an optional curated fallback may run, but only after it confirms the intended account and scans for previously published content.

Design publication as a state machine

Use explicit states: approved, claimed, published, and failed. Save the remote ID and URL after the provider confirms creation. A remote ID is stronger evidence than a successful process exit; a publicly accessible post URL is stronger still. A network timeout during a create request is an uncertain outcome, not proof of failure.


async function publishClaimed(item, provider) {

  const known = await provider.findByFingerprint(item.fingerprint);

  if (known) return { status: 'published', remoteId: known.id };

  try {

    const created = await provider.create(item);

    if (!created?.id) throw new Error('NO_REMOTE_ID');

    return { status: 'published', remoteId: created.id };

  } catch (error) {

    // A write may have succeeded before the connection failed.

    if (error.name === 'TimeoutError' || error.name === 'AbortError') {

      return { status: 'uncertain', reason: 'CHECK_REMOTE_BEFORE_RETRY' };

    }

    throw error;

  }

}

The provider adapter must implement findByFingerprint using the platform's actual read API. This example is a design pattern, not a drop-in implementation: some providers do not support a fingerprint query. In those cases, scan the account's recent posts, store a durable local publication ledger, or use an idempotency key if the API supports one.

Handle pagination and duplicate detection honestly

Fetching the first 100 posts is not a complete history check for a long-lived account. Follow pagination until you have covered the entire relevant window, or stop publishing if you cannot establish that a candidate is unused. A fixed list of ten fallback messages will eventually be exhausted. Treat exhaustion as a normal no-op and generate new editorially reviewed content separately; do not silently cycle the same copy.

Use a stable fingerprint derived from normalized text and destination URL. Preserve the original text for exact comparisons when platforms expose it. A cryptographic hash can identify your own content, but a remote API cannot search by that hash unless you also store a ledger or add an allowed marker.

Configure GitHub Actions to fail visibly

Run syntax checks and dry-run authentication on code changes. Reserve live publication for a scheduled or explicitly authorized job. GitHub cron is best-effort: it can start late, so evaluate publishing windows in the intended timezone and account for daylight saving time. Avoid broad windows that let multiple delayed runs publish the same fallback post without a shared lock.


name: Safe publisher

on:

  workflow_dispatch:

    inputs:

      dry_run:

        type: boolean

        default: true

jobs:

  check:

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4

        with:

          node-version: '24'

      - run: node --check publisher.js

      - name: Run publisher

        env:

          PLATFORM_TOKEN: ${{ secrets.PLATFORM_TOKEN }}

          DRY_RUN: ${{ inputs.dry_run }}

        run: node publisher.js

Never interpret a successful syntax check or dry run as a published post. For live runs, emit structured, non-sensitive fields such as platform, slot, claimed item ID, remote post ID, public URL, and final outcome. The monitoring process should flag missing IDs and repeated empty queues without inventing publication success.

Test the failure paths before enabling automation

Test with an invalid token, an empty queue, a 429 response, a delayed cron execution, and a simulated timeout immediately after a write. Also test the case where the remote post exists but the local database update fails. In that case, reconcile the remote ID instead of posting again. Keep dry runs side-effect-free, including avoiding state changes that would consume queued content.

A practical operating checklist

Before enabling a new destination, verify the account identity, permissions, token rotation process, duplicate detection, timeout behavior, scheduled timezone, and evidence logs. Confirm that an error produces a failed job and that a genuinely empty queue produces an explicit no-op. Review published links periodically, because an API response can succeed even if a platform later moderates or removes content.

Conclusion

Reliable multi-platform publishing depends on state, not just schedules. Keep secrets isolated, make queue claims atomic, reconcile uncertain writes, and require remote evidence before calling a publication successful. Those controls are less glamorous than adding another social network, but they are what make unattended automation maintainable.

Source: https://github.com/danielmason77/danielmason-threads-publisher/blob/main/content-library/dev-community/api-authentication-across-multiple-platforms-a-practical-implementation-guide.md