Why a Batch Publisher Should Isolate Article Failures

Why this matters

A batch job has two audiences.

Each item needs an independent result: one malformed article should not prevent
an unrelated article from being prepared. The scheduler or CI job needs an
honest aggregate result: if any item failed, the batch must not report success.

The naive implementations satisfy only one side:

for (const article of articles) {
  await publish(article); // first rejection aborts the loop
}

This is fail-fast. It is simple, but one bad file blocks every later file.

The opposite mistake catches everything and exits successfully:

for (const article of articles) {
  try {
    await publish(article);
  } catch {
    // keep going
  }
}

This keeps the batch moving but lies to automation. CI turns green even though
some work was lost.

The useful middle is collect and continue: isolate each item, retain its
failure, process the rest, and return a non-zero final result.

What I built or tested

I traced a TypeScript CLI with a publish-all command and exercised it using a
no-network fixture:

• two valid Markdown articles;
• one Markdown article missing its required title;
• one non-Markdown file; and
• one valid article inside a nested directory.

The observed result was:

{
  "exitStatus": 1,
  "completed": ["batch-valid-one", "batch-valid-two"],
  "failedArticles": 1,
  "aggregateFailure": "1 article(s) failed",
  "ignoredNonMarkdown": true
}

Both valid articles completed even though the invalid article failed. The
non-Markdown file was ignored. After every discovered Markdown file had been
considered, the command exited with status 1.

The experiment ran with --dry-run --platform devto. It used in-memory
publication state and preview storage, so it made no platform or asset-storage
write.

Setup

The project requires Node.js 22 or later and uses a TypeScript CLI. The relevant
command shape is:

blog-publisher publish-all ./ready-articles \
  --dry-run \
  --platform devto

Use a deliberately scoped input directory. The discovery function recursively
accepts every filename ending in .md; it does not know that README.md,
private experiment notes, an authoring source.md, and a rendered index.md
have different roles.

A safe input boundary might look like this:

ready-articles/
├── article-a.md
├── article-b.md
└── release-3/
    └── article-c.md

Do not point the command at a repository root or an article workspace
containing multiple Markdown artifacts unless every matching file is truly a
publishable input.

The experiment used dry-run intentionally. Testing failure isolation does not
require three real publication attempts.

Step-by-step walkthrough

1. Discover candidates recursively

The CLI walks a directory using readdir() with directory entries. A directory
recurses; a file is included only when its name ends with .md:

async function articleFiles(directory: string): Promise<string[]> {
  const entries = await readdir(directory, { withFileTypes: true });
  const nested = await Promise.all(
    entries.map(async (entry) => {
      const entryPath = path.join(directory, entry.name);
      if (entry.isDirectory()) return articleFiles(entryPath);
      return entry.isFile() && entry.name.endsWith(".md")
        ? [entryPath]
        : [];
    }),
  );
  return nested.flat();
}

This explains two experiment results: the nested valid article was found, and
the .txt fixture was ignored.

It also exposes a constraint. The implementation does not filter by a canonical
filename such as index.md, does not exclude private directories, and does not
explicitly sort the resulting paths. Treat the directory contents as the
manifest; do not make ordering a business dependency.

2. Await one article at a time

The batch loop is sequential:

const failures: string[] = [];

for (const article of files) {
  try {
    const output = await publishArticle(article, config, options);
    logger.info({ article, slug: output.slug }, "article completed");
  } catch (error) {
    failures.push(article);
    logger.error({ err: error, article }, "article failed");
  }
}

The await inside for...of matters. The next article does not begin until
the current promise settles.

Sequential execution is not a throughput claim. It is a conservative default
for a workflow with shared credentials, platform rate limits, image state, and
publication records. Parallelism may be appropriate later, but only after those
shared boundaries have explicit concurrency guarantees.

3. Catch at the item boundary

The try block covers one call to publishArticle, not the whole loop. That is
the isolation boundary.

In the experiment, the invalid fixture failed front matter validation because
it lacked a title. The catch block recorded that file and returned control to
the loop. The later valid article still ran.

This is different from hiding the error. The logger retains the original error
beside the article path, and the path is added to the failure collection.

4. Fail the batch after processing the remainder

After the loop:

if (failures.length) {
  throw new Error(`${failures.length} article(s) failed`);
}

The command's top-level error handler sets a non-zero process exit code. CI
therefore receives a failed job, even though successful items were not
discarded.

The full control flow is:

Per-item isolation preserves progress; the final branch preserves batch
truthfulness.

What went wrong

The intentionally malformed file demonstrated the desired item isolation, but
source inspection revealed three broader limits.

First, the final thrown error contains only a count. The detailed per-item
errors exist in logs, while the in-memory failures array stores only paths.
A machine that receives only the process result knows that one item failed but
does not receive a structured outcome manifest.

Second, the batch has no durable checkpoint. If the process crashes halfway
through, publish-all itself cannot resume at item four. A later invocation
rediscovers the directory. The lower publication layer retains platform
records, which helps individual create/update safety, but that is not a batch
run ledger.

Third, recursive “all Markdown” discovery is broader than many article
repositories expect. A generated article workspace can contain:

source.md
index.md
published.md
.work/experiment.md

All four names end in .md. Pointing a generic recursive batch at that
workspace can attempt multiple representations of the same article and private
notes that were never intended for publication. Failure isolation limits the
blast radius of bad input; it does not make input selection correct.

Fix or mitigation

Keep the simple loop for small, carefully scoped batches, but strengthen its
contract.

Return item-level outcomes

Represent every attempted item explicitly:

type BatchOutcome =
  | { article: string; status: "succeeded"; slug: string }
  | { article: string; status: "failed"; error: string }
  | { article: string; status: "unknown"; error: string };

const outcomes: BatchOutcome[] = [];

for (const article of files) {
  try {
    const output = await publishArticle(article, config, options);
    outcomes.push({ article, status: "succeeded", slug: output.slug });
  } catch (error) {
    outcomes.push({
      article,
      status: "failed",
      error: error instanceof Error ? error.message : String(error),
    });
  }
}

await writeBatchReport(outcomes);

if (outcomes.some((item) => item.status !== "succeeded")) {
  process.exitCode = 1;
}

Persisting the report makes the partial result usable by CI, operators, and a
future resume command.

Do not flatten an ambiguous remote write into ordinary failed. If an
individual publisher cannot prove whether a create succeeded, retain
unknown and require reconciliation before another create attempt.

Make discovery a contract

Choose one of these input policies:

• accept an explicit manifest of article paths;
• include only a canonical filename such as index.md;
• exclude private and generated directories by rule; or
• copy approved inputs into a clean staging directory.

An explicit manifest is the strongest option when order, review, or per-item
metadata matters.

Add a focused regression test

The current repository has no focused publish-all test. A regression test
should inject or spawn a publisher with three items:

1. success;
2. failure;
3. success.

Assert that all three were attempted in sequence, both successes were retained,
and the aggregate result was non-zero. Add separate cases for an empty
directory, nested discovery, and excluded files.

Trade-offs

• Sequential processing is easy to reason about and gentler on shared systems, but total duration grows with every article.
• Continuing after failure preserves independent progress, but the batch is no longer atomic. Some items may be public while the final command fails.
• A structured outcome manifest improves recovery and observability, but it introduces another durable artifact with retention and redaction concerns.
• A broad recursive scanner is convenient for simple staging directories but dangerous in repositories containing multiple Markdown representations.
• An explicit manifest adds preparation work, but it makes the intended set and order reviewable.
• Dry-run testing verifies local control flow without publication risk. It cannot prove remote rate limits, credentials, or delivery.

How I verified it

I used four checks:

1. Source trace: inspected recursive discovery, the awaited for...of loop, the per-item catch, and the aggregate throw.
2. Isolated experiment: spawned the real CLI in DEV.to dry-run mode against two valid articles, one invalid article, a nested directory, and an ignored non-Markdown file.
3. Assertions: required both valid slugs, exactly one recorded failure, exit status 1, and the aggregate failure message.
4. Release gates: ran the article validator, Mermaid render, publisher dry-run, TypeScript check, and complete repository test suite before the authorized public write.

The experiment verified local discovery and error-control semantics. It did
not contact DEV.to, upload an image, benchmark batch speed, or simulate a
process crash.

Conclusion

A resilient batch publisher should not choose between “stop at the first bad
article” and “pretend every article succeeded.”

Catch failures at the item boundary, continue independent work, and return a
non-zero aggregate result. Then make the input set explicit and persist
item-level outcomes when the job must survive restarts.

That pattern is small enough for a CLI and strong enough to provide the two
truths a batch needs: which items completed, and whether the batch as a whole
was clean.

AI assistance disclosure

AI assisted with outlining and drafting. Every implementation claim was
checked against the repository, and the failure-isolation behavior was
verified with a local no-network CLI experiment before publication.