DEV Community

Roberto Luna
Roberto Luna

Posted on

Automating Multi‑Channel Content Publishing with a Git‑Driven Pipeline

Automating Multi‑Channel Content Publishing with a Git‑Driven Pipeline

TL;DR: I built a Git‑hook‑triggered pipeline that reads markdown assets, updates metadata.json, and pushes the same article to Medium, Substack, Bluesky, and Dev.to. The change eliminates manual copy‑pastes and rate‑limit crashes by centralizing content generation in Docker.


The Problem

Our VibeCoding blog lives on several platforms. Every time I wrote a post, I had to:

  1. Copy the markdown into Medium’s editor, fix front‑matter, and hit publish.
  2. Do the same for Substack, then Bluesky, then Dev.to.

On October 17 2026 the Slack bot that auto‑generates these drafts hit a rate‑limit panic (429 Too Many Requests) while trying to post to Medium, and the Docker build that bundles the publishing scripts failed with merge‑conflict errors in package-lock.json. The result was a silent bot and a backlog of unpublished content.

The symptoms were:

Error: request to https://api.medium.com/v1/users/.../posts failed, status code 429
Docker build failed: conflict in package-lock.json at line 1123
Enter fullscreen mode Exit fullscreen mode

We needed a reliable, single source of truth for content and a way to push it atomically to all destinations.


What I Tried First

My first attempt was a bash script (publish.sh) that iterated over a hard‑coded list of platforms:

#!/usr/bin/env bash
for platform in medium substack devto; do
  node publish-${platform}.js ./content/$DATE/$POST.md
done
Enter fullscreen mode Exit fullscreen mode

Problems:

  • The script ran sequentially, so a single 429 error halted the whole pipeline.
  • Platform‑specific metadata (tags, canonical URLs) was duplicated across scripts, leading to drift.
  • The script lived outside Docker, so local environment mismatches caused the package-lock.json merge conflict during CI builds.

The approach failed fast, confirming that a more robust, declarative system was required.


The Implementation

1. File Layout

content/
└─ 2026/10/18/
   ├─ VS/
   │   ├─ changelog.md
   │   ├─ medium_en.md
   │   ├─ medium_es.md
   │   ├─ substack_en.md
   │   ├─ substack_es.md
   │   ├─ bluesky_en.json
   │   └─ metadata.json
   └─ … (other projects)
scripts/
├─ generate-index.ts
├─ publish.ts
└─ utils/
    └─ rateLimiter.ts
Dockerfile
Enter fullscreen mode Exit fullscreen mode

All platform‑specific markdown files sit next to a metadata.json that tracks generation status:

{
  "title": "Next.js Minor Version Bump Felt Like a Roadblock",
  "date": "2026-10-18",
  "platforms": {
    "medium": true,
    "substack": true,
    "bluesky": false,
    "devto": false
  },
  "pull_requests": 0,
  "releases": 0,
  "closed_issues": 0,
  "medium_generated": true,
  "substack_generated": true
}
Enter fullscreen mode Exit fullscreen mode

(Diff snippet from the commit – note the flag changes)

@@ -14,8 +14,8 @@
   "pull_requests": 0,
   "releases": 0,
   "closed_issues": 0,
-  "medium_generated": false,
-  "substack_generated": false,
+  "medium_generated": true,
+  "substack_generated": true,
Enter fullscreen mode Exit fullscreen mode

2. Docker‑Encapsulated Runner

The pipeline runs inside a Docker container to guarantee identical Node versions and lockfile integrity.

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run build   # TypeScript compilation

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY package*.json ./
RUN npm ci --production

ENTRYPOINT ["node", "dist/publish.js"]
Enter fullscreen mode Exit fullscreen mode

The Dockerfile ensures the package-lock.json is generated once (npm ci) and never conflicts during CI runs.

3. Central Publishing Engine (publish.ts)

import { readFileSync, writeFileSync } from 'fs';
import { join } from 'path';
import { postToMedium } from './utils/medium';
import { postToSubstack } from './utils/substack';
import { postToBluesky } from './utils/bluesky';
import { postToDevto } from './utils/devto';
import { rateLimit } from './utils/rateLimiter';

interface Meta {
  title: string;
  date: string;
  platforms: Record<string, boolean>;
  [key: string]: any;
}

async function publish(postDir: string) {
  const metaPath = join(postDir, 'metadata.json');
  const meta: Meta = JSON.parse(readFileSync(metaPath, 'utf8'));

  const tasks = [
    { name: 'medium', fn: postToMedium },
    { name: 'substack', fn: postToSubstack },
    { name: 'bluesky', fn: postToBluesky },
    { name: 'devto', fn: postToDevto },
  ];

  for (const { name, fn } of tasks) {
    if (meta.platforms[name]) {
      console.log(`✅ ${name} already published`);
      continue;
    }

    try {
      const content = readFileSync(
        join(postDir, `${name}_en.md`),
        'utf8'
      );
      await rateLimit(name, async () => fn(content, meta));
      meta.platforms[name] = true;
      console.log(`🚀 Published to ${name}`);
    } catch (err) {
      console.error(`❌ Failed ${name}:`, (err as Error).message);
      // Do not abort; continue with other platforms
    }
  }

  writeFileSync(metaPath, JSON.stringify(meta, null, 2));
}

// Entry point – receives the directory via env var set by CI
const postDir = process.env.POST_DIR!;
publish(postDir).catch(console.error);
Enter fullscreen mode Exit fullscreen mode

Key points:

  • Rate limiting is abstracted in rateLimiter.ts, which uses an exponential back‑off strategy per platform.
  • The script reads the per‑platform markdown (medium_en.md, substack_en.md, etc.) and posts them via dedicated helpers.
  • After a successful post, the metadata.json flag is flipped, making the operation idempotent.

4. Rate Limiter (rateLimiter.ts)


ts
const backoff = new Map<string, number>();

export async function rateLimit(
  platform: string,
  fn: () => Promise<void>
) {
  const delay = backoff.get(platform) ?? 0;
  if (delay) await new Promise(r => setTimeout(r, delay));

  try {
    await fn();
    backoff.set(platform, 0); // reset on success
  } catch (e) {
    const err = e as any;
    if (err.status === 429) {
      const newDelay = (backoff.get(platform) ?? 1000) * 2;
      backoff.set(platform, newDelay);
      console.warn(`⏳ ${platform} rate‑limited, retrying in ${new

---

*Part of my [Build in Public](https://dev.to/zaerohell) series — sharing the real process of building SaaS projects from Playa del Carmen, México.*

*Repo: `zaerohell/content-automation` · 2026-10-19*

\#playadev #buildinpublic
Enter fullscreen mode Exit fullscreen mode

Top comments (0)