Last month I built a small pipeline that takes one draft and publishes it to five platforms with per-site formatting. The idea was to write once, publish everywhere. What I got instead was three distinct failure modes that only showed up when the volume scaled.
Here's what broke, in order, and what fixed each one.
1. The shared-draft collision
I used one JSON file as the handoff between the writer and the publisher. Two publisher tasks ran in parallel, both reading the same file. One finished, the other overwrote it — and the second platform got the first platform's draft.
The fix was boring and correct: drop the shared file entirely. Each draft now lives in a database row with its own id, and the publisher reads by id. One writer, one row, no shared mutable state.
2. Markdown that isn't Markdown
Each platform parses Markdown slightly differently. Code fences that rendered fine on one site got mangled on another — inline code lost its backticks, headings collapsed, a numbered list became a paragraph.
The fix wasn't a universal formatter. It was a per-platform template: the same body, but headers, fences, and tags rewritten to each site's dialect before send. Formatting is now a field on the mechanic, not an assumption in the code.
3. Rate limits that only fire at cadence
Single posts sailed through. Publishing two or three a week to the same platform was fine — until I hit the same platform twice in one hour. Then the second post silently throttled, no error, just a post that never surfaced.
The fix was a scheduler with a real cadence: each pair gets a minimum gap, and no two posts share an hour. Slowing the loop down fixed more than any retry logic ever did.
The three lessons compress to one rule: never let two parts of the pipeline share mutable state, and never let the scheduler run faster than the platform's patience. Everything else is just plumbing.
If you're building something similar, I'd start with the database row and the per-platform template — the scheduler you can tune later.
Top comments (0)