Automating Multi‑Platform Content Publishing: Refactoring JSON Metadata and Platform Flags

TL;DR: I rewrote the metadata handling in the content-automation repo so the daily generation script can reliably track which assets have been published to Medium, Substack, Dev.to, and Bluesky. The change introduces a structured metadata.json schema with language‑scoped URI maps and idempotent flag updates, eliminating JSON‑schema errors and missed publishes.

The Problem

Our nightly CI job generates a set of markdown files (Medium, Substack, Dev.to) under content/YYYY/MM/DD/content-automation/ and then pushes them to the respective platforms. The job also updates a metadata.json file that records which platforms have successfully received the content:

{
  "repo": "content-automation",
  "date": "2026-08-25",
  "medium_generated": false,
  "substack_generated": false,
  "bluesky_published": false,
  "bluesky_uris": {}
}

On 2026‑08‑25 the script crashed with a KeyError: 'bluesky_uris' when trying to append the newly‑published Bluesky URLs. The CI logs showed:

Traceback (most recent call last):
  File "scripts/publish_bluesky.py", line 42, in main
    metadata["bluesky_uris"][lang].append(uri)
KeyError: 'bluesky_uris'

Because the key existed but was an empty object ({}), Python treated it as a dict without the language sub‑keys (en, es). The subsequent append failed, and the job aborted before marking bluesky_published as true. Downstream steps (e.g., the Dev.to post) read the stale metadata.json and skipped publishing, leaving the content in limbo.

What I Tried First

My first attempt was a quick patch in publish_bluesky.py:

if not metadata.get("bluesky_uris"):
    metadata["bluesky_uris"] = {"en": [], "es": []}

I added this guard right before the append. It prevented the KeyError, but it introduced a new problem: the guard overwrote any existing URIs from previous runs, causing duplicate publishes when the job retried. The metadata file ended up looking like:

{
  "bluesky_uris": {
    "en": [],
    "es": []
  }
}

All previously stored URLs vanished, and the platform flags (bluesky_published) were reset to false on each retry. The root cause was that the script rewrote the entire metadata.json after each step instead of merging changes.

The Implementation

1. Define a Stable Schema

I introduced a versioned schema in content/2026/08/24/content-automation/metadata.json. The new file now looks like this (excerpt from the commit diff):

@@ -13,13 +13,15 @@
   "pull_requests": 0,
   "releases": 0,
   "closed_issues": 0,
-  "medium_generated": false,
-  "substack_generated": false,
+  "medium_generated": true,
+  "substack_generated": false,
   "bluesky_published": false,
-  "bluesky_uris": {},
+  "bluesky_uris": {
+    "es": [],
+    "en": []
+  },
   "devto_published": false,
   "devto_uri": ""

Key changes:

• bluesky_uris now contains explicit language keys (en, es) initialized to empty arrays.
• Platform‑specific flags (medium_generated, substack_generated, bluesky_published, devto_published) are persisted as booleans.
• The file lives next to the generated markdown assets, making it a single source of truth per date.

2. Centralize Metadata Operations

I created scripts/metadata_manager.py to abstract loading, updating, and saving the JSON file. This ensures every publishing script follows the same merge logic.

# scripts/metadata_manager.py
import json
from pathlib import Path
from typing import Dict, Any

METADATA_PATH = Path(__file__).parent.parent / "content" / "2026" / "08" / "24" / "content-automation" / "metadata.json"

def load_metadata() -> Dict[str, Any]:
    if not METADATA_PATH.exists():
        raise FileNotFoundError(f"Metadata not found at {METADATA_PATH}")
    with METADATA_PATH.open("r", encoding="utf-8") as f:
        return json.load(f)

def save_metadata(data: Dict[str, Any]) -> None:
    # Write atomically to avoid partial writes on CI failures
    tmp_path = METADATA_PATH.with_suffix(".tmp")
    with tmp_path.open("w", encoding="utf-8") as f:
        json.dump(data, f, indent=2, ensure_ascii=False)
    tmp_path.replace(METADATA_PATH)

def update_flag(flag: str, value: bool) -> None:
    meta = load_metadata()
    meta[flag] = value
    save_metadata(meta)

def append_bluesky_uri(lang: str, uri: str) -> None:
    meta = load_metadata()
    # Ensure language bucket exists
    meta.setdefault("bluesky_uris", {"en": [], "es": []})
    meta["bluesky_uris"].setdefault(lang, [])
    if uri not in meta["bluesky_uris"][lang]:
        meta["bluesky_uris"][lang].append(uri)
    save_metadata(meta)

Why this matters:

• load_metadata raises early if the file is missing, preventing silent failures.

---

Part of my Build in Public series — sharing the real process of building SaaS projects from Playa del Carmen, México.

Repo: zaerohell/content-automation · 2026-08-25

#playadev #buildinpublic