Adding JSON‑Schema Validation & Unicode Normalization to My Daily Publishing Bot

TL;DR: I added a JSON‑schema validator and a Unicode‑normalization step to the content‑automation pipeline, which stopped the bot from crashing on “ñ” characters and other non‑ASCII symbols. The change is isolated in a few helper modules and a tiny schema file, but it now protects every auto‑generated payload (Medium, Substack, Bluesky).

---

The Problem

My publishing bot runs every morning, reads the markdown files under content/2026/11/04/content-automation/, converts them to the JSON payloads required by each platform, and pushes them via the respective APIs. On 2026‑11‑04 the bot threw an uncaught exception:

UnicodeEncodeError: 'utf-8' codec can't encode character '\xf1' in position 124: invalid start byte

The offending character was the “ñ” in the Spanish title “Una “ñ” que paralizó mi bot de publicación diaria”. Because the pipeline streamed the raw markdown into a JSON string without normalizing the Unicode, the JSON serializer emitted an invalid UTF‑8 sequence, causing the HTTP request to fail for all platforms. The bot stopped before generating any payload, leaving the day’s posts unpublished.

---

What I Tried First

My first instinct was to patch the failing line in src/publish.js:

const payload = JSON.stringify(markdownContent);

I wrapped it in a try/catch and forced a .replace(/ñ/g, 'n'). That silenced the error, but it was a hack that broke the Spanish copy and didn’t protect against other Unicode characters (accents, emojis, etc.). I also tried setting the Node process flag --encoding=utf8, but the underlying issue was still the raw markdown being fed directly into JSON.stringify.

---

The Implementation

1. Define a JSON schema for each platform

I created a tiny schema file schemas/payload.schema.json that describes the minimal required fields for every platform:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "PlatformPayload",
  "type": "object",
  "required": ["type", "text"],
  "properties": {
    "type": { "type": "string", "enum": ["progress", "avance"] },
    "text": { "type": "string", "minLength": 1 }
  },
  "additionalProperties": false
}

The schema lives next to the other content files, making it easy to version alongside the markdown.

2. Add a validation helper

In src/utils/validate.js I introduced a small wrapper around the popular ajv library:

// src/utils/validate.js
const Ajv = require('ajv');
const schema = require('../../schemas/payload.schema.json');

const ajv = new Ajv({ allErrors: true });

function validatePayload(payload) {
  const valid = ajv.validate(schema, payload);
  if (!valid) {
    const errors = ajv.errors.map(e => `${e.dataPath} ${e.message}`).join('; ');
    throw new Error(`Payload validation failed: ${errors}`);
  }
  return payload;
}

module.exports = { validatePayload };

Now every platform module must call validatePayload before sending the request.

3. Normalize Unicode before serialization

I added a utility src/utils/normalize.js that uses Node’s built‑in String.prototype.normalize:

// src/utils/normalize.js
function normalizeText(text) {
  // NFC is the most common form for web content
  return text.normalize('NFC');
}

module.exports = { normalizeText };

4. Refactor the platform generators

Each generator now imports the two helpers. Below is the updated snippet for the Bluesky payload generator (src/platforms/bluesky.js), which also shows the diff from the commit:

--- a/src/platforms/bluesky.js
+++ b/src/platforms/bluesky.js
@@ -1,7 +1,9 @@
 const fetch = require('node-fetch');
+const { validatePayload } = require('../utils/validate');
+const { normalizeText } = require('../utils/normalize');

 function buildBlueskyPayload(md) {
-  const payload = {
-    type: "progress",
-    text: md.title + "\n\n" + md.body
-  };
+  const raw = {
+    type: "progress",
+    text: `${md.title}\n\n${md.body}`
+  };
+
+  // Normalize Unicode before validation/serialization
+  const normalized = {
+    ...raw,
+    text: normalizeText(raw.text)
+  };
+
+  // Validate against schema – will throw if something is wrong
+  return validatePayload(normalized);
 }

 async function postToBluesky(payload) {
   const response = await fetch('https://api.bluesky.social/v1/post', {

The same pattern was applied to src/platforms/medium.js and src/platforms/substack.js. The only difference is the type enum ("avance" for Spanish posts).

5. Update the automation script

The main script (scripts/generate.js) now reads the markdown files, builds the payloads, and writes the JSON files used by the CI step. The diff shows the new write logic:

--- a/scripts/generate.js
+++ b/scripts/generate.js
@@ -12,6 +12,9 @@
 const path = require('path');
 const fs = require('fs');
+const { validatePayload } = require('../src/utils/validate');
+const { normalizeText } = require('../src/utils/normalize');

 function generatePayloads() {
   const contentDir = path.join(__dirname, '../content/2026/11/04/content-automation');
@@ -45,12 +48,20 @@
     const md = parseMarkdown(fileContent);
     const bluesky = buildBlueskyPayload(md);
-    const outPath = path.join(contentDir, `bluesky_${lang}.json`);
-    fs.writeFileSync(outPath, JSON.stringify(bluesky, null, 2));
+    // Normalize and validate before persisting
+    const normalized = {
+      ...bluesky,
+      text: normalizeText(bluesky.text)
+    };
+    validatePayload(normalized);
+    const outPath = path.join(contentDir, `bluesky_${lang}.json`);
+    fs.writeFileSync(outPath, JSON.stringify(normalized, null, 2));
   });
 }

6. Metadata flag updates

The metadata.json file now reflects the new generation state. The diff from the commit shows the change:

--- a/content/2026/11/04/content-automation/metadata.json
+++ b/content/2026/11/04/content-automation/metadata.json
@@ -12,8 +12,8 @@
   "pull_requests": 0,
   "releases": 0,
   "closed_issues": 0,
-  "medium_generated": false,
-  "substack_generated": false,
+  "medium_generated": true,
+  "substack_generated": true,
   "bluesky_generated": true,
   "last_generated": "2026-11-05T08:12:00Z"

Now the CI pipeline can check these flags to decide whether to push or retry.

---

Key Takeaway

Never

---

Part of my Build in Public series — sharing the real process of building SaaS projects from Playa del Carmen, México.

Repo: zaerohell/content-automation · 2026-11-05

#playadev #buildinpublic