What I learned adding diagram and chart slides to a CI-rendered YouTube pipeline

The conclusion first: pre-rendering diagrams and charts to PNG before compositing them onto slides — rather than generating visual content inline or inside ffmpeg — is the right architecture for a CI video pipeline. The tooling gap between Chromium-backed Mermaid rendering, headless matplotlib, and ffmpeg's static frame expectation makes a shared PNG handoff the only approach that keeps each piece testable and replaceable.

I added three new slide types to the YouTube slide renderer last week: diagram (Mermaid flowcharts and sequence diagrams), chart (branded horizontal bar charts via matplotlib), and image (license-clear photos from Openverse). The existing slides — title, bullets, table, tool, outro — all draw directly with Pillow. These three render externally, produce a PNG, and get pasted into the same Pillow canvas. Same output contract, different render path.

Why pre-render instead of embed

The two-host pipeline assembles video by compositing a still image for each dialogue segment, synthesizing audio with edge-tts, and using ffmpeg to concatenate the clips. ffmpeg expects the still to be a file or a stream of identical frames — it does not run JavaScript, and it cannot call a browser mid-concat.

Mermaid runs through Puppeteer and Chromium. Pillow draws directly on a numpy-backed image. There is no in-process way to make these talk. The only clean option is: mmdc produces a PNG, Pillow pastes the PNG.

matplotlib is different — it could theoretically produce an image buffer in the same process. But having a consistent "render to PNG file, paste PNG file" pattern for all three visual types means they all take the same shape: render, paste, and degrade to a still-usable slide if the renderer is missing. One pattern is better than two.

Mermaid via mmdc: the CI-specific configuration

render_mermaid() in visuals.py writes two config files before calling mmdc:

cfg = os.path.join(tempfile.gettempdir(), "bs-mmd-theme.json")
with open(cfg, "w", encoding="utf-8") as f:
    json.dump(_MMD_THEME, f)

pcfg = os.path.join(tempfile.gettempdir(), "bs-puppeteer.json")
with open(pcfg, "w", encoding="utf-8") as f:
    json.dump({"args": ["--no-sandbox", "--disable-setuid-sandbox"]}, f)

The puppeteer config is the one that bit me first. Chromium refuses to start as root without --no-sandbox, and GitHub Actions runs as root inside the Ubuntu container. Without --disable-setuid-sandbox, it will also fail on containers where setuid is restricted. Both flags are needed.

The theme config uses Mermaid's base theme, not dark or forest. The other named themes override themeVariables, so color injection does not work reliably with them. Only base respects the custom palette (confirmed in the Mermaid.js theme docs):

_MMD_THEME = {
    "theme": "base",
    "themeVariables": {
        "primaryColor": PANEL, "primaryTextColor": INK, "primaryBorderColor": ACCENT,
        "lineColor": MUTED, "secondaryColor": "#1B2B4A", "tertiaryColor": BG,
        "fontFamily": "Arial", "fontSize": "22px",
        "clusterBkg": PANEL, "clusterBorder": ACCENT,
    },
}

The mmdc binary resolution tries three paths:

def _mmdc_cmd() -> list[str]:
    local = os.path.join(_DIR, "node_modules", ".bin", "mmdc")
    if os.path.isfile(local):
        return [local]
    if shutil.which("mmdc"):
        return ["mmdc"]
    return ["npx", "--yes", "@mermaid-js/mermaid-cli"]

In CI the middle branch is the one that fires: the GitHub Actions workflow runs npm install -g @mermaid-js/mermaid-cli, so mmdc is on PATH and the local node_modules check misses. The first branch only fires if mermaid-cli has been installed explicitly under that directory (npm install --prefix scripts/yt-longform @mermaid-js/mermaid-cli) — there is no package.json there, so a bare npm install in that folder installs nothing. The npx branch is the last resort for a machine with neither. Do not make npx the primary path — it downloads the package on every invocation, which adds 20-30 seconds per diagram slide in a cold runner.

The -w 1600 width flag matters. At 1920x1080, the content area after chrome and heading is roughly 1700x700. Rendering at 1600px wide gives mmdc enough resolution to produce readable text without scaling artifacts when _paste_visual() thumbnails it into the slot.

matplotlib horizontal bars with a custom dark palette

render_chart() takes a simple spec:

# spec = {"items": [["Tool A", 41200], ["Tool B", 28900], ...], "unit": "stars", "highlight": 0}

Three things that tripped me up when trying to match the slide background:

First: matplotlib.use("Agg") must be called before import matplotlib.pyplot as plt. In Python, the backend selection call and the pyplot import are order-dependent — if you call use("Agg") after pyplot is imported, it either silently fails or raises. The function imports matplotlib inside the function body to avoid this at module load time:

def render_chart(spec, out_png):
    import matplotlib
    matplotlib.use("Agg")
    import matplotlib.pyplot as plt

Second: setting the background requires two separate calls:

fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)

fig.patch is the outer figure background; ax is the axes area. Missing either one leaves a white rectangle where the background should be dark navy.

Third: plt.tight_layout(pad=1.2) is not enough on its own. Adding bbox_inches="tight" to fig.savefig() is required to clip the white padding matplotlib adds around the bounding box by default. Without it, the saved PNG has a white border that composites badly onto the dark slide background.

The highlight index accentuates one bar in ACCENT2 (green) instead of ACCENT (blue). The spec author sets it to mark the bar that is the point of the slide — the tool with the most stars, or the benchmark winner. It is optional; when absent, all bars render in blue.

The _visual_slide scaffold and graceful degradation

The diagram and chart types share one scaffold — slide_image has its own copy of it, for reasons I get to below:

def _visual_slide(spec, render_fn):
    img, d = _base()
    _chrome(img, d, spec.get("page", ""))
    heading = spec.get("heading", "")
    if heading:
        d.text((MARGIN, 150), heading, font=font(56, "bold"), fill=INK)
        d.rectangle([MARGIN, 228, MARGIN + 120, 236], fill=ACCENT)
    import tempfile
    fd, tmp = tempfile.mkstemp(suffix=".png")
    os.close(fd)
    try:
        render_fn(tmp)
        _paste_visual(img, tmp, top=270 if heading else 150)
    except Exception as e:
        sys.stderr.write(f"WARN: visual render failed ({e})\n")
        d.text((MARGIN, 420), "[visual unavailable]", font=font(40), fill=MUTED)
    finally:
        if os.path.exists(tmp):
            os.unlink(tmp)
    return img

The tempfile.mkstemp pattern (create and close descriptor separately) is deliberately cross-platform: on Windows, NamedTemporaryFile with delete=False sometimes holds a lock that prevents a subprocess from writing to the same path. mkstemp avoids this. On Linux it makes no practical difference, but the code runs locally on macOS too.

The except Exception fallback is intentional. If mmdc is not installed or matplotlib is not present, the slide renders as a clean heading card with a muted "[visual unavailable]" placeholder. slide_image is the one that does not share this scaffold, and that is exactly why: when Openverse returns no usable image there is nothing to caption or credit, so its except branch leaves the branded heading slide as-is rather than stamping placeholder text on it. Either way the video build continues. A missing diagram is not a build failure.

This matches the design of the rest of the pipeline: the CI job should produce a video even when external dependencies are partially absent. A video with a placeholder slide is reviewable; a failed build is not.

Where the approach has limits

mmdc startup cost. Each mmdc call launches a Chromium process, renders the SVG, and exits. That takes 2-4 seconds per diagram on the GitHub Actions ubuntu-latest runner. A video spec with five diagram slides adds 10-20 seconds to the build. For the current video lengths (15-25 segments), this is acceptable. If the format grew to 50+ slides, pre-rendering all diagrams in parallel before the main loop would matter.

matplotlib version pinning. The chart code relies on a specific call signature for barh() and tight_layout(). matplotlib has changed these interfaces across minor versions. The workflow currently installs matplotlib unpinned, so a runner image update can move the version under me — pinning it is on the list.

Mermaid syntax drift. mmdc's supported Mermaid syntax depends on the installed package version. Sequence diagrams work. gitGraph works. More recent additions (xychart-beta, sankey-beta) require a newer mmdc version than what ships with npm without explicit pinning. The solution is to add @mermaid-js/mermaid-cli@^11 to the workflow's npm install step rather than relying on the default version.

Preview depends on local tooling. The analytics feedback loop tells me which slides hold attention. slides.py does take --spec FILE and --demo, both of which call render_spec() and write PNGs straight to --outdir with no TTS, no ffmpeg, no commit and no CI — so previewing a slide locally is a one-liner. The catch is that the diagram and chart paths still need mmdc and matplotlib installed on the machine doing the preview; without them the local render degrades to the same "[visual unavailable]" card CI would produce, and the only place I see the real thing is the CI build. The single CI pipeline takes about 4 minutes end-to-end for a 20-segment video, which is reasonable for final validation but too slow for visual design iteration.

FAQ

Does mmdc need Node.js pre-installed on the runner?
Yes. The ubuntu-latest runner ships with Node.js, so the workflow just needs npm install -g @mermaid-js/mermaid-cli in a setup step.

Can I use Mermaid's dark theme instead of base with custom variables?
No. The dark theme resets most themeVariables to its own palette. The only theme that lets you fully override colors via themeVariables is base.

What happens if Openverse returns zero results for a query?
fetch_image() raises RuntimeError("no usable Openverse image for query: ..."). slide_image() catches it and leaves the branded heading card as it was — no placeholder text, no build failure.

Why horizontal bar charts instead of vertical?
Longer tool or model names truncate or require rotation on vertical axes. Horizontal bars handle 20-40 character labels without layout code.

Is the PNG-to-Pillow paste lossless?
Pillow opens the PNG fully decoded at 24-bit color depth. The save to final PNG at the end of slide composition uses default Pillow compression (level 6), which is lossless. There is no quality loss from the round trip.

Related: YouTube slide renderer — eight kinds, no browser — Two-host video pipeline with edge-tts and ffmpeg — Free neural TTS options for CI pipelines

Part of an ongoing 6-month experiment running three AI-curated directory sites. The technical claims here are real; this article was AI-assisted.