How to Validate JSON-LD Schema Markup in a CI Pipeline

Structured data bugs are boring to debug manually and cheap to catch automatically, which makes them a good candidate for a CI check that nobody has to remember to run. Here's a working setup: fetch rendered HTML for a set of representative pages, extract the JSON-LD blocks, and run a handful of checks that catch the mistakes that actually show up in production.

Why this needs rendered HTML, not template source

If any part of your JSON-LD generation happens client-side, or your templating engine does conditional logic before final output, the raw template file in your repo won't match what a crawler actually receives. The check has to run against rendered output, either from a headless browser or from your server's actual HTML response, not from source files that might diverge from production behavior in ways nobody's tracking.

from playwright.sync_api import sync_playwright

def get_rendered_html(url):
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page()
        page.goto(url, wait_until="networkidle")
        html = page.content()
        browser.close()
        return html

For server-rendered pages you can skip the headless browser entirely and just fetch the URL directly, which is faster and simpler when your stack doesn't depend on client-side JSON-LD injection.

Extracting the blocks

import re
import json

def extract_json_ld(html):
    pattern = r'<script type="application/ld\+json">(.*?)</script>'
    blocks = re.findall(pattern, html, re.DOTALL)
    parsed = []
    for raw in blocks:
        try:
            parsed.append(json.loads(raw))
        except json.JSONDecodeError as e:
            raise AssertionError(f"Invalid JSON-LD syntax: {e}")
    return parsed

This is the cheapest check in the whole pipeline and it should run first, because a syntax error makes every downstream check meaningless. Fail fast here rather than trying to validate a schema you can't even parse.

Checking @type casing against a known-good list

Schema.org types are case sensitive, and this is one of the more common bugs that produces zero errors anywhere in a typical pipeline while still silently breaking eligibility for a rich result.

VALID_TYPES = {
    "Product", "Article", "FAQPage", "Organization",
    "AggregateRating", "Review", "Offer", "BreadcrumbList",
    "WebSite", "Person",
}

def check_type_casing(obj, path="root"):
    errors = []
    if isinstance(obj, dict):
        t = obj.get("@type")
        if t and t not in VALID_TYPES and t.lower() in {v.lower() for v in VALID_TYPES}:
            errors.append(f"{path}: '@type' casing mismatch: got '{t}'")
        for key, value in obj.items():
            errors.extend(check_type_casing(value, f"{path}.{key}"))
    elif isinstance(obj, list):
        for i, item in enumerate(obj):
            errors.extend(check_type_casing(item, f"{path}[{i}]"))
    return errors

Extend VALID_TYPES to whatever schema.org types your templates actually emit. The point isn't exhaustive coverage of every schema.org type in existence, it's catching the specific casing drift that happens when someone copies an example from an outdated tutorial.

Catching leaked template placeholders

def check_for_placeholders(obj):
    text = json.dumps(obj)
    suspicious_patterns = ["{{", "}}", "%%", "${", "<%"]
    return [p for p in suspicious_patterns if p in text]

This one catches a specific, embarrassing failure mode: a templating variable that never got substituted, shipped to production, sitting in your JSON-LD as literal {{product_name}} text. It's syntactically valid JSON and it passes every schema-shape check while being completely wrong.

Verifying nested types have their own @type

NESTED_TYPE_FIELDS = {"aggregateRating", "author", "offers", "review", "publisher"}

def check_nested_types(obj, path="root"):
    errors = []
    if isinstance(obj, dict):
        for key, value in obj.items():
            if key in NESTED_TYPE_FIELDS and isinstance(value, dict):
                if "@type" not in value:
                    errors.append(f"{path}.{key}: nested object missing @type")
            errors.extend(check_nested_types(value, f"{path}.{key}"))
    return errors

Missing @type on a nested object is a quiet failure. The parent object still validates, the JSON is still well-formed, and the nested entity just doesn't exist as far as any schema-aware consumer is concerned.

Wiring it into CI

name: structured-data-check
on: [pull_request]
jobs:
  validate-schema:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run structured data checks
        run: python scripts/check_structured_data.py --pages fixtures/template_urls.txt

Keep fixtures/template_urls.txt as a short, maintained list of one representative URL per template, product page, article, homepage, category page, rather than trying to crawl the entire site on every PR. If you're new to JSON-LD as a format, it's worth a quick read before extending this script, since the syntax has a few quirks around @context and @graph that aren't obvious from schema examples alone. The goal is fast feedback on template changes, not a full-site audit running on every commit.

What this deliberately doesn't check

This setup validates structure and syntax, not the harder question of whether structured data matches visible page content, which is a real requirement and one of the more common causes of rich results getting suppressed. That check needs either a human reviewer or a more involved DOM-comparison script that's worth building once your basic structural checks are solid and stable. Start with syntax and type-casing, since those catch the bulk of accidental regressions for the least implementation effort, then layer content-matching checks on top once the team trusts the simpler pipeline.

Why this is worth the setup cost

The honest pitch for building this is that structured data bugs are almost uniquely bad at announcing themselves. A broken button gets reported by a user within hours. A broken API integration throws an error in your logs. A broken JSON-LD block just silently stops earning a rich result, and the only signal you get is a slow decline in a metric nobody's watching closely enough to notice the exact day it started dropping. By the time someone connects a click-through rate dip to a missing star rating, you're often looking at weeks of lost visibility that a thirty-second CI check would have caught on the pull request that introduced it.

The setup cost here is genuinely small relative to that risk. Most of the code above is under a hundred lines total, runs in a few seconds per page, and doesn't require any new infrastructure beyond whatever CI runner you're already using for the rest of your test suite. The hardest part isn't the code, it's remembering to maintain the fixture list of representative URLs as new templates get added, which is a five-minute task each time rather than an ongoing burden.

Common objections and why they don't hold up

The usual pushback is "we don't have that many pages with structured data, it's not worth automating." That's usually true for the initial build, and false for what happens over the following year. Templates get copied, extended, and modified by different people over time, and each modification is a chance for one of the mistakes above to slip in unnoticed. A check that takes five minutes to write once and runs automatically forever is a better investment than trusting the same manual vigilance to hold indefinitely across every future contributor who touches a template.

The other common objection is that a headless browser adds too much CI overhead. If your JSON-LD is fully server-rendered, which is true for a lot of stacks, you can skip Playwright entirely and just fetch the raw HTML response with a plain HTTP request, cutting both the dependency and the runtime significantly. Only reach for a headless browser if you actually have client-side schema injection to account for.

Keeping it maintainable

Resist the urge to build an exhaustive schema.org type checker. VALID_TYPES only needs the types your site actually emits, not the entire vocabulary. A smaller, accurate list that matches your real templates catches real regressions faster than a comprehensive one that's harder to keep in sync as your schema usage evolves.

We went deeper into the categories of structured data bugs that pass basic validation but still fail rich results, plus how to read Search Console's notoriously vague error reports when something does slip through, in our full writeup at 137Foundry. For the canonical property requirements per type when you're deciding what belongs in a fixture, schema.org's documentation is the source of truth, and Google's Rich Results tool is still worth a manual spot-check before merging any template change that touches structured data.