How to Extract and Check JSON-LD Structured Data with Python (Standard Library Only)

Structured data is the small block of JSON most sites put in their HTML to tell search engines what a page is: an article, a product, an organization, a breadcrumb trail. When it breaks, nothing on the page looks wrong. The page renders, the content is there, and the only symptom is that a rich result quietly stops appearing or a search tool reports an error weeks later.

Google's own Rich Results Test is the right tool for checking one page in depth. But if you want to check 200 pages after a template change, or run a quick check in CI before a deploy, it helps to have a script you can run from the terminal. This post builds one in about 170 lines of Python using only the standard library: no pip install, no BeautifulSoup, no requests.

The script:

• fetches a page (or reads a local HTML file)
• pulls out every <script type="application/ld+json"> block
• reports blocks that are not valid JSON, with the line and column
• lists every @type it finds, including nested ones and items inside @graph
• flags missing properties for a few common types, based on Google's documentation
• accepts a list of URLs from a file, so you can check a whole sitemap

Google documentation details are as of 4 Nov 2026, from Google Search Central's structured data pages. Appie tool descriptions are from appie.run/tools on the same day.

Disclosure: Appie.run publishes this article. Two free tools mentioned near the end are ours.

What Google actually says about JSON-LD

A few points from Google Search Central's introduction to structured data and its general guidelines shape what the script does:

• Google supports JSON-LD, Microdata and RDFa, and recommends JSON-LD.
• Google can read JSON-LD that JavaScript injects into the page after load. Our script reads the HTML as the server sends it, so it will not see injected markup. That is a real limit, and the script says so when it finds nothing.
• Required properties must be present for a page to be eligible for a rich result. Recommended properties are optional but can help.
• Google's documentation, not schema.org, is the reference for what Google uses. schema.org defines far more types and properties than Google supports.
• Valid markup does not guarantee a rich result.
• Do not mark up content that is not visible on the page.

The "required" versus "recommended" split varies by type. A few examples from Google's type pages:

• Article, BlogPosting, NewsArticle: Required by Google: None; Recommended by Google: headline, image, datePublished, dateModified, author
• Organization: Required by Google: None; Recommended by Google: Many, including name, url, logo, sameAs
• Product (product snippet): Required by Google: name, plus one of review, aggregateRating or offers; Recommended by Google: Several
• BreadcrumbList: Required by Google: itemListElement; each ListItem needs position, name, and item (the last item may omit item); Recommended by Google: None listed

The script encodes exactly these and nothing more. You can extend the CHECKS dictionary for the types you use.

The script

Save this as jsonld_audit.py. It needs Python 3.8 or later; we tested it on Python 3.13.

#!/usr/bin/env python3
"""Extract JSON-LD blocks from web pages and flag common problems.

Standard library only. Usage:
    python3 jsonld_audit.py https://example.com/page [more URLs or local .html files]
    python3 jsonld_audit.py @urls.txt      (one URL or file path per line)
"""
import json
import sys
import urllib.request
from html.parser import HTMLParser

# Checks based on Google Search Central docs (read 4 Nov 2026).
# "required" means Google lists it as required for that rich result;
# "recommended" means Google lists it as recommended.
CHECKS = {
    "Article":        {"required": [], "recommended": ["headline", "image", "datePublished", "dateModified", "author"]},
    "NewsArticle":    {"required": [], "recommended": ["headline", "image", "datePublished", "dateModified", "author"]},
    "BlogPosting":    {"required": [], "recommended": ["headline", "image", "datePublished", "dateModified", "author"]},
    "Organization":   {"required": [], "recommended": ["name", "url", "logo", "sameAs"]},
    "Product":        {"required": ["name"], "one_of": ["offers", "review", "aggregateRating"]},
    "BreadcrumbList": {"required": ["itemListElement"]},
}


class JSONLDParser(HTMLParser):
    """Collects the text inside every <script type="application/ld+json">."""

    def __init__(self):
        super().__init__(convert_charrefs=True)
        self.blocks = []
        self._in_ld = False
        self._buf = []

    def handle_starttag(self, tag, attrs):
        if tag == "script":
            kind = (dict(attrs).get("type") or "").split(";")[0].strip().lower()
            if kind == "application/ld+json":
                self._in_ld = True
                self._buf = []

    def handle_data(self, data):
        if self._in_ld:
            self._buf.append(data)

    def handle_endtag(self, tag):
        if tag == "script" and self._in_ld:
            self.blocks.append("".join(self._buf))
            self._in_ld = False


def load(source):
    if source.startswith(("http://", "https://")):
        req = urllib.request.Request(source, headers={"User-Agent": "jsonld-audit/1.0"})
        with urllib.request.urlopen(req, timeout=20) as resp:
            charset = resp.headers.get_content_charset() or "utf-8"
            return resp.read().decode(charset, errors="replace")
    with open(source, encoding="utf-8", errors="replace") as fh:
        return fh.read()


def iter_items(node):
    """Yield every dict that has an @type, walking @graph, lists and nesting."""
    if isinstance(node, list):
        for item in node:
            yield from iter_items(item)
    elif isinstance(node, dict):
        if "@type" in node:
            yield node
        for key, value in node.items():
            if key != "@type":
                yield from iter_items(value)


def top_level(data):
    """The items a block declares directly: a list, an @graph, or one object."""
    if isinstance(data, list):
        return [d for d in data if isinstance(d, dict)]
    if isinstance(data, dict) and isinstance(data.get("@graph"), list):
        return [d for d in data["@graph"] if isinstance(d, dict)]
    return [data] if isinstance(data, dict) else []


def types_of(item):
    t = item.get("@type")
    return t if isinstance(t, list) else [t]


def empty(value):
    return value in (None, "", [], {})


def check_item(item):
    problems = []
    for t in types_of(item):
        rule = CHECKS.get(t)
        if not rule:
            continue
        for prop in rule.get("required", []):
            if empty(item.get(prop)):
                problems.append(f"{t}: missing required '{prop}'")
        one_of = rule.get("one_of")
        if one_of and all(empty(item.get(p)) for p in one_of):
            problems.append(f"{t}: needs one of {', '.join(one_of)}")
        for prop in rule.get("recommended", []):
            if empty(item.get(prop)):
                problems.append(f"{t}: no recommended '{prop}'")
        if t == "BreadcrumbList":
            elements = item.get("itemListElement") or []
            for i, li in enumerate(elements, 1):
                if not isinstance(li, dict):
                    continue
                if empty(li.get("position")):
                    problems.append(f"BreadcrumbList item {i}: missing 'position'")
                if empty(li.get("name")) and not isinstance(li.get("item"), dict):
                    problems.append(f"BreadcrumbList item {i}: missing 'name'")
                if empty(li.get("item")) and i < len(elements):
                    problems.append(f"BreadcrumbList item {i}: missing 'item' (only the last may omit it)")
    return problems


def audit(source):
    print(f"\n== {source}")
    try:
        html = load(source)
    except Exception as exc:  # network errors, 404s, missing files
        print(f"  could not load: {exc}")
        return 1
    parser = JSONLDParser()
    parser.feed(html)
    if not parser.blocks:
        print("  no JSON-LD found in the HTML as served (JavaScript-injected markup is not seen)")
        return 1
    issues = 0
    for n, raw in enumerate(parser.blocks, 1):
        try:
            data = json.loads(raw)
        except json.JSONDecodeError as exc:
            print(f"  block {n}: invalid JSON ({exc.msg} at line {exc.lineno}, col {exc.colno})")
            issues += 1
            continue
        items = list(iter_items(data))
        names = sorted({str(t) for item in items for t in types_of(item)})
        print(f"  block {n}: {len(items)} item(s): {', '.join(names) or 'no @type'}")
        if isinstance(data, dict) and "@context" not in data and "@graph" not in data:
            print(f"    warning: no @context")
        for item in top_level(data):
            for p in check_item(item):
                print(f"    {p}")
                issues += 1
    return 1 if issues else 0


if __name__ == "__main__":
    if len(sys.argv) < 2:
        sys.exit(__doc__)
    sources = []
    for arg in sys.argv[1:]:
        if arg.startswith("@"):
            with open(arg[1:], encoding="utf-8") as fh:
                sources += [line.strip() for line in fh if line.strip() and not line.startswith("#")]
        else:
            sources.append(arg)
    results = [audit(src) for src in sources]
    sys.exit(1 if any(results) else 0)

How it works

Finding the blocks. html.parser.HTMLParser walks the document tag by tag. When it sees a <script> tag whose type is application/ld+json, it starts collecting text until the closing </script>. The split(";") handles the occasional type="application/ld+json; charset=utf-8". This is more reliable than a regular expression, which tends to break on attributes in a different order or on extra whitespace.

Parsing. Each block goes through json.loads. When that fails, the JSONDecodeError gives a message and a position. Trailing commas, unescaped quotes inside strings, and template tags that rendered as empty values are the usual causes. A broken block is skipped entirely by parsers, so this is the most important thing the script catches.

Listing types. iter_items walks the parsed JSON recursively and yields any object with an @type. That picks up nested items such as an author that is a Person or an offers that is an Offer, and it handles @graph arrays, which many CMS plugins use to put several items in one block.

Checking. top_level returns only the items a block declares directly (the object itself, a list, or the members of @graph). The checks run on those, not on every nested object. That keeps the output quiet: a publisher Organization nested inside an Article is not expected to carry every recommended Organization property.

Exit code. The script exits with 1 if any page could not be loaded, had no JSON-LD, or had at least one issue. That makes it easy to use in CI.

Try it on a test page

Here is a deliberately flawed file, sample.html. It has a blog post without an image or modified date, a correct breadcrumb, a Product block with a trailing comma, and a Product with no offers, review or rating:

<html><head>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"BlogPosting","headline":"How we price","datePublished":"2026-11-01","author":{"@type":"Person","name":"RG"}}
</script>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[
 {"@type":"ListItem","position":1,"name":"Blog","item":"https://example.com/blog"},
 {"@type":"ListItem","position":2,"name":"How we price"}]}
</script>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"Product","name":"Starter kit",}
</script>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"Product","name":"Pro kit"}
</script>
</head><body></body></html>

Running python3 jsonld_audit.py sample.html printed this on our machine:

== sample.html
  block 1: 2 item(s): BlogPosting, Person
    BlogPosting: no recommended 'image'
    BlogPosting: no recommended 'dateModified'
  block 2: 3 item(s): BreadcrumbList, ListItem
  block 3: invalid JSON (Illegal trailing comma before end of object at line 2, col 72)
  block 4: 1 item(s): Product
    Product: needs one of offers, review, aggregateRating

Each problem maps to something you would fix in the template: add image and dateModified to the blog post, remove the trailing comma, and either add offers to the second product or stop marking it up as a product.

Try it on live pages

We ran it against three public pages on 4 Nov 2026:

== https://appie.run/
  block 1: 6 item(s): Offer, Organization, SoftwareApplication, WebSite

== https://www.python.org/
  block 1: 2 item(s): SearchAction, WebSite

== https://example.com/
  no JSON-LD found in the HTML as served (JavaScript-injected markup is not seen)

The first two pages have valid JSON-LD with no flagged issues for the types the script checks. The third has no JSON-LD in its served HTML, which is expected for a placeholder domain. On your own site, "no JSON-LD found" on a page you expected to have markup means one of two things: the markup is missing, or it is injected by JavaScript. Open the page in Google's Rich Results Test to tell which.

Check a whole site from its sitemap

Put one URL per line in a text file and pass it with @:

python3 jsonld_audit.py @urls.txt > jsonld-report.txt

To build urls.txt from a sitemap, you can reuse the parser from our earlier post on validating an XML sitemap with Python, or paste the sitemap into a free extractor (more on that below). For large sites, sample one or two URLs per template (home, category, product, article) rather than every page. Structured data is almost always generated by templates, so if one product page is wrong, they usually all are.

Be polite when you run it against a site you do not own: add a short time.sleep() between requests and keep the list small.

Running it in CI

Because the script exits non-zero on problems, you can run it against a staging build before deploy:

python3 jsonld_audit.py https://staging.example.com/ https://staging.example.com/blog/sample-post/ || exit 1

If "no recommended property" warnings are too noisy for a blocking check, split them out: return 1 only for invalid JSON and missing required properties, and print the rest as warnings.

Ideas for extending it

• More types. Add the types you use to CHECKS, taking required and recommended properties from Google's page for each type, not from schema.org.
• Date format. Check that datePublished and dateModified parse with datetime.fromisoformat.
• URL checks. Confirm that url, image and breadcrumb item values are absolute URLs on the expected host.
• Visible content. Compare headline against the page's <h1> or <title> to catch markup that has drifted from what users see, which Google's guidelines warn against.
• CSV output. Write one row per page and issue with the csv module for a spreadsheet-friendly report.

Tools that help around the script

The script covers the checking. A couple of free tools help on either side of it.

Google's Rich Results Test and Search Console. The Rich Results Test is the authority on whether a page is eligible for a specific rich result, and it renders JavaScript, so it sees injected markup. Search Console reports structured data issues across your site over time.

Appie Sitemap URL Extractor (ours, free). Paste sitemap XML or enter a sitemap URL and it returns one URL per line, which is the format @urls.txt expects. If you give it a sitemap index, it lists the child sitemaps; run it again on a child sitemap to get page URLs. It is at appie.run/tools/sitemap-url-extractor.

Appie Webpage to FAQ Generator (ours, free). If your audit shows pages with thin content and you want to add an FAQ section, this tool reads a single public page and writes FAQs from its text, skipping claims the page does not make. Review the result before you publish it. We covered how FAQ content fits AI answers now that Google has changed how FAQ rich results appear in this earlier post.

FAQ

Why not use BeautifulSoup or extruct?
They are good libraries. This script avoids dependencies so it runs anywhere Python does, including minimal CI images, with nothing to install.

Does valid JSON-LD guarantee a rich result?
No. Google's guidelines say structured data makes a page eligible but does not guarantee a rich result.

Why does the script say no JSON-LD on a page that has it?
The markup is probably added by JavaScript after the page loads. Google can read that, but this script reads only the HTML the server returns. Check with the Rich Results Test.

Should I follow schema.org or Google's docs?
For Google features, Google's docs. schema.org defines the vocabulary; Google documents which parts it uses and what it requires.

Does it support Microdata or RDFa?
No, only JSON-LD, which is the format Google recommends.

Can I run it on hundreds of URLs?
Yes, with the @urls.txt option. Add a delay between requests and sample by template where you can.

Wrap-up

Structured data breaks silently, usually because of a template change, and a small script catches most of it before a search tool does. If you want the sitemap URL list without writing more code, the Appie free tools include a sitemap URL extractor, and Appie's apps start on a Free plan with 500 credits and no credit card.

RG, Founder, Appie.run