Your sitemap is an API — a whole website as one JSON document in 35 lines

Every time I want to give a model a site rather than a page, I end up writing the same throwaway script. Fetch the sitemap. Loop the URLs. Pull the title, the H1, the meta description. Strip the HTML down to something readable. Glue it into one file. Delete the script. Write it again three weeks later.

So I made it a URL instead:

https://hyperlambda.dev/website-to-json — paste a domain, get one JSON document back. Free, no signup, no key.

What comes back

One array, one object per page:

[
  {
    "url": "https://example.com/pricing",
    "h1": "Pricing",
    "title": "Pricing | Example",
    "description": "One flat price, no surprises.",
    "markdown": "# Pricing\n\nOne flat price…"
  }
]

That shape is the whole point. markdown is the part you feed a model; url, h1, title and description are the parts you filter, dedupe and cite with. A folder of 50 Markdown files makes you rebuild that metadata yourself. One JSON document doesn't.

Useful immediately for: seeding a RAG index, diffing a site before and after a migration, auditing which pages are missing a meta description, or handing a competitor's docs to a model without writing a crawler.

The entire backend

Not a snippet of it. This is the deployed file:

.arguments
   url:string

// Throttling to maximum 5 requests per IP per minute!
execution.throttle.create:website-to-json
   limit:int:5
   window:int:60
   per:ip
execution.throttle:website-to-json

strings.concat
   get-value:x:@.arguments/*/url
   .:/sitemap.xml
http.get:x:-
xml2lambda:x:-/*/content
.result
for-each:x:@xml2lambda/*/urlset/*/url/*/loc/*/#text/[0,50]
   http.get:x:@.dp/#
   html2lambda:x:-/*/content
   .h1
   strings.join:x:@html2lambda/*/html/*/body/**/h1/[0,1]/**/#text
      .:" "
   set-value:x:@.h1
      strings.trim:x:@strings.join
   .title
   set-value:x:@.title
      get-value:x:@html2lambda/*/html/*/head/*/title/*/#text
   .description
   set-value:x:@.description
      get-value:x:"@html2lambda/*/html/*/head/*/meta/*/\\@name/=description/./*/\\@content"
   lambda2html:x:@html2lambda/*/html/*/body
   html2markdown:x:@lambda2html
      url:x:@.arguments/*/url
   unwrap:x:+/*/*/*
   add:x:@.result
      .
         .
            url:x:@.dp/#
            h1:x:@.h1
            title:x:@.title
            description:x:@.description
            markdown:x:@html2markdown
return-nodes:x:@.result/*

No dependencies, no build step, no project scaffold. That file is the deployment — saving it publishes the endpoint.

The parts worth stealing

This is Hyperlambda, where code is a tree rather than text, and the interesting bits are the expressions that walk it.

The sitemap is parsed, not regexed. xml2lambda turns the XML into nodes, and then the loop selects every URL in one expression:

for-each:x:@xml2lambda/*/urlset/*/url/*/loc/*/#text/[0,50]

Read it right to left: every #text under every loc under every url in the urlset — then [0,50] slices the first fifty. The page limit isn't an if with a counter; it's part of the address of the data.

Attribute matching without a DOM library. Pulling <meta name="description" content="…"> is one expression:

@html2lambda/*/html/*/head/*/meta/*/\@name/=description/./*/\@content

Find the @name attribute whose value equals description, step back up to its element with ., then take that element's @content. Attributes are \@-prefixed nodes, so they're addressable the same way elements are.

H1 text is joined, not assumed. An <h1> containing a <span> and a stray <br> has several text nodes. **/#text collects all of them under the first H1 and strings.join puts them back together with a space — so <h1>Build <span>fast</span></h1> yields Build fast, not Build.

Relative links survive. The body is serialized back to HTML and converted with the original domain passed in:

html2markdown:x:@lambda2html
   url:x:@.arguments/*/url

Without that, every /pricing in the output would be a dead relative link the moment the JSON left the site it came from.

What it will not do

Worth stating plainly, because a crawler that pretends otherwise wastes your afternoon:

• It needs a sitemap. The URL you pass gets /sitemap.xml appended, so pass the origin with no trailing slash (https://example.com). No sitemap, no output. Sites that keep a sitemap index rather than a flat urlset won't enumerate either.
• It does not run JavaScript. Server-rendered HTML only. A client-rendered app returns its shell, exactly like curl would.
• Fifty pages, hard. Fine for a docs section or a marketing site; not a crawler for your 40,000-page store.
• Five requests per minute per IP. It's a free shared endpoint doing 50 HTTP fetches per call. Hammer it and you'll just get throttled — the source is above, run your own.

It also costs the target site 50 requests, so point it at your own property or something that welcomes it.

The part I actually find interesting

The file above was written from a plain-English description by a code generator, and every function in it was verified against the functions that actually exist on the server before it was saved. Not "the model probably got the API right" — checked, then deployed, with no restart.

That inverts the usual trade-off with generated backend code. The risk isn't that the model writes ugly code, it's that it writes code that references something imaginary and fails at 3am. If the runtime refuses to save anything that invokes a function it doesn't have, the failure happens at generation time instead.

Same property makes it an agent tool rather than a web page: on this stack every HTTP endpoint is also an MCP tool, role-filtered, so an agent can call website-to-json and get the same array without a browser in the loop.

Try it

• The tool: https://hyperlambda.dev/website-to-json
• More free ones — page to JSON, dead link checker, link preview checker, llms.txt generator: https://hyperlambda.dev/tools
• Run the whole platform yourself, MIT-licensed:

curl -fsSL https://hyperlambda.dev/docker-compose.yaml | docker compose -f - up

If you build something with the output, I'd like to hear what you pointed it at.