How to Extract & Simplify a Webpage DOM

Building AI agents or AI-powered web automations, working with raw webpages can be difficult. Most HTML pages include useless elements that can become problematic when you are trying to extract meaningful data or feed clean content into an LLM. These elements include things like <style> tags, linked stylesheets, <script> blocks, tracking pixels, SVG icons, and empty or hidden nodes. These add weight to the raw DOM, increasing token usage and making LLM hallucinations more likely.

Gaffa uses a real browser to render pages by default, we call it headful as the opposite to headless which is the default with most other scraping or browser as a service products. This means the page loads exactly like it would on an actual device with JavaScript fully rendered and the DOM including all dynamic content, essential for scraping modern web apps.

Gaffa's Browser Request API has two actions that help you work with the DOM for a webpage, capture_dom and generate_simplified_dom.

• capture_dom lets you capture the entire DOM of a webpage, including JavaScript-loaded content, which you can then process in your existing codebase or data extraction workflow
• generate_simplified_dom distils that raw HTML down to just the useful parts, removing unnecessary tags, styles, scripts, and other clutter, leaving you with a clean, structured DOM ready for further processing

Simplifying the DOM is especially useful as it can reduce HTML size by 50 to 80 percent on some pages, saving thousands of tokens when feeding content into an LLM, without losing essential context. It's equally useful as input for further processing or data extraction in your scripts, such as extracting elements with Python and BeautifulSoup.

Using capture_dom to retrieve raw HTML

The capture_dom action lets you retrieve the complete Document Object Model (DOM) of a web page. This includes not just the static HTML served by the server, but also any JavaScript-generated content.

To retrieve raw HTML with fully rendered JavaScript from a web page using Gaffa, you send a POST request to the /v1/browser/requests endpoint with the following JSON payload.

Extracting the rendered DOM from a web page

{
  "url": "https://www.freecodecamp.org/news/how-to-contribute-to-open-source-handbook/",
  "proxy_location": "us",
  "async": false,
  "max_cache_age": 0,
  "settings": {
    "record_request": false,
    "actions": [
      {
        "type": "capture_dom",
        "continue_on_fail": false
      }
    ]
  }
}

As a side note, Gaffa makes it easy to rotate IPs or geolocate requests using residential proxies. Just set the proxy_location field to your desired region, like "us", "de", or "sg".

This will return a .txt file containing the raw HTML DOM of the current page at the time the request is made, with this response.

Raw DOM response

{
    "data": {
        "id": "brq_VKi21MiGEKM6VMv5RKhXV1JQJychNr",
        "url": "https://www.freecodecamp.org/news/how-to-contribute-to-open-source-handbook/",
        "state": "completed",
        "credit_usage": 1,
        "http_status_code": 204,
        "from_cache": false,
        "started_at": "2025-06-24T07:58:09.4820891Z",
        "completed_at": "2025-06-24T07:58:19.8187492Z",
        "running_time": "00:00:09.4322002",
        "page_load_time": "00:00:04.8159109",
        "actions": [
            {
                "id": "act_VKi21LnenxQN2hoZs6hx1Li8X2ExUU",
                "type": "capture_dom",
                "query": "capture_dom?continue_on_fail=false",
                "timestamp": "2025-06-24T07:58:14.2980107Z",
                "output": "https://storage.gaffa.dev/brq/dom/brq_VKi21MiGEKM6VMv5RKhXV1JQJychNr/act_VKi21LnenxQN2hoZs6hx1Li8X2ExUU_raw.txt"
            }
        ]
    }
}

Getting the raw DOM means we are getting a lot of content including scripts, nav bars, stylesheets, sidebars, ads, and other elements that don't contribute to the actual content, and if sent to an LLM this will lead to extra token usage and potentially harm performance.

That's where the generate_simplified_dom action comes in.

Using generate_simplified_dom to clean the DOM

Using the generate_simplified_dom action, we can clean and distill the HTML down to just the essential parts. It removes elements that typically add unnecessary noise to the HTML without containing any meaningful information for later data extraction, helping you focus on what's valuable.

The JSON payload stays the same, the only change is to update the action type to match the current action.

Simplify DOM code

{
  "actions": [
    {
      "type": "generate_simplified_dom"
    }
  ]
}

This will output a cleaner version of the page's DOM to a .txt file, stripped of scripts, stylesheets, inline styles, and SVG paths. This reduction means fewer tokens used when feeding the data into an LLM, ideal for summarization, Q&A, or fine-tuning tasks. It could also be useful if training a model on the HTML structure of different sites.

Capturing and simplifying the DOM with Gaffa makes it easier to work with and extract data from fully rendered webpages. Depending on the task, capture_dom gives you the complete HTML structure of the rendered page, while generate_simplified_dom reduces it to a cleaner, more focused version that's easier to process.

These actions can also serve as building blocks for more advanced scraping and automation workflows.

• Explore the full list of Gaffa actions in the official documentation
• Experiment with pre-built requests in the API Playground and adapt them to your own use case

Frequently asked questions

How do I capture the full rendered HTML of a page using Gaffa?

Use the capture_dom action. It returns the complete DOM including JavaScript-loaded content, exactly as it appears in a real browser, not just the initial static HTML.

How do I get a clean, stripped-down version of a page's HTML?

Use the generate_simplified_dom action. It removes scripts, stylesheets, SVGs, tracking pixels, and empty nodes, leaving only the content that matters for extraction or LLM input.

Why does raw HTML cause problems when fed into an LLM?

Raw pages contain scripts, tracking pixels, and inline styles that add no useful context. This inflates token usage and increases the likelihood of LLM hallucinations.

How much can generate_simplified_dom reduce HTML size?

It can reduce HTML size by 50 to 80 percent on some pages, saving thousands of tokens without losing the essential content needed for extraction or LLM processing.

Does Gaffa render JavaScript before capturing the DOM?

Yes. Gaffa uses a headful browser by default, meaning pages load exactly as they would on a real device, with JavaScript fully executed and dynamic content included.

Can I use capture_dom output with my own parsing tools, such as BeautifulSoup?

Yes. Both capture_dom and generate_simplified_dom return a .txt file containing the HTML, which you can fetch and process with any library in your existing workflow.

How do I route requests through a specific country with Gaffa?

Set the proxy_location field in your request payload to your desired region, such as "us", "de", or "sg", and Gaffa routes the request through a residential proxy in that location.